Closed makortel closed 3 years ago
assign reconstruction
New categories assigned: reconstruction
@slava77,@perrotta,@jpata you have been requested to review this Pull request/Issue and eventually sign? Thanks
A new Issue was created by @makortel Matti Kortelainen.
@Dr15Jones, @dpiparo, @silviodonato, @smuzaffar, @makortel, @qliphy can you please review it and eventually sign/assign? Thanks.
cms-bot commands are listed here
From the set of PRs merged since the previous ASAN builds (that did not have this error) https://github.com/cms-sw/cmssw/pull/32052 could be the cause (the only one with "CSC" in its name, and it touches code in the same package.
@nvoytish
please take a look
to reproduce you will need to setup CMSSW_11_2_ASAN_X_2020-11-23-2300 IB
and then there runTheMatrix.py -l 4.37 --ibeos
@nvoytish please take a look to reproduce you will need to setup CMSSW_11_2_ASAN_X_2020-11-23-2300 IB and then there
runTheMatrix.py -l 4.37 --ibeos
@ptcox
Can you clarify what we are looking for? Does this mean some dynamically assigned memory has been overwritten, or can it mean some simpler thing like a misaligned variable? I do not know what ASAN tests exactly. But the issue might be easier to track down if it could just be some value passed in to findXOnStrip is invalid. (That would seem more likely since the recent updates to the code can affect those.)
The complaint is about a read (or write) beyond an allocated memory block. After a quick peek into the code I'd guess one of these array accesses goes out of bounds https://github.com/cms-sw/cmssw/blob/dd492f74e6c2e9a9dadaa5990245803ec0e76d45/RecoLocalMuon/CSCRecHitD/src/CSCXonStrip_MatchGatti.cc#L549-L570
So the problem could well be
if it could just be some value passed in to findXOnStrip is invalid.
@ptcox @nvoytish please clarify on the status of resolving this. we should avoid an option of having to revert the last CSC changes.
Hi @slava77 ! Unfortunately I was too busy last week. I am on it.
@ptcox I managed to find what causes the problem. In a particular event the strip from one chamber (ME11A) is combined with a wire that it physically can not intersect (last wg from ME11B). This causes a mess in Gatti correction calculation. I'll try to understand which change in code lead to this. Sorry for the slow progress!
That’s good progress! And something like I was expecting. Maybe something I missed in the intrinsic traps in the new code, and we still need some sort of ‘inside’ check.
On Dec 2, 2020, at 11:31, nvoytish notifications@github.com wrote:
@ptcox I managed to find what causes the problem. In a particular event the strip from one chamber (ME11A) is combined with a wire that it physically can not intersect (last wg from ME11B). This causes a mess in Gatti correction calculation. I'll try to understand which change in code lead to this. Sorry for the slow progress!
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.
I'll keep digging. You were right, when you said that Gatti corrections are "a pain")
Hi all!
We are trying to come up with a right fix for the bug. I was working on it today, but couple hours ago after a successful compile I tried to run "cmsenv" command and got: [nvoytish@lxplus723 src]$ cmsenv Unable to find SCRAM version V2_2_9_pre11. Something wrong with the sw installation.
and [nvoytish@lxplus723 ~]$ scram list CMSSW_11_2_ASAN* SCRAM warning: >>>> No SCRAM project CMSSW version CMSSW_11_2_ASAN_X_2020-11-23-2300 available. <<<< You can run "scram list CMSSW" to see the available versions. gives no results. Was this release temporary and is not available any more? Is there another one that will reproduce the failure?
Was this release temporary and is not available any more? Is there another one that will reproduce the failure?
IBs stay for about 2 weeks; it looks like 11-23 has expired. CMSSW_11_3_ASAN_X_2020-12-04-2300 is a more recent build The same problem in CSC is still there.
Hi all!
It turned out that we need to keep the omitted "if" https://github.com/cms-sw/cmssw/commit/7501f0ca9d85f3f5f2e6f20ba2743b1794a48e24#diff-89c0bedb1d5ff71760f4a99102f4c06386cdac66bb8527527d41a7fe28e5dab8L160 We forgot that there is a difference in detid for wires in simulation and real data. For real data all wires are labelled as ME11B. This "if" does not allow for a strip from ME11A to be assembled with a wire that geometrically is outside the ME11A sensitive area. Do I understand correctly, that I need to make a new PR for this change?
Do I understand correctly, that I need to make a new PR for this change?
Yes, thanks!
+1
I do not find the CSC-related error in https://cmssdt.cern.ch/SDT/html/cmssdt-ib/#/relVal/CMSSW_11_3/2020-12-16-2300?selectedArchs=slc7_amd64_gcc900&selectedFlavors=ASAN_X&selectedStatus=failed anymore (after #32442 was merged on Dec 15)
This issue is fully signed and ready to be closed.
Workflow 4.37 step 3 fails in CMSSW_11_2_ASAN_X_2020-11-23-2300 with