Open waredjeb opened 8 months ago
cmsbuild
commented
8 months ago A new Issue was created by @waredjeb Wahid Redjeb.
@Dr15Jones, @rappoccio, @sextonkennedy, @smuzaffar, @makortel, @antoniovilela can you please review it and eventually sign/assign? Thanks.
cms-bot commands are listed here
felicepantaleo
commented
8 months ago @bsunanda fyi
mmusich
commented
8 months ago Certainly the GlobalTag here
does not look right.
makortel
commented
8 months ago assign geometry, visualization
cmsbuild
commented
8 months ago New categories assigned: geometry,visualization
@Dr15Jones,@Dr15Jones,@alja,@civanch,@bsunanda,@makortel,@makortel,@mdhildreth you have been requested to review this Pull request/Issue and eventually sign? Thanks
antoniovilela
commented
8 months ago Marco's comment, to be followed up: https://github.com/cms-sw/cmssw/pull/43098#issuecomment-1777091606
waredjeb
commented
1 week ago Hello! I am reviving this issue because I am still having some problem in dumping the geometry.
Release = CMSSW_14_1_0_pre4
cmsRun dumpRecoGeometry_cfg.py tag=2026 version=D110
[wredjeb@olivb-27 python]$ cmsRun dumpRecoGeometry_cfg.py tag=2026 version=D110
Loading configuration for tag 2026 ...
Dumping geometry in cmsRecoGeom-2026.root
Begin processing the 1st record. Run 1, Event 1, LumiSection 1 on stream 0 at 20-Jun-2024 09:45:40.738 CEST
----- Begin Fatal Exception 20-Jun-2024 09:46:08 CEST-----------------------
An exception of category 'GeometryMismatch' occurred while
[0] Processing Event run: 1 lumi: 1 event: 1 stream: 0
[1] Running path 'p'
[2] Prefetching for module DumpFWRecoGeometry/'dump'
[3] Prefetching for EventSetup module FWRecoGeometryESProducer/''
[4] Prefetching for EventSetup module GlobalTrackingGeometryESProducer/''
[5] Calling method for EventSetup module TrackerDigiGeometryESModule/'trackerGeometry'
Exception Message:
Size mismatch between geometry (size=43708) and alignments (size=43492)
----- End Fatal Exception -------------------------------------------------This time, the GlobalTag looks good to me! The problem seems to come from the Tracker, nevertheless disabling Tracker (and also MTD) doesn't help.
@rovere FYI
cmsbuild
commented
1 week ago cms-bot internal usage
mmusich
commented
1 week ago @waredjeb Hi Wahid,
This time, the GlobalTag looks good to me!
unfortunately it's not.
D110 = T35+C18+M11+I17+O9+F8
so you need auto:phase2_realitisc_T33 (don't ask ... T35 ~ T33) because the alignment in auto:phase2_realistic is still the one for T25:
$ getAutoCond -k phase2_realistic
theRelease: CMSSW_14_1_0_pre4
phase2_realistic : 140X_mcRun4_realistic_v3and
$ conddb list 140X_mcRun4_realistic_v3 | grep TrackerAlignment
[2024-06-20 10:20:55,856] INFO: Connecting to pro [frontier://PromptProd/cms_conditions]
TrackerAlignmentErrorExtendedRcd - TrackerAlignmentErrorsExtended_Upgrade2026_T25_design_v0
TrackerAlignmentRcd - TrackerAlignment_Upgrade2026_T25_design_v0 @cms-sw/alca-l2 @cms-sw/upgrade-l2 FYI.
waredjeb
commented
1 week ago @waredjeb Hi Wahid,
This time, the GlobalTag looks good to me!
unfortunately it's not.
D110 = T35+C18+M11+I17+O9+F8
so you need
auto:phase2_realitisc_T33(don't ask ... T35 ~ T33) because the alignment inauto:phase2_realisticis still the one for T25:$ getAutoCond -k phase2_realistic theRelease: CMSSW_14_1_0_pre4 phase2_realistic : 140X_mcRun4_realistic_v3and
$ conddb list 140X_mcRun4_realistic_v3 | grep TrackerAlignment [2024-06-20 10:20:55,856] INFO: Connecting to pro [frontier://PromptProd/cms_conditions] TrackerAlignmentErrorExtendedRcd - TrackerAlignmentErrorsExtended_Upgrade2026_T25_design_v0 TrackerAlignmentRcd - TrackerAlignment_Upgrade2026_T25_design_v0@cms-sw/alca-l2 @cms-sw/upgrade-l2 FYI.
Hi Marco,
I see, thank you very much! By replacing: https://github.com/cms-sw/cmssw/blob/master/Fireworks/Geometry/python/dumpRecoGeometry_cfg.py#L81-L82 With
from Configuration.AlCa.GlobalTag import GlobalTag
process.GlobalTag = GlobalTag(process.GlobalTag, 'auto:phase2_realistic_T33', '')Seems to work, not sure if this Exception Message is concerning:
[wredjeb@olivb-27 python]$ cmsRun dumpRecoGeometry_cfg.py tag=2026 version=D110
Loading configuration for tag 2026 ...
Dumping geometry in cmsRecoGeom-2026.root
Begin processing the 1st record. Run 1, Event 1, LumiSection 1 on stream 0 at 20-Jun-2024 10:59:50.808 CEST
%MSG-e FWRecoGeometry: FWRecoGeometryESProducer:FWRecoGeometryESProducer@callESModule 20-Jun-2024 11:00:18 CEST Run: 1 Event: 1
ME0 geometry not found An exception of category 'NoGeometry' occurred.
Exception Message:
No Tracking Geometry is available for DetId 704643072
%MSGSeems to work, not sure if this Exception Message is concerning:
I don't know either, but seems to concern ME0 so the Muon system (not sure who needs to be tagged about it).
waredjeb
commented
1 week ago Ok, including the Tracker raises another problem:
[wredjeb@olivb-27 python]$ cmsRun dumpRecoGeometry_cfg.py tag=2026 version=D110 tracker=True
Loading configuration for tag 2026 ...
Dumping geometry in cmsRecoGeom-2026.root
Begin processing the 1st record. Run 1, Event 1, LumiSection 1 on stream 0 at 20-Jun-2024 11:13:22.083 CEST
----- Begin Fatal Exception 20-Jun-2024 11:13:49 CEST-----------------------
An exception of category 'WrongTrackerSubDet' occurred while
[0] Processing Event run: 1 lumi: 1 event: 1 stream: 0
[1] Running path 'p'
[2] Prefetching for module DumpFWRecoGeometry/'dump'
[3] Calling method for EventSetup module FWRecoGeometryESProducer/''
Exception Message:
Invalid DetID: no GeomDetUnit associated with raw ID 303042564 of subdet ID 1
----- End Fatal Exception -------------------------------------------------Invalid DetID: no GeomDetUnit associated with raw ID 303042564 of subdet ID 1
this looks more concerning and might have to do with the sensor splitting in TBPix layer 1 (subdet ID 1). @emiglior @cms-sw/trk-dpg-l2
perrotta
commented
1 week ago Curious, the GT for the T33 tracker geometry was ready since some while, and I was convinced it had been already ported to autoCond. Evidently, it was not: thank you @mmusich for noticing it.
Now PR https://github.com/cms-sw/cmssw/pull/45274 updates it in autoCond
mmusich
commented
1 week ago so this:
diff --git a/Fireworks/Geometry/python/dumpRecoGeometry_cfg.py b/Fireworks/Geometry/python/dumpRecoGeometry_cfg.py
index 73c80aa320d..f975d372dd5 100644
--- a/Fireworks/Geometry/python/dumpRecoGeometry_cfg.py
+++ b/Fireworks/Geometry/python/dumpRecoGeometry_cfg.py
@@ -43,7 +43,7 @@ def versionCheck(ver):
print("")
help()
-def recoGeoLoad(score):
+def recoGeoLoad(score, properties):
print("Loading configuration for tag ", options.tag ,"...\n")
if score == "Run1":
@@ -78,8 +78,29 @@ def recoGeoLoad(score):
elif "2026" in score:
versionCheck(options.version)
process.load("Configuration.StandardSequences.FrontierConditions_GlobalTag_cff")
- from Configuration.AlCa.autoCond import autoCond
- process.GlobalTag.globaltag = autoCond['phase2_realistic']
+
+ # Import the required configuration from the CMSSW module
+ from Configuration.AlCa.autoCond import autoCond # Ensure autoCond is imported
+
+ # Ensure options.version is defined and set correctly
+ version_key = '2026' + options.version # This constructs the key for accessing the properties dictionary
+ print(f"Constructed version key: {version_key}")
+
+ # Check if the key exists in properties for 2026
+ if version_key in properties[2026]:
+ # Get the specific global tag for this version
+ global_tag_key = properties[2026][version_key]['GT']
+ print(f"Global tag key from properties: {global_tag_key}")
+
+ # Check if this key exists in autoCond
+ if global_tag_key.replace("auto:", "") in autoCond:
+ # Set the global tag
+ from Configuration.AlCa.GlobalTag import GlobalTag
+ process.GlobalTag = GlobalTag(process.GlobalTag, global_tag_key, '')
+ else:
+ raise KeyError(f"Global tag key '{global_tag_key}' not found in autoCond.")
+ else:
+ raise KeyError(f"Version key '{version_key}' not found in properties[2026].")
process.load('Configuration.Geometry.GeometryExtended2026'+options.version+'Reco_cff')
elif score == "MaPSA":
@@ -114,11 +135,8 @@ def recoGeoLoad(score):
help()
-
-
options = VarParsing.VarParsing ()
-
defaultOutputFileName="cmsRecoGeom.root"
options.register ('tag',
@@ -171,16 +189,33 @@ options.register ('out',
options.parseArguments()
-
-
-
-process = cms.Process("DUMP")
+from Configuration.PyReleaseValidation.upgradeWorkflowComponents import upgradeProperties as properties
+# Determine version_key based on the value of options.tag
+if options.tag == "2026" or options.tag == "MaPSA":
+ prop_key = 2026
+ version_key = options.tag + options.version
+elif options.tag == "2017" or options.tag == "2021":
+ prop_key = 2017
+ version_key = options.tag
+else:
+ prop_key = None
+ version_key = None
+
+if(prop_key and version_key):
+ print(f"Constructed version key: {version_key}")
+ era_key = properties[prop_key][str(version_key)]['Era']
+ print(f"Constructed era key: {era_key}")
+ from Configuration.StandardSequences.Eras import eras
+ era = getattr(eras, era_key)
+ process = cms.Process("DUMP",era)
+else:
+ process = cms.Process("DUMP")
process.add_(cms.Service("InitRootHandlers", ResetRootErrHandler = cms.untracked.bool(False)))
process.source = cms.Source("EmptySource")
process.maxEvents = cms.untracked.PSet(input = cms.untracked.int32(1))
-recoGeoLoad(options.tag)
+recoGeoLoad(options.tag,properties)
if ( options.tgeo == True):
if (options.out == defaultOutputFileName )allows for a slightly better automatic treatement of GlobalTags and eras in the configuration. Unfortunately it does not solve neither the warnings nor the crash for Tracker in D110.
mmusich
commented
1 week ago Unfortunately it does not solve neither the warnings nor the crash for Tracker in D110.
concerning the crash in the Tracker part, it looks it's due this line:
in particular at the very beginning of the DetId loop in BPix (presumably Layer1 but I didn't explicitly cross-check), this call
m_trackerGeom->idToDetUnit(detid)
fails on detid 303042564 while it works on the "other" companion detid 303042565, so it looks like the detUnit mapping is not working correctly in this setup.
felicepantaleo
commented
1 week ago hi, is there any way this dumper can be run as a test automatically for every PR? Or at least for any PR that introduces a new version of the geometry?
mmusich
commented
1 week ago hi, is there any way this dumper can be run as a test automatically for every PR? Or at least for any PR that introduces a new version of the geometry?
sure there is, this is what I was suggesting: https://github.com/cms-sw/cmssw/pull/43098#issuecomment-1777091606
alja
commented
1 week ago @makortel Do I need to open an issue to add unit tests?
( PS: I'm on vacation starting tomorrow until Tuesday)
makortel
commented
1 week ago @makortel Do I need to open an issue to add unit tests?
( PS: I'm on vacation starting tomorrow until Tuesday)
I'm not sure I understand your question. Unit tests can be added in just a PR. Having a unit test run in PR tests of an unrelated package (*) is generally unresolved problem (for which we might already have an issue open). In any case @smuzaffar should be able to help further.
(*) I didn't check if the package defining the geometries, and the straightforward place for the unit test are related in any way or not (being on vacation myself)
smuzaffar
commented
5 days ago @alja , as @makortel wrote, you can open the PR to add the unit test in one of the existing packages. About running the new unit test for all PR might be over killed. Is there any way to find if a PR is changing geometry? If yes then we can teach bot to checkout the package which contains this unit test Or we can also teach bot to check out the package container new unit tests if PR requires geometry approval ( this will allow the new unit test to run for such PRs)
alja
commented
3 days ago @smuzaffar I don't know how the PR in some other package can trigger unit tests in Fireworks/Geometry. Now I'm not even sure if reco geometry and sim geometry dumper are supposed to be in Fireworks module. I also don't know if more cmssw modules can cause the crashing of Fireworks reco geometry dumper. Should I post the question at cms mattermost?
felicepantaleo
commented
2 days ago As workaround, we are removing the exception here: https://github.com/cms-sw/cmssw/blob/64d4f3b087ce5832f186bc811757c17e16fb9ddd/Geometry/TrackerGeometryBuilder/src/TrackerGeometry.cc#L183-L191
and skipping the tracker detIds that give nullptr. We will keep this workaround private until @cms-sw/trk-dpg-l2 fixes the map.
mmusich
commented
2 days ago sure there is, this is what I was suggesting: https://github.com/cms-sw/cmssw/pull/43098#issuecomment-1777091606
so I packaged two (would-be) improvements in https://github.com/cms-sw/cmssw/pull/45328:
dumpRecoGeometry that currently fails (because of the reasons above). This at least keeps the problem on the radar.
I am having some problems in dumping a new Fireworks Geometry.
Release =
CMSSW_13_3_0_pre3Command:
cmsRun dumpRecoGeometry_cfg.py tag=2026 version=D98Error:I am seeing the same thing in
CMSSW_13_2_0_pre3