Handle DB payloads for Phase 2 geometries via (meta-)GTs

kpedro88 commented 3 years ago

During the CMS week Simulation meeting, there was a discussion of using DB payloads for Phase 2 geometries to improve performance with DD4hep.

Previously, this idea had been disfavored because of the infeasibility of maintaining O(20) separate GT queues to handle all of the various Phase 2 geometry configurations.

However, to handle different tracker conditions for different subdetector versions, there are now "meta"-GTs in autoCondPhase2.py. This script starts from a baseline and then specifies different records for specific conditions for each tracker subdetector version.

This approach could be extended to create a "meta"-GT for each Phase 2 detector scenario with minimal maintenance required.

Possible development items:

[ ] Add functionality to the geometry script to generate DB payloads automatically or on demand
[ ] Make autoCondPhase2.py import the available geometries and their components directly from the geometry script (automate creation of meta-GT for each geometry)
[ ] Include a switch for upgrade workflows to swap between XML and DB geometries (development will still need to use XML)

attn: @civanch @ianna @bsunanda

kpedro88 commented 3 years ago

assign upgrade,geometry,alca

cmsbuild commented 3 years ago

New categories assigned: geometry,upgrade,alca

@Dr15Jones,@cvuosalo,@mdhildreth,@christopheralanwest,@tlampen,@pohsun,@yuanchao,@makortel,@ianna,@kpedro88,@civanch you have been requested to review this Pull request/Issue and eventually sign? Thanks

cmsbuild commented 3 years ago

A new Issue was created by @kpedro88 Kevin Pedro.

@Dr15Jones, @dpiparo, @silviodonato, @smuzaffar, @makortel, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

ianna commented 3 years ago

@kpedro88 and @bsunanda - we need to check that an obsolete HGCal schema was never uploaded to ProdDB accidentally. If that's the case, it can be safely deleted. Or, the class name can be reused to make it more generic as a set of std::vectors and std::maps without specific parameter names.

However, if it was uploaded, we have to make sure that the payload is not included in the GT and we will have to come with a different class name.

tvami commented 3 years ago

Hi, I'm just reading open AlCa pending issues. The Phase-2 conditions are using the meta-GT https://github.com/cms-sw/cmssw/blob/master/Configuration/AlCa/python/autoCondPhase2.py for each geometry, so is this issue ready to be closed? @cms-sw/upgrade-l2 @cms-sw/geometry-l2

srimanob commented 3 years ago

@tvami If I understand from the original message, the proposal is to extend this tracker way to other detectors. Do you know their current situation?

tvami commented 3 years ago

Hi @cvuosalo do you maybe have a better view of what the plans are related to this issue?

kpedro88 commented 3 years ago

The point of this issue is to use the meta-GT approach to enable the use of geometry DB files. Currently, all Phase 2 geometries are read from XML because it's prohibitive to maintain N separate GT queues.

srimanob commented 3 years ago

Ah, so the key point is that we can load DB geometries of all versions of all detectors and compose the GT. So, we have 1 queue only. Right? Yeah, this looks like a good way to move on, maybe for DD4hep.

tvami commented 2 years ago

@srimanob @bsunanda what is the timeline now for the Phase-2 DD4hep geometries?

bsunanda commented 2 years ago

Once we find one stable scenario for Phase2 description, we can think of either DDD or DD4Hep version to be loaded to DB. Currently only DDD versions for Phase2 geometries are running satisfactorily. We need to resolve the DD4hep issue and then reach a stable scenario. Both tracker and HGCal geometries are evolving and none of the Phase2 detectors have gone for "construction" phase as yet

From: Tamas Vami @.*** Sent: 27 July 2022 21:43 To: cms-sw/cmssw Cc: Sunanda Banerjee; Mention Subject: Re: [cms-sw/cmssw] Handle DB payloads for Phase 2 geometries via (meta-)GTs (#32357)

@srimanobhttps://github.com/srimanob @bsunandahttps://github.com/bsunanda what is the timeline now for the Phase-2 DD4hep geometries?

— Reply to this email directly, view it on GitHubhttps://github.com/cms-sw/cmssw/issues/32357#issuecomment-1197288659, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABGMZOW5XGPMSLRV2PWQ7I3VWGGPLANCNFSM4UJHORQA. You are receiving this because you were mentioned.Message ID: @.***>

tvami commented 1 year ago

+alca

I'm signing this as an agreement from the AlCa side, the action items depend on the geometry team, not on AlCa

cms-sw / cmssw

Handle DB payloads for Phase 2 geometries via (meta-)GTs #32357