search

cms-sw / cmssw

CMS Offline Software
http://cms-sw.github.io/
Apache License 2.0
1.08k stars 4.3k forks source link

Migrate CMSSW from HepMC2 to HepMC3 #36662

Open mkirsano opened 2 years ago

mkirsano commented 2 years ago

HepMC is a HEP event record used in CMSSW. HepMC2 is currently used. HepMC3 is a completely rewritten version. Many methods are different from HepMC2. These two records can live together in one program. I have a partly working prototype (working at ~10% functionality, pythia8 only) in my private area. The evolving plan follows

  1. Cope with library structure. By default the HepMC3 libraries structures are lib64/libHepMC3.so->libHepMC3.so.3. CMSSW does not understand it, I have to copy this structure in CMSSW_12_3_0_pre2/external/slc7_amd64_gcc10/lib by hands. - done
  2. Rewrite code created for ExternalDecays. Seems still to be needed for the photos++ package (tauola++?)
  3. Create GenEventInfoProduct3 class similarly to GenEventInfoProduct. Some methods need rewriting. - done privately
  4. Create HepMC3Product similarly to HepMCProduct - done privately
  5. Rewrite the code where the HepMC record is fed in G4. Seems to be here (Siewyan):https://github.com/cms-sw/cmssw/blob/master/SimG4Core/Generators/src/Generator.cc#L112. Problem: SimG4Core compiled locally does not work. In contact with V. Ivanchenko
  6. At some point a special CMSSW branch will be needed?
cmsbuild commented 2 years ago

A new Issue was created by @mkirsano Mikhail Kirsanov.

@Dr15Jones, @perrotta, @dpiparo, @makortel, @smuzaffar, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

mkirsano commented 2 years ago

assign generators

cmsbuild commented 2 years ago

New categories assigned: generators

@mkirsano,@alberto-sanchez,@SiewYan,@GurpreetSinghChahal,@Saptaparna,@agrohsje you have been requested to review this Pull request/Issue and eventually sign? Thanks

mkirsano commented 2 years ago

assign simulation

cmsbuild commented 2 years ago

New categories assigned: simulation

@mdhildreth,@civanch you have been requested to review this Pull request/Issue and eventually sign? Thanks

mkirsano commented 2 years ago

I solved the problem with library structure, preparing PR to add external

mkirsano commented 2 years ago

PR cmsdist #7655

mkirsano commented 2 years ago

PR to include hepmc3 merged

mkirsano commented 2 years ago

I found that it is impossible to include both pythia8 - HepMC2 and pythia8 - HepMC3 converters in the same code. To have a possibility to switch I will have to create another plugin: Pythia8HepMC3Hadroniser

civanch commented 2 years ago

@mkirsano , do you make a survey of all places inside CMSSW, which require a change? Likely it will be the same situation - new alternative code will be required.

mkirsano commented 2 years ago

No, this concerns only generator -> HepMC conversion. In other codes, such as analysis, using "if" can be sufficient.

mkirsano commented 2 years ago

This is a preliminary list of packages concerned. In addition, there is a class HepMCCandidate, but it seems to be internal in CMSSW (does not depend on hepmc) listhepmc.txt

smuzaffar commented 2 years ago

@mkirsano , note that many externals are build using hepmc (https://github.com/cms-sw/cmsdist/search?q=hepmc ) including pythia8 . so if you use pythia8 then you automatically get hepmc.

mkirsano commented 2 years ago

I made a prototype of Pythia8Interface that can produce either HepMC2 or HepMC3 record. Does it have sense to make a PR?

civanch commented 2 years ago

I think if by default it would implement HepMC2 it would be fine.

mkirsano commented 2 years ago

I will probably need help with adding HepMC3 products to edm. The instruction here https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideCreatingNewProducts is rather old, probably not up-to-date

makortel commented 2 years ago

I will probably need help with adding HepMC3 products to edm. The instruction here https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideCreatingNewProducts is rather old, probably not up-to-date

That twiki page is pretty much up to date. But if you find something there which is not (or suspect or are unsure), let us know.

civanch commented 2 years ago

@mkirsano , HepMC3 migration was discussed at the today ORP meeting. @smuzaffar , explain us that several GEN packages should be compiled versus HepMC3. This can be achieved normally if a new git BRANCH will be created. So, the plan may be following:

1) we use a switch HepM2/HepMC3 as a python parameter defined somewhere upstream 2) development is ongoing within master, depending on the switch HepMC2 or HepMC3 is called inside CMSSW 3) validation will mean comparison of the new branch results versus master results

The link to the new product is correct (from my point of view). What is essential that depending on the switch producers will consume HepMC2 or HepMC3

smuzaffar commented 2 years ago

The externals packages which uses HepMC are https://github.com/cms-sw/cmsdist/search?q=hepmc

We still need to see if all of these can build/run with new HepMC3 :-)

SiewYan commented 2 years ago

@smuzaffar, @mkirsano , i can help to check on those.

mkirsano commented 2 years ago

All these packages can be compiled against HepMC3. I tried photospp, tauolapp. Another question is if they can be compiled against both. But the critical path is the integration of HepMC3Product in edm. Then we can start to look at all the chain (G4, analysis, validation etc.)

mkirsano commented 2 years ago

I am ready for the next PRs, adding HepMC3 products to GeneratorInterface/Core BaseHadronizer and adding Pythia8HepMC3Hadronizer plugin. Work on adding HepMC3 products to edm is to be forced.

mkirsano commented 2 years ago

GeneratorInterface/Core : PR #37689

mkirsano commented 2 years ago

GeneratorInterface/Pythia8Interface : PR #37753 Depends on #37689

SiewYan commented 2 years ago

To reiterate the plan:

  1. [x] Cope with library structure. By default the HepMC3 libraries structures are lib64/libHepMC3.so->libHepMC3.so.3. CMSSW does not understand it, I have to copy this structure in CMSSW_12_3_0_pre2/external/slc7_amd64_gcc10/lib by hands.
  2. [ ] Rewrite code created for ExternalDecays. Seems still to be needed for the photos++ package (tauola++?)
  3. [x] Create GenEventInfoProduct3 class similarly to GenEventInfoProduct. Some methods need rewriting. PR: #37187
  4. [x] Create HepMC3Product similarly to HepMCProduct. PR: #37187
  5. [ ] Rewrite the code where the HepMC record is fed in G4. Seems to be here (Siewyan):https://github.com/cms-sw/cmssw/blob/master/SimG4Core/Generators/src/Generator.cc#L112. Problem: SimG4Core compiled locally does not work. In contact with V. Ivanchenko
  6. [ ] At some point a special CMSSW branch will be needed?

PR #38033 opened to address EDMProduct definition effort from #37187