search

cms-sw / cmssw

CMS Offline Software
http://cms-sw.github.io/
Apache License 2.0
1.08k stars 4.3k forks source link

features for bParking UL rereco and miniAOD in 10_6_X (10_6_26?) #33917

Closed slava77 closed 2 years ago

slava77 commented 3 years ago

This issue is to collect PRs/features for integration towards b-parking dataset rereco and miniAOD in 10_6_X

Deadline for integration is June 7, 2021 (PRs should be merged by this date)

RECO/AOD part:

miniAOD:

@cms-sw/xpog-l2 please check the list and clarify if more is needed/planned

slava77 commented 3 years ago

assign reconstruction,xpog

cmsbuild commented 3 years ago

New categories assigned: xpog,reconstruction

@slava77,@fgolf,@mariadalfonso,@gouskos,@perrotta,@jpata you have been requested to review this Pull request/Issue and eventually sign? Thanks

cmsbuild commented 3 years ago

A new Issue was created by @slava77 Slava Krutelyov.

@Dr15Jones, @dpiparo, @silviodonato, @smuzaffar, @makortel, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

slava77 commented 3 years ago

@rappoccio (tagging for PPD, I don't have Andreas' git at hand)

slava77 commented 3 years ago

from https://hypernews.cern.ch/HyperNews/CMS/get/recoDevelopment/1716/1.html code introduced in #33774 is discussed (potentially) as an addition in the MC dataset processing. @mtosi @vmariani

slava77 commented 3 years ago

updates in the issue description/list from earlier today:

mariadalfonso commented 3 years ago

There is the idea to keep the muonHits in AOD or a reduced collection in miniAOD. The idea is to enable some EXO analysis on the B-parking dataset. It was discussed at PC-coord may26 and PPD-coord june2 Marco Pieri will follow up with proponents

slava77 commented 3 years ago

There is the idea to keep the muonHits in AOD or a reduced collection in miniAOD. The idea is to enable some EXO analysis on the B-parking dataset. It was discussed at PC-coord may26 and PPD-coord june2 Marco Pieri will follow up with proponents

isn't this enabled by default in AOD? or is it about something other than what was added in #31350 in 10_6_17?

slava77 commented 3 years ago

There is the idea to keep the muonHits in AOD or a reduced collection in miniAOD. The idea is to enable some EXO analysis on the B-parking dataset. It was discussed at PC-coord may26 and PPD-coord june2 Marco Pieri will follow up with proponents

isn't this enabled by default in AOD? or is it about something other than what was added in #31350 in 10_6_17?

based on the discussion in the PPD today, this is different, to include all muon rechits 

• Muon system acts like a sampling calorimeter
• LLP decays produce a shower of particles
• RecHits used to form clusters and identify a signal

IIUC, there is no plan to keep AOD on disk for this processing. Are the analyses targeting this prepared to use miniAOD for parts other than muon hits?

rappoccio commented 3 years ago

Correct, we don't have any plans to keep this on disk, but if we do a future miniaod (v3) later, it would be available.

AdrianoDee commented 3 years ago

From the low pt tracks point of view, we should be fine. Thanks a lot for the (super fast) effort done for reviewing the code!

slava77 commented 3 years ago

current status of the list of features

Screen Shot 2021-06-04 at 7 22 36 AM

I added two PRs which are described as targeting nanoAOD (v9), but they also include modifications in the miniAOD products. Both a backport of #33817 and #33759 will need to appropriately disable the updates in miniAOD except for bParking | run2_miniAOD_devel to follow the no-change policy for the existing old miniAOD production(s).

slava77 commented 3 years ago

I updated the list with the following:

RECO/AOD part:

slava77 commented 3 years ago 
* NEED A PATCH TO CMSSW_10_2_16_UL with keep statements from #33996  in SimTracker/Configuration/python/SimTracker_EventContent_cff.py

the above came from the discussion in https://github.com/cms-sw/cmssw/pull/33996#issuecomment-858084961

@silviodonato @qliphy what is your preference to make this update? There is no branch for patches for CMSSW_10_2_16 ( = CMSSW_10_2_16_UL). Should the update be made to CMSSW_10_2_X? (The difference is more than a few files https://github.com/cms-sw/cmssw/compare/CMSSW_10_2_16_UL..CMSSW_10_2_X)

qliphy commented 3 years ago

probably we can make CMSSW_10_2_26_patch2 ? @silviodonato may comment more.

silviodonato commented 3 years ago

@slava77 once we have a backport of #33996 to CMSSW_10_2_X, we can build a special CMSSW_10_2_16_UL2 that is CMSSW_10_2_16_UL + backport of #33996. The file differences should be harmless, but I think it is more safe to avoid the big jump CMSSW_10_2_16_UL -> CMSSW_10_2_27_UL only for b-parking UL.

@smuzaffar what is the best way to build a CMSSW_10_2_16_UL2 from the technical point of view? Do we need a new branch CMSSW_10_2_16_UL_patchX or can we directly cherry-pick the commits that we want to add?

smuzaffar commented 3 years ago

@silviodonato , if CMSSW_10_2_16_UL2 should be CMSSW_10_2_16_UL + #33996 then we need a cmssw tag or branch with CMSSW_10_2_16_UL + #33996 which you can use as CMSSW_COMMIT in the build issue.

smuzaffar commented 3 years ago

and you also should use 

CMSDIST_COMMIT=REL/CMSSW_10_2_16_UL/slc7_amd64_gcc700
PRODUCTION_ARCHITECTURE:slc7_amd64_gcc700

in order to not get all the external merged in IB/CMSSW_10_2_X/gcc700 branch

slava77 commented 3 years ago

@silviodonato , if CMSSW_10_2_16_UL2 should be CMSSW_10_2_16_UL + #33996 then we need a cmssw tag or branch with CMSSW_10_2_16_UL + #33996 which you can use as CMSSW_COMMIT in the build issue.

I'm not familiar with this procedure, in the past we used PRs to make updates.

So, somebody will

silviodonato commented 3 years ago

I would say - first of all - to merge the 102X backport to the standard CMSSW_10_2_X.

Then, I've just created the branch CMSSW_10_2_16_UL_patchX, so you can make a PR towards CMSSW_10_2_16_UL_patchX and then we'll build the release

qliphy commented 3 years ago

@smuzaffar @silviodonato
Please check whether this is ok: https://github.com/cms-sw/cmssw/issues/34089

slava77 commented 3 years ago

@smuzaffar @silviodonato Please check whether this is ok:

34089

the tag looks correct: https://github.com/cms-sw/cmssw/compare/CMSSW_10_2_16_UL...CMSSW_10_2_16_UL2

qliphy commented 3 years ago

superseded by https://github.com/cms-sw/cmssw/issues/34090 thanks to @smuzaffar

qliphy commented 3 years ago

CMSSW_10_2_16_UL2 is now ready https://github.com/cms-sw/cmssw/releases/CMSSW_10_2_16_UL2

slava77 commented 3 years ago

Current status is now down to one PR:

The other incomplete item is " to define test workflows" could probably be done now outside of the cmssw source.

slava77 commented 3 years ago

it looks like all PRs are now merged

slava77 commented 2 years ago

+reconstruction

the production processing is well in progress . I guess this issue can be closed and if some bug fixes are needed, they can be tracked in a separate issue

slava77 commented 2 years ago

@cms-sw/pdmv-l2

this is connected with the MC setup discussion earlier today. When I closed the issue I thought that the MC production was done/in progress as well.