search

cms-sw / cmssw

CMS Offline Software
http://cms-sw.github.io/
Apache License 2.0
1.09k stars 4.33k forks source link

Use `TriggerResultsFilterFromDB` instead of `HLTHighLevel`, in order to allow prescaled HLT paths in `EcalESAlign` and `HcalCalIterativePhiSym` #46742

Open mmusich opened 1 day ago

mmusich commented 1 day ago

PR description:

EcalESAlign and HcalCalIterativePhiSym have been reported by Tier-0 experts to be resource intensive in terms of output data volume when run over the Raw Prime PDs during 2024 data-taking. This lead to PR https://github.com/dmwm/T0/pull/5025 where these producers are removed from the from Raw Prime PDs to reduce output volume. To avoid having to manually change again Tier0 configuration in the future, I propose this PR to allow prescales in AlCaRecoTriggerBits for the two mentioned producers. This follows the same pattern used for https://github.com/cms-sw/cmssw/pull/42965.

PR validation:

runTheMatrix.py -l 1000.0 -t 4 -j 8 runs fine.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Not a backport, not to be backported.

cmsbuild commented 1 day ago

cms-bot internal usage

cmsbuild commented 1 day ago

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46742/42716

cmsbuild commented 1 day ago

A new Pull Request was created by @mmusich for master.

It involves the following packages:

@atpathak, @cmsbuild, @consuegs, @perrotta can you please review it and eventually sign? Thanks. @ReyerBand, @argiro, @bsunanda, @mmusich, @rchatter, @rsreds, @thomreis, @tocheng, @wang0jin, @yuanchao this is something you requested to watch as well. @antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

mmusich commented 1 day ago

@cmsbuild, please test

cmsbuild commented 1 day ago

+1

Size: This PR adds an extra 20KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f64d27/42970/summary.html COMMIT: 383f9c45bc8b582a8abac66e282eb7fa4401eb8f CMSSW: CMSSW_14_2_X_2024-11-19-2300/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46742/42970/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

mmusich commented 1 day ago

hold

cmsbuild commented 1 day ago

Pull request has been put on hold by @mmusich They need to issue an unhold command to remove the hold state or L1 can unhold it for all

mmusich commented 1 day ago

not sure why this happens, but it's not good.

perrotta commented 1 day ago

@mmusich there are differences in the step2 of wf 1000.0, in the accepted paths:

image

Moreover, in the log of that step2 this new message sorts out (among the sea of other log messages that make hard to retrieve the relevant new one):

%MSG-w Configuration:  TriggerResultsFilterFromDB:hcalphisymHLT  20-Nov-2024 12:57:55 CET Run: 165121 Event: 23609118
Empty trigger results expression

Is that understood?

mmusich commented 1 day ago

Is that understood?

@perrotta https://github.com/cms-sw/cmssw/pull/46742#issuecomment-2488821542

mmusich commented 1 day ago

@perrotta OK this happens for the following reason. Wf 1000.0 runs on 165121 (Run1):

1000.0 RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT [1]: input from: /MinimumBias/Run2011A-v1/RAW with run [165121] 
                                           [2]: cmsDriver.py step2  --conditions auto:run1_data -s RAW2DIGI,L1Reco,RECO,ALCAPRODUCER:@allForPrompt,DQM:@allForPrompt,ENDJOB --datatier RECO,AOD,ALCARECO,DQMIO --eventcontent RECO,AOD,ALCARECO,DQM --data  --process RECO --scenario pp --customise Configuration/DataProcessing/RecoTLR.customisePrompt -n 100 
                                           [3]: cmsDriver.py step3  -s SKIM:all --conditions auto:run1_data --data  --scenario pp --filein file:step2.root --secondfilein filelist:step1_dasquery.log -n 100 
                                           [4]: cmsDriver.py step4  -s HARVESTING:@standardDQMFakeHLT+@miniAODDQM --conditions auto:run1_data --data  --filetype DQM --scenario pp --filein file:step2_inDQM.root -n 100 
                                           [5]: cmsDriver.py step5  -s ALCAOUTPUT:@allForPrompt --conditions auto:run1_data --scenario pp --data  --triggerResultsProcess RECO --filein file:step2_inALCARECO.root -n 100 

1 workflows with 5 steps

currently auto:run1_data points to 141X_dataRun2_v2 which has in it AlCaRecoHLTpaths_2017_offline_v6. Dumping that tag with:

 cmsRun AlCaRecoTriggerBitsRcdRead_TEMPL_cfg.py inputTag=AlCaRecoHLTpaths_2017_offline_v6

I see in the resulting file for the IOV of interest:

---+++++ *IOV*: 165100-165969
| *TriggerBits list key* | *HLT paths* |

[...]

| 'HcalCalIterativePhiSym' |  |

this is of course not good. I suggest you fix the tag (and Global Tag).

mmusich commented 1 day ago

unhold