Closed makortel closed 1 week ago
cms-bot internal usage
A new Issue was created by @makortel.
@smuzaffar, @makortel, @rappoccio, @antoniovilela, @sextonkennedy, @Dr15Jones can you please review it and eventually sign/assign? Thanks.
cms-bot commands are listed here
assign RecoPPS/Local
New categories assigned: reconstruction
@jfernan2,@mandrenguyen you have been requested to review this Pull request/Issue and eventually sign? Thanks
@cms-sw/ctpps-dpg-l2 The test fails because the streamer format was changed. Does this test really need to read a streamer file, or could it read a root file?
type ctpps
Could someone look into this? (or should we look into disabling the test?)
Could someone look into this? (or should we look into disabling the test?)
I've investigated the test a bit. It seems that we can convert the input streamer file into RAW ROOT format and run the problematic test on it. If everything goes well with the conversion I'll put converted file on CERNBOX and make the PR with fixes (once the file is available in the area where testing machinery can access it).
@makortel I've put the converted RAW file here: https://cernbox.cern.ch/s/OD136wlNpQfH4X8 , would it be possible to upload it somewhere where the Jenkins can access it ?
Thanks @grzanka! To me it would make sense to place the new file where the present file is, whose location is defined in https://github.com/cms-data/RecoPPS-Local , but I don't know how to get the file in http://cmsrep.cern.ch/cmssw/download . @smuzaffar could you help?
@grzanka , if the new file is less than 50MB then I would suggest to open a PR and add it to https://github.com/cms-data/RecoPPS-Local . If it ia over 50MB then please copy it to your afs public directory
@grzanka , if the new file is less than 50MB then I would suggest to open a PR and add it to https://github.com/cms-data/RecoPPS-Local . If it ia over 50MB then please copy it to your afs public directory
The file is ~77MB, I've copied it to /afs/cern.ch/user/g/grzankal/public
thanks @grzanka , so do we need to keep both
run364983_ls0001_streamA_StorageManager.dat
run364983_ls0001_raw.root
files in RecoPPS/Local or just the new one is enough?
By the way, we should not use http://cmsrep.cern.ch/cmssw/download/data/RecoPPS/Local
to download this file. This file is path of data-RecoPPS-Local external and should be accessed/opened using FileInPath RecoPPS/Local/data/run364983_ls0001_streamA_StorageManager.dat
or RecoPPS/Local/data/run364983_ls0001_raw.root
.
thanks @grzanka , so do we need to keep both
- existing
run364983_ls0001_streamA_StorageManager.dat
- new
run364983_ls0001_raw.root
files in RecoPPS/Local or just the new one is enough?
By the way, we should not use
http://cmsrep.cern.ch/cmssw/download/data/RecoPPS/Local
to download this file. This file is path of data-RecoPPS-Local external and should be accessed/opened using FileInPathRecoPPS/Local/data/run364983_ls0001_streamA_StorageManager.dat
orRecoPPS/Local/data/run364983_ls0001_raw.root
.
For the moment we would prefer to keep both files. Thank you for the hints with the paths, I will now prepare a PR with fix
By the way, we should not use
http://cmsrep.cern.ch/cmssw/download/data/RecoPPS/Local
to download this file. This file is path of data-RecoPPS-Local external and should be accessed/opened using FileInPathRecoPPS/Local/data/run364983_ls0001_streamA_StorageManager.dat
orRecoPPS/Local/data/run364983_ls0001_raw.root
.
How to use FileInPath
in context of cms.Source('PoolSource'
? The code below doesn't make much sense....
process.source = cms.Source('PoolSource',
fileNames = cms.untracked.vstring(
cms.untracked.FileInPath('RecoPPS/Local/data/run364983_ls0001_streamA_StorageManager.dat'),
)
)
How to use
FileInPath
in context ofcms.Source('PoolSource'
? The code below doesn't make much sense....
You indeed can't. However, there is a utility that resolves the search in python and returns vstring
, see
https://github.com/cms-sw/cmssw/blob/a9682e8ca89d4942c6676bf8c5f0d2c6a70abb0e/FWCore/Modules/test/testBunchCrossingFilter.py#L27
https://github.com/cms-sw/cmssw/blob/a9682e8ca89d4942c6676bf8c5f0d2c6a70abb0e/FWCore/Modules/test/testBunchCrossingFilter.py#L42-L44
for an example
https://github.com/cms-data/RecoPPS-Local/pull/2 is now open, cmssw PR should use this for testing
For the moment we would prefer to keep both files.
If the old file would be removed, the unit test would start to fail in old release cycles. Which reminds me, the fix in the unit test needs to be backported to 14_0_X too (or merging https://github.com/cms-sw/cmssw/pull/44978 will cause the test to fail there)
I meant, do we need to drop the old file from data-RecoPPS-Local package? i.e. it will not be removed from the cmsrep.cern.ch download area but from the package we distribute on cvmfs. Old releases will keep one accessing it from cmsrep
I meant, do we need to drop the old file from data-RecoPPS-Local package? i.e. it will not be removed from the cmsrep.cern.ch download area but from the package we distribute on cvmfs. Old releases will keep one accessing it from cmsrep
Ah, then I'd be fine with removing the old file from data-RecoPPS-Local package. Do I understand correctly the benefit would be that the new releases would no longer distribute the old file (that would be unnecessary there).
yes, new tag/version of data-RecoPPS-Local external will only have one file (run364983_ls0001_raw.root
) .
I've made the PR with necessary fixes: https://github.com/cms-sw/cmssw/pull/45144 Unfortunately I wasn't able to make full test locally:
[grzankal@lxplus952 test]$ cmsRun totemT2NewDigi_reco_cfg.py
----- Begin Fatal Exception 05-Jun-2024 16:26:09 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
[0] Processing the python configuration file named totemT2NewDigi_reco_cfg.py
Exception Message:
unknown python problem occurred.
OSError: No such file or directory 'RecoPPS/Local/data/run364983_ls0001_raw.root' in the CMSSW_SEARCH_PATH
At:
/cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-06-02-2300/src/FWCore/ParameterSet/python/pfnInPath.py(10): pfnInPath
/cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-06-02-2300/src/FWCore/ParameterSet/python/pfnInPath.py(16): <genexpr>
----- End Fatal Exception -------------------------------------------------
I suspect that would require a new release or the PR being merged. Can you run necessary Jenkins tests on the PR ?
Jenkins tests are running for https://github.com/cms-sw/cmssw/pull/45144 . In order to test it locally you can just copy run364983_ls0001_raw.root
file in your <CMSSW-dev-area>/src/RecoPPS/Local/data/
path and run the test.
Jenkins tests are running for #45144 . In order to test it locally you can just copy
run364983_ls0001_raw.root
file in your<CMSSW-dev-area>/src/RecoPPS/Local/data/
path and run the test.
Thanks, it worked !
It seems the tests on https://github.com/cms-sw/cmssw/pull/45144 succeeded, what would be the next step ?
It will be reviewed by corresponding L1 (reconstruction in this case), then by ORPs, which will then merge it.
Will we need the backport to 14_0_X? The streamer file change will be merged soon in 14_0_X. Remember to make the cmsdist backport, to point to the updated cms-data tag.
Will we need the backport to 14_0_X? The streamer file change will be merged soon in 14_0_X. Remember to make the cmsdist backport, to point to the updated cms-data tag.
As requested I've made a backport https://github.com/cms-sw/cmssw/pull/45192
+1 Addressed by https://github.com/cms-sw/cmssw/pull/45192
This issue is fully signed and ready to be closed.
The unit test
RecoPPSLocalNewT2
fails in CMSSW_14_1_X_2024-05-30-1100 ashttps://cmssdt.cern.ch/SDT/cgi-bin/logreader/el8_amd64_gcc12/CMSSW_14_1_X_2024-05-30-1100/unitTestLogs/RecoPPS/Local#/
(noted also in https://github.com/cms-sw/cmssw/pull/44892#issuecomment-2139443543)
The failure is a consequence of the merge of https://github.com/cms-sw/cmssw/pull/44892, but in this case it is the test that needs an update.