cernopendata / data-curation

Data ingestion and curation tools
GNU General Public License v2.0
18 stars 22 forks source link

CMS - test 2016 GT reading #206

Open katilp opened 1 year ago

katilp commented 1 year ago

(from #124)

Test the GT reading

Use https://github.com/cms-opendata-analyses/PFNanoProducerTool with data, change the input file and the GT in PFNano/pfnano_data_2016UL_OpenData.py:

    fileNames = cms.untracked.vstring('root://eospublic.cern.ch//eos/opendata/cms/Run2016G/JetHT/MINIAOD/UL2016_MiniAODv2-v2/130000/35017A26-8C9D-204D-92B6-3ABFBBD4ADF3.root'),
process.GlobalTag.connect = cms.string('sqlite_file:/cvmfs/cms-opendata-conddb.cern.ch/106X_dataRun2_v37.db')
process.GlobalTag.globaltag = '106X_dataRun2_v37'

and, eventually, redo for MC when some OD files get transferred, change the input file and the following in PFNano/pfnano_mc_2016UL_OpenData.py:

process.GlobalTag.connect = cms.string('sqlite_file:/cvmfs/cms-opendata-conddb.cern.ch/106X_mcRun2_asymptotic_v17.db')
process.GlobalTag.globaltag = '106X_mcRun2_asymptotic_v17'

Also, think of an automated test wf to be added as a test to the container image build

katilp commented 9 months ago

Testing in a new VM image from the cernvm group, with slc7 singularity shell.

The GT access is fine but the PFNano test job fails with

Did not find frontier-connect in calib-data section in /cvmfs/cms.cern.ch/SITECONF/local/JobConfig/site-local-config.xml

indicating an attempt to access the condition database outside of GT. The problem was traced to the RecoTau package (see discussion

The code retrieves a db (or more). It was verified that it is not in the default GT.

This is an undesired feature of the production code, not related to the OD example, and a fix will be implemented in CMSSW and a new GT can be provided with the necessary tags.

An update to most of the already prepared records/docs will be needed, a new container needs to be built and the updated GTs moved to /cmvfs and eos.

See also in the release guide how to disconnect frontier in the CMS OD container to test that database access in only to the GT on /cvmfs. It will speed up testing.