TestDQMOnlineClient unit failures

qliphy commented 2 years ago

Appearing in IB now, possibly related to https://github.com/cms-data/DQM-Integration/pull/2 https://github.com/cms-sw/cmssw/pull/38925#issuecomment-1202139464

cmsbuild commented 2 years ago

A new Issue was created by @qliphy Qiang Li.

@Dr15Jones, @perrotta, @dpiparo, @rappoccio, @makortel, @smuzaffar, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

qliphy commented 2 years ago

===== Test "TestDQMOnlineClient-l1tstage2_dqm_sourceclient" ====

[[ 1 -eq 0 ]]
[[ -z '' ]]
LOCAL_TEST_DIR=.
[[ -z '' ]]
CLIENTS_DIR=./src/DQM/Integration/python/clients
mkdir -p ./upload
cmsRun ./src/DQM/Integration/python/clients/l1tstage2_dqm_sourceclient-live_cfg.py unitTest=True Querying DAS for files... the query is file run=355380 dataset=/ExpressPhysics/Run2022B-Express-v1/FEVT lumi=19 DAS succeeded after 1 attempts 0 found files: ['/store/express/Run2022B/ExpressPhysics/FEVT/Express-v1/000/355/380/00000/b8a57fc4-5656-42b4-9b7b-2e647baf65e8.root'] edmFileUtil --catalog file:/cvmfs/cms-ib.cern.ch/SITECONF/local/PhEDEx/storage.xml?protocol=xrootd --events /store/express/Run2022B/ExpressPhysics/FEVT/Express-v1/000/355/380/00000/b8a57fc4-5656-42b4-9b7b-2e647baf65e8.root | tail -n +9 | head -n -5 | awk '{ print $3 }' the query is file run=355380 dataset=/ExpressPhysics/Run2022B-Express-v1/FEVT lumi=20 DAS succeeded after 1 attempts 0 found files: ['/store/express/Run2022B/ExpressPhysics/FEVT/Express-v1/000/355/380/00000/2b3f408f-883a-4f64-8083-9e5786589069.root'] edmFileUtil --catalog file:/cvmfs/cms-ib.cern.ch/SITECONF/local/PhEDEx/storage.xml?protocol=xrootd --events /store/express/Run2022B/ExpressPhysics/FEVT/Express-v1/000/355/380/00000/2b3f408f-883a-4f64-8083-9e5786589069.root | tail -n +9 | head -n -5 | awk '{ print $3 }' Got 2 files. Loaded configuration file from: [] dqmRunConfig: cms.PSet( collectorHost = cms.untracked.string('127.0.0.1'), collectorPort = cms.untracked.int32(9190), type = cms.untracked.string('userarea') ) ----- Begin Fatal Exception 08-Aug-2022 02:59:03 CEST----------------------- An exception of category 'ConfigFileReadError' occurred while [0] Processing the python configuration file named ./src/DQM/Integration/python/clients/l1tstage2_dqm_sourceclient-live_cfg.py Exception Message: unknown python problem occurred. AttributeError: 'Process' object has no attribute 'totemTriggerRawToDigi'

At: ./src/DQM/Integration/python/clients/l1tstage2_dqm_sourceclient-live_cfg.py(66):

qliphy commented 2 years ago

===== Test "TestDQMOnlineClient-l1tstage2emulator_dqm_sourceclient" ====

[[ 1 -eq 0 ]]
[[ -z '' ]]
LOCAL_TEST_DIR=.
[[ -z '' ]]
CLIENTS_DIR=./src/DQM/Integration/python/clients
mkdir -p ./upload
cmsRun ./src/DQM/Integration/python/clients/l1tstage2emulator_dqm_sourceclient-live_cfg.py unitTest=True Querying DAS for files... the query is file run=355380 dataset=/ExpressPhysics/Run2022B-Express-v1/FEVT lumi=19 DAS succeeded after 1 attempts 0 found files: ['/store/express/Run2022B/ExpressPhysics/FEVT/Express-v1/000/355/380/00000/b8a57fc4-5656-42b4-9b7b-2e647baf65e8.root'] edmFileUtil --catalog file:/cvmfs/cms-ib.cern.ch/SITECONF/local/PhEDEx/storage.xml?protocol=xrootd --events /store/express/Run2022B/ExpressPhysics/FEVT/Express-v1/000/355/380/00000/b8a57fc4-5656-42b4-9b7b-2e647baf65e8.root | tail -n +9 | head -n -5 | awk '{ print $3 }' the query is file run=355380 dataset=/ExpressPhysics/Run2022B-Express-v1/FEVT lumi=20 DAS succeeded after 1 attempts 0 found files: ['/store/express/Run2022B/ExpressPhysics/FEVT/Express-v1/000/355/380/00000/2b3f408f-883a-4f64-8083-9e5786589069.root'] edmFileUtil --catalog file:/cvmfs/cms-ib.cern.ch/SITECONF/local/PhEDEx/storage.xml?protocol=xrootd --events /store/express/Run2022B/ExpressPhysics/FEVT/Express-v1/000/355/380/00000/2b3f408f-883a-4f64-8083-9e5786589069.root | tail -n +9 | head -n -5 | awk '{ print $3 }' Got 2 files. Loaded configuration file from: [] dqmRunConfig: cms.PSet( collectorHost = cms.untracked.string('127.0.0.1'), collectorPort = cms.untracked.int32(9190), type = cms.untracked.string('userarea') ) ----- Begin Fatal Exception 08-Aug-2022 02:59:08 CEST----------------------- An exception of category 'ConfigFileReadError' occurred while [0] Processing the python configuration file named ./src/DQM/Integration/python/clients/l1tstage2emulator_dqm_sourceclient-live_cfg.py Exception Message: unknown python problem occurred. AttributeError: 'Process' object has no attribute 'totemTriggerRawToDigi'

At: ./src/DQM/Integration/python/clients/l1tstage2emulator_dqm_sourceclient-live_cfg.py(66):

qliphy commented 2 years ago

assign dqm, alca

cmsbuild commented 2 years ago

New categories assigned: dqm,alca

@jfernan2,@ahmad3213,@yuanchao,@micsucmed,@rvenditti,@emanueleusai,@francescobrivio,@malbouis,@saumyaphor4252,@tvami,@ChrisMisan,@pmandrik you have been requested to review this Pull request/Issue and eventually sign? Thanks

qliphy commented 2 years ago

adding also @cms-sw/ctpps-dpg-l2

francescobrivio commented 2 years ago

unassign alca

the PRs mentioned in the Issue description only touch the unitTest of the beamhlt DQM client (and its input streamer files) and have nothing to do with the l1tstage2_dqm and l1tstage2emulator_dqm clients

qliphy commented 2 years ago

This issue seems to be due to https://github.com/cms-sw/cmssw/pull/38886/

I can reproduce the error locally.

1) First CMSSW_12_5_X_2022-08-07-0000 works well 2) Then CMSSW_12_5_X_2022-08-07-2300 + https://github.com/cms-sw/cmssw/pull/38886/ will give the error on 'totemTriggerRawToDigi'

cd DQM/Integration/python/clients cmsRun l1tstage2emulator_dqm_sourceclient-live_cfg.py unitTest=True

qliphy commented 2 years ago

@forthommel would you please have a quick look and fix? @cms-sw/ctpps-dpg-l2

ChrisMisan commented 2 years ago

This is probably because of the changes in EventFilter/CTPPSRawToDigi/python/ctppsRawToDigi_cff.py

mmusich commented 2 years ago

I see failures seemingly related to this in the unit test of PR cms-sw/cmssw#38760

https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5b625f/26728/unitTests/failed.html

+ cmsRun ./src/DQM/Integration/python/clients/l1tstage2_dqm_sourceclient-live_cfg.py unitTest=True
----- Begin Fatal Exception 09-Aug-2022 18:14:01 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named ./src/DQM/Integration/python/clients/l1tstage2_dqm_sourceclient-live_cfg.py
Exception Message:
 unknown python problem occurred.
AttributeError: 'Process' object has no attribute 'totemTriggerRawToDigi'

At:
  ./src/DQM/Integration/python/clients/l1tstage2_dqm_sourceclient-live_cfg.py(66): <module>

----- End Fatal Exception -------------------------------------------------

Will it get fixed as well?

tvami commented 2 years ago

I think https://github.com/cms-sw/cmssw/pull/38991 is fixing it already, it's just that the DQM friends havent looked at it yet

cms-sw / cmssw

TestDQMOnlineClient unit failures #38987