search

cms-sw / cmssw

CMS Offline Software
http://cms-sw.github.io/
Apache License 2.0
1.07k stars 4.29k forks source link

edmConfigSplit broken in CMSSW_12_3_0_pre3 #36939

Open trtomei opened 2 years ago

trtomei commented 2 years ago

To reproduce:

cmsrel CMSSW_12_3_0_pre4
cd CMSSW_12_3_0_pre4/src
cmsenv
cmsDriver.py test -s RECO --processName=RECO --conditions auto:phase2_realistic_T21 --geometry Extended2026D77 --era Phase2C11I13M9 --eventcontent FEVTDEBUGHLT --datatier GEN-SIM-DIGI-RECO --filein file:input.root --nThreads 8 --no_exec
edmConfigSplit test_RECO.py

There will be:

  1. a bunch of warnings of the type: Warning: unsupported element 'hcalGainsGPUESProducer' in Task 'hbheRecHitProducerGPUTask' related to all kinds of GPUESProducer
  2. A crash that ends with module has no label. Perhaps it wasn't inserted into the process?. Full traceback below.
Traceback (most recent call last):
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/bin/slc7_amd64_gcc10/edmConfigSplit", line 55, in <module>
    files = process.splitPython(options)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/Config.py", line 1057, in splitPython
    parts.update(self._splitPythonList('sequences', self._itemsInDependencyOrder(self.sequences), options))
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/Config.py", line 884, in _splitPythonList
    dependencies = item.directDependencies()
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/SequenceTypes.py", line 382, in directDependencies
    result += findDirectDependencies(self, self._tasks,sortByType=sortByType)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/SequenceTypes.py", line 243, in findDirectDependencies
    dependencies += item.directDependencies(sortByType)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/SequenceTypes.py", line 1504, in directDependencies
    return findDirectDependencies(self, self._collection,sortByType=sortByType)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/SequenceTypes.py", line 243, in findDirectDependencies
    dependencies += item.directDependencies(sortByType)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/SequenceTypes.py", line 1504, in directDependencies
    return findDirectDependencies(self, self._collection,sortByType=sortByType)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/SequenceTypes.py", line 243, in findDirectDependencies
    dependencies += item.directDependencies(sortByType)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/SequenceTypes.py", line 1504, in directDependencies
    return findDirectDependencies(self, self._collection,sortByType=sortByType)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/SequenceTypes.py", line 243, in findDirectDependencies
    dependencies += item.directDependencies(sortByType)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/SequenceTypes.py", line 1504, in directDependencies
    return findDirectDependencies(self, self._collection,sortByType=sortByType)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/SequenceTypes.py", line 243, in findDirectDependencies
    dependencies += item.directDependencies(sortByType)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/SequenceTypes.py", line 1504, in directDependencies
    return findDirectDependencies(self, self._collection,sortByType=sortByType)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/SequenceTypes.py", line 243, in findDirectDependencies
    dependencies += item.directDependencies(sortByType)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/SequenceTypes.py", line 1504, in directDependencies
    return findDirectDependencies(self, self._collection,sortByType=sortByType)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/SequenceTypes.py", line 251, in findDirectDependencies
    sys.stderr.write("Warning: unsupported element '%s' in %s '%s'\n" % (str(item), type(element).__name__, element.label_()))
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre4/python/FWCore/ParameterSet/Mixins.py", line 521, in label_
    raise RuntimeError("module has no label.  Perhaps it wasn't inserted into the process?")
RuntimeError: module has no label.  Perhaps it wasn't inserted into the process?
cmsbuild commented 2 years ago

A new Issue was created by @trtomei Thiago Tomei.

@Dr15Jones, @perrotta, @dpiparo, @makortel, @smuzaffar, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

makortel commented 2 years ago

assign core

makortel commented 2 years ago

Adding @fwyzard (as the original author)

cmsbuild commented 2 years ago

New categories assigned: core

@Dr15Jones,@smuzaffar,@makortel you have been requested to review this Pull request/Issue and eventually sign? Thanks

makortel commented 2 years ago

I took a quick look. The warnings are caused by the https://github.com/cms-sw/cmssw/blob/4e164a96828c3ab1405b39d99365cebf70ef2790/FWCore/ParameterSet/python/SequenceTypes.py#L214 function not supporting ESProducers, ESSources, or Services in Task.

The actual error is caused on the line issuing the warning message https://github.com/cms-sw/cmssw/blob/4e164a96828c3ab1405b39d99365cebf70ef2790/FWCore/ParameterSet/python/SequenceTypes.py#L251 by asking a label from an anonymous Task.

makortel commented 2 years ago

I believe it would be sufficient to just make the findDirectDependencies() function to recognize ESProducer, ESSource, and Service, and handle the anonymous cases in the warning message printout. @fwyzard, do you agree?