dmwm / CRABServer

16 stars 38 forks source link

make wrapper work for jobs submitted with CMSSW release which uses python 3.12 #8805

Open belforte opened 22 hours ago

belforte commented 22 hours ago

see https://github.com/dmwm/CRABClient/pull/5350

the job stdout for a job submitted with CMSSW_14_2_PY312_X_2024-11-20-2300 has the following. Is WMCore python env. leaking into CMSSW ?


==== CMSSW JOB Execution started at Thu Nov 21 10:30:36 2024 UTC ====
==== Will execute stdbuf -oL -eL cmsRun -j FrameworkJobReport.xml PSet.py > cmsRun-stdout.log.tmp 2>&1
Error executing application in CMSSW environment.
    See stdout log
==== Execution FAILED at Thu Nov 21 10:30:38 2024 UTC ====
==== CMSSW JOB Execution completed at Thu Nov 21 10:30:38 2024 UTC ====
Application exit code: 5
==== Execution FINISHED at Thu Nov 21 10:30:38 2024 UTC ====
======== CMSSW OUTPUT STARTING ========
NOTICE: lines longer than 3000 characters will be truncated
== CMSSW: /cvmfs/cms.cern.ch/cc8_amd64_gcc9/external/py3-future/0.18.2/lib/python3.8/site-packages/past/types/oldstr.py:23: SyntaxWarning: invalid escape sequence '\d'
== CMSSW:   """
== CMSSW: ----- Begin Fatal Exception 21-Nov-2024 05:30:38 EST-----------------------
== CMSSW: An exception of category 'ConfigFileReadError' occurred while
== CMSSW:    [0] Processing the python configuration file named PSet.py
== CMSSW: Exception Message:
== CMSSW:  unknown python problem occurred.
== CMSSW: ModuleNotFoundError: No module named 'imp'
== CMSSW:
== CMSSW: At:
== CMSSW:   /cvmfs/cms.cern.ch/cc8_amd64_gcc9/external/py3-future/0.18.2/lib/python3.8/site-packages/past/builtins/misc.py(45): <module>
== CMSSW:   <frozen importlib._bootstrap>(488): _call_with_frames_removed
== CMSSW:   <frozen importlib._bootstrap_external>(995): exec_module
== CMSSW:   <frozen importlib._bootstrap>(950): _load_unlocked
== CMSSW:   <frozen importlib._bootstrap>(1334): _find_and_load_unlocked
== CMSSW:   <frozen importlib._bootstrap>(1360): _find_and_load
== CMSSW:   /cvmfs/cms.cern.ch/cc8_amd64_gcc9/external/py3-future/0.18.2/lib/python3.8/site-packages/past/builtins/__init__.py(54): <module>
== CMSSW:   <frozen importlib._bootstrap>(488): _call_with_frames_removed
== CMSSW:   <frozen importlib._bootstrap_external>(995): exec_module
== CMSSW:   <frozen importlib._bootstrap>(950): _load_unlocked
== CMSSW:   <frozen importlib._bootstrap>(1334): _find_and_load_unlocked
== CMSSW:   <frozen importlib._bootstrap>(1360): _find_and_load
== CMSSW:   /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_14_2_PY312_X_2024-11-20-2300/src/FWCore/ParameterSet/python/Types.py(6): <module>
== CMSSW:   <frozen importlib._bootstrap>(488): _call_with_frames_removed
== CMSSW:   <frozen importlib._bootstrap_external>(995): exec_module
== CMSSW:   <frozen importlib._bootstrap>(950): _load_unlocked
== CMSSW:   <frozen importlib._bootstrap>(1334): _find_and_load_unlocked
== CMSSW:   <frozen importlib._bootstrap>(1360): _find_and_load
== CMSSW:   /cvmfs/cms-ib.cern.ch/sw/x86_64/week0/el8_amd64_gcc12/cms/cmssw/CMSSW_14_2_PY312_X_2024-11-20-2300/src/FWCore/ParameterSet/python/Config.py(16): <module>
== CMSSW:   <frozen importlib._bootstrap>(488): _call_with_frames_removed
== CMSSW:   <frozen importlib._bootstrap_external>(995): exec_module
== CMSSW:   <frozen importlib._bootstrap>(950): _load_unlocked
== CMSSW:   <frozen importlib._bootstrap>(1334): _find_and_load_unlocked
== CMSSW:   <frozen importlib._bootstrap>(1360): _find_and_load
== CMSSW:   PSet.py(1): <module>
== CMSSW:
== CMSSW: ----- End Fatal Exception -------------------------------------------------
== CMSSW:
belforte commented 16 hours ago

I do not like lack of environment control. Moving to top of list