Problem running jobs with HTCondor

bundocka commented 5 years ago

Description

When trying to submit jobs to HTCondor with cmsl1t_batch, the jobs fail with the following error. This is using Kreczko's branch here to fix condor submission. https://github.com/cms-l1t-offline/cms-l1t-analysis/pull/161

Steps/code to reproduce issue

Set up cms-l1t-analysis as usual.

Expected results

Completed jobs

Actual results

The job error file shows:

You are using pip version 8.1.2, however version 19.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
ERROR:ROOT.TSystem.ExpandFileName] input: $HOME/.root.mimes, output: $HOME/.root.mimes
Traceback (most recent call last):
  File "/afs/cern.ch/user/b/bundocka/work/201118/CMSSW_10_3_1/src/fix/cms-l1t-analysis/bin/cmsl1t", line 9, in <module>
    from cmsl1t.io.eventreader import EventReader
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc62-opt/lib/ROOT.py", line 303, in _importhook
    return _orig_ihook( name, *args, **kwds )
  File "/afs/cern.ch/user/b/bundocka/work/201118/CMSSW_10_3_1/src/fix/cms-l1t-analysis/cmsl1t/io/__init__.py", line 2, in <module>
    from rootpy.plotting.hist import Hist, _HistBase
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc62-opt/lib/ROOT.py", line 303, in _importhook
    return _orig_ihook( name, *args, **kwds )
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc62-opt/lib/python2.7/site-packages/rootpy-0.8.1-py2.7.egg/rootpy/plotting/__init__.py", line 16, in <module>
    from .legend import Legend
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc62-opt/lib/ROOT.py", line 303, in _importhook
    return _orig_ihook( name, *args, **kwds )
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc62-opt/lib/python2.7/site-packages/rootpy-0.8.1-py2.7.egg/rootpy/plotting/legend.py", line 12, in <module>
    from .box import _Positionable
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc62-opt/lib/ROOT.py", line 303, in _importhook
    return _orig_ihook( name, *args, **kwds )
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc62-opt/lib/python2.7/site-packages/rootpy-0.8.1-py2.7.egg/rootpy/plotting/box.py", line 9, in <module>
    from .utils import canvases_with
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc62-opt/lib/ROOT.py", line 303, in _importhook
    return _orig_ihook( name, *args, **kwds )
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc62-opt/lib/python2.7/site-packages/rootpy-0.8.1-py2.7.egg/rootpy/plotting/utils.py", line 10, in <module>
    from .canvas import _PadBase
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc62-opt/lib/ROOT.py", line 303, in _importhook
    return _orig_ihook( name, *args, **kwds )
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc62-opt/lib/python2.7/site-packages/rootpy-0.8.1-py2.7.egg/rootpy/plotting/canvas.py", line 186, in <module>
    class Pad(_PadBase, QROOT.TPad):
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc62-opt/lib/python2.7/site-packages/rootpy-0.8.1-py2.7.egg/rootpy/utils/module_facade.py", line 84, in __getattr__
    result = sup.__getattr__(key)
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc62-opt/lib/python2.7/site-packages/rootpy-0.8.1-py2.7.egg/rootpy/utils/quickroot.py", line 67, in __getattr__
    if Load(libname) == 0:
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc62-opt/lib/python2.7/site-packages/rootpy-0.8.1-py2.7.egg/rootpy/utils/quickroot.py", line 67, in __getattr__
    if Load(libname) == 0:
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc62-opt/lib/python2.7/site-packages/rootpy-0.8.1-py2.7.egg/rootpy/logger/magic.py", line 247, in intercept_next_line
    raise exception
rootpy.ROOTError: level=3000, loc='TSystem::ExpandFileName', msg='input: $HOME/.root.mimes, output: $HOME/.root.mimes'

kreczko commented 5 years ago

To reproduce:

git clone https://github.com/bundocka/cms-l1t-analysis.git
cd cms-l1t-analysis
git remote add upstream https://github.com/cms-l1t-offline/cms-l1t-analysis.git
git pull --rebase upstream master
source bin/env.sh
voms-proxy-init
git remote add bk https://github.com/benkrikler/cms-l1t-analysis.git
git pull bk BK_add_home_variable
git remote add bundocka https://github.com/bundocka/cms-l1t-analysis.git
git pull bundocka trigPlots
make setup

Note: CERN has moved to C7, we need to update env.sh

kreczko commented 5 years ago

git clone https://github.com/bundocka/cms-l1t-analysis.git
cd cms-l1t-analysis
git remote add upstream https://github.com/cms-l1t-offline/cms-l1t-analysis.git
git pull --rebase upstream master

git remote add bk https://github.com/benkrikler/cms-l1t-analysis.git
git pull bk BK_add_home_variable

git remote add bundocka https://github.com/bundocka/cms-l1t-analysis.git
git pull bundocka trigPlots

git remote add lk https://github.com/kreczko/cms-l1t-analysis.git
git pull lk kreczko-issue-164
# remove the conflicting line from env.sh (remove the line with the explicit LCG version).

source setup.sh
voms-proxy-init
make setup-build-dir
make setup-external

kreczko commented 5 years ago

Fixed in master:

git clone git@github.com:cms-l1t-offline/cms-l1t-analysis.git
source setup.sh
voms-proxy-init
make setup-build-dir
make setup-external
cmsl1t_batch -c config/rate_vs_pu.yaml -f 3

cms-l1t-offline / cms-l1t-analysis