Starting from #713 by @shane-breeze and changing some things:
when using the HTCondor transfer, ship also the lib, bin and python directory to avoid compiling on the worker node
implement also HTCondor submission using the AFS shared filesystem
allow specifying the max job duration (cern condor default is 20 mins, which is usually too short)
allow remote stage out of trees to EOS also from HTCondor jobs
This PR keeps LSF as default, but all three submission modes are possible. E.g. for a job of about 4 hours wall clock time:
To run on LSF, do heppy_batch.py .... -b 'bsub -q 8nh < batchScript.sh'
To run on Condor using AFS heppy_batch.py .... -b 'run_condor_simple.sh -t 240 < batchScript.sh'
To run on Condor using Condor transfer heppy_batch.py .... -b 'run_condor.sh -t 240 < batchScript.sh'
Note that when using Condor transfer one must run heppy_untarcfg.py . in the output directory after the jobs are complete, while when using AFS it behaves like LSF so it needs no untarring.
Starting from #713 by @shane-breeze and changing some things:
This PR keeps LSF as default, but all three submission modes are possible. E.g. for a job of about 4 hours wall clock time:
heppy_batch.py .... -b 'bsub -q 8nh < batchScript.sh'heppy_batch.py .... -b 'run_condor_simple.sh -t 240 < batchScript.sh'heppy_batch.py .... -b 'run_condor.sh -t 240 < batchScript.sh'Note that when using Condor transfer one must run
heppy_untarcfg.py .in the output directory after the jobs are complete, while when using AFS it behaves like LSF so it needs no untarring.