Cern batch

[gerbaudo]

Hi Ted and Burt,

This is a first try at running supy jobs on lxbatch. I had to change a few things, but now it seems about to work.

• with lxbatch, it looks like the script job.sh needs to be (1) available from any node (it looks like bsub is reading the file to memory from a different machine, so /tmp is not an option here). Currently using pwd/supy/batch/. (2) is has to be chmod'ed +x
• configurable queue (now '8nh')?
• sites/cernJob.sh not quite generic. Maybe we can have one that provides some reasonable version of python+root, and allow the user to specify one from configuration.py?
• useCachedFileLists relies on /tmp. Is /tmp accessible from any node in your setups? on lxplus this file is available when I submit the jobs, but then it's not available when the job starts.It's actually not clear to me what's going on: even if I set useCachedFileLists to false it still tries to use it, and it cannot access it from /tmp

There are still some rough edges that should be improved; please let me know what you think. Thanks!

davide

[gerbaudo]

Sorry, I realized now that I created this branch on top of the changes below (I needed some of them to test my analysis...): git merge origin/eos-access git merge origin/no-root-ext git merge origin/bn-batch

I don't know if there is an easy way to diff only of these changes I made, and not the ones I pulled in. The only modifications for this pull request are in bin/supy sites/cernJob.sh sites/cernSub.sh

Sorry for this mess! davide

[gerbaudo]

We will close this and include it in multiexperiment