LSF Support

[gdevenyi]

I worked out how LSF works with bsub a while ago for magetbrain reasons, worth adding?

[gdevenyi]

Future notes: LSF sucks for a couple of reasons:

• Job magics only accepted if submitted via STDIN
• Doesn't allow seconds in walltime specification.

[gdevenyi]

LSF doesn't automatically name jobs since it can only accept via stdin

[gdevenyi]

Need to write "run process and feed it this file via STDIN" function.

[pipitone]

I don't use LSF myself, so not a priority unless you or others we want to support use it.

[gdevenyi]

I have users on LSF to support, so I will write this at some point.

[gdevenyi]

Notes:

-n <number> for number of cpus to request
-M <number> the units for which at set on a cluster configuration, could be kB, MB, GB etc
-J <jobname>
-W <walltime> cannot handle seconds
-o <directory> place to store job logs

[gdevenyi]

LSF supports pattern matching for job names

-w done(afterok_pattern)

[gdevenyi]

LSF array jobs are handled in the name specification: https://sites.google.com/site/anshulkundaje/inotes/programming/clustersubmit/lsf

Relevant parameters:

Relevant environment variables LSB_JOBINDEX : For a job array this gives the index of the job in the job array

[gdevenyi]

@pipitone do you know how to do piping in python? I tried this before and ended up an inelegant wrapper script so that bash would save me.

[gdevenyi]

Note for later: open source LSF is called OpenLava http://www.openlava.org/