Installation

To install BatchQ type:

$ easy_install batchq

Documentation

The documentation can be found at http://batchq.rtfd.org

RENAMING AND MOVING

• (V) Shell.status -> shell.state
• (V) Shell.status: return shell.state.texts[shell.state]
• batchq.core.Shell -> batchq.core.QueueElement
• batchq.core.Collection -> batchq.core.JobList
• FileCommander -> batchq.core.batch.Connection, batchq.core.batch.Machine or something
• batchq.core.batch -> batchq.batch.init
• end_machine -> previous_machine(), deselect_machine()

TODO

• There might occur an error when large output are retrieved (len(buf) > 10000 )
• Add SHA/MD5 ID generation to the batch modules
• Combine objects of the same kind
• Add caching on the status call
• (V) Add file caching to submission modules
• Move batchq files to a central location

OPTIMISATIONS

• Reduce the amount of calls of read_non_blocking_until

• Reduce the amount of calls to writeChar / put

• Many calls to base.py:274(fix_buffer)

  • Remove len from fix buffer and you will reduce the calls to len by a factor 1/2 and save ~0.5 sec pr. run call.
  • Last half of the function can be deleted if empty lines are always appended in the size of the line length

• Also many calls to patternregister.py:152(parse)

• Make Shell + Shell, Subshell + Subshell and LSF + LSF reduable to a script

• Checkout wether it is possible to introduce additional caching

BUGS

• Following input: 400075#-->if [ -f '.batchq.qa_main-16qubit_975_400paramh5-16qubit_975_400outh5.p then echo "SUCCESS"; fi SUCCESS is interpreted as

  -->cat .batchq.qa_main-16qubit_950_400paramh5-16qubit_950_400outh5.pid

  400075#-->if [ -f '.batchq.qa_main-16qubit_975_400paramh5-16qubit_975_400outh5.p then echo "SUCCESS"; fi SUCCESS

• In the LSF submission commmand, if no ".batchq.id.finished' exists, if [ $(cat blah ) -ne 0 ] fails. Add a check wether the file exists first. If not create .failed

• Commands with "find some*" are wrong and should be replaced with their regex equivalents

• When things are subshelled you may run into problems with output that arrives asyncronously. This means that consume output will fail.