search

pombreda / ruffus

Automatically exported from code.google.com/p/ruffus
MIT License
0 stars 0 forks source link

how to run into a Sun Grid Engine system? (qsub, qmake) #2

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Hi,
thank you very much for developing this, I have been looking for a good
alternative to make for a long time.

I want to run a pipeline into Sun Grid Engine system.
I am not a systemist, but I know that every process must be run throught a
command called 'qsub', to send it into the grid queue where it will be run.

With make, I used a tool called 'qmake' which was provided by Sun as well.

Does ruffus provide support for this popular grid system? Did I miss
something in the docs?
Cheers

Original issue reported on code.google.com by dalloli...@gmail.com on 2 Jun 2009 at 3:00

GoogleCodeExporter commented 9 years ago 
We use "qrsh" which doesn't return until the task is finished. This means if 
anything
goes wrong in your job running on your cluster, qrsh will return false. You can 
throw
an exception from the task if you want and halt the pipeline. "qsub" returns
immediately before the job has finished. If you have subsequent downstream 
tasks, you
then need to keep polling to see if the job has finished before proceeding.

There is a gotcha with qrsh which is detailed in the FAQ.
See src/ruffus/test/qrsh_workaround.py

Original comment by bunbu...@gmail.com on 5 Jun 2009 at 5:58

GoogleCodeExporter commented 9 years ago

Original comment by bunbu...@gmail.com on 15 Jun 2009 at 9:17