bionode / bionode-watermill

💧Bionode-Watermill: A (Not Yet Streaming) Workflow Engine
https://bionode.gitbooks.io/bionode-watermill/content/
MIT License
37 stars 11 forks source link

Add support for cluster computing #63

Open tiagofilipe12 opened 6 years ago

tiagofilipe12 commented 6 years ago

Resulting from a discussion with @thejmazz in gitter, it would be great to have support for cluster computing (such as qsub and slurm (srun and so on...)) as a plugin. It would be a matter of instead of passing bash -c "./program" pass something like qsub ..opts --cmd 'bash -c "./program" as an option for bionode-watermill (maybe with the CLI). Then some options could be added to this module where this options could be a parameter for task object (something like task.props).