search

DataBiosphere / toil

A scalable, efficient, cross-platform (Linux/macOS) and easy-to-use workflow engine in pure Python.
http://toil.ucsc-cgl.org/.
Apache License 2.0
881 stars 237 forks source link

toil + MPI thoughts #3578

Open mr-c opened 3 years ago

mr-c commented 3 years ago

Follow on from https://github.com/DataBiosphere/toil/issues/3517#issuecomment-825609742

Should the Toil batch backends have a concept of requesting multiple nodes? Or shall we keep the current hack mentioned above?

┆Issue is synchronized with this Jira Story ┆Issue Number: TOIL-884

DailyDreaming commented 3 years ago

One thing I should add, we need a better way of testing SLURM with MPI in toil.

gmloose commented 1 month ago

Any chance this will be picked up as a feature request any time soon? The details of the current "problem" are in the comment referenced above.