search

matsengrp / cft

Clonal family tree
5 stars 3 forks source link

How to run the workflow? #3

Closed matsen closed 7 years ago

matsen commented 7 years ago

My current thinking is Toil, which apparently can use slurm as an execution engine: http://toil.readthedocs.io/en/latest/batchSystem.html?highlight=slurm

matsen commented 7 years ago

CWL (which is the "language" of Toil) is sounding cooler the more I learn about it. Another post: http://b-t.cr/t/defining-an-api-for-repertoire-sequence-analysis-tools/141/10?u=ematsen

matsen commented 7 years ago

Seems like we are up and running with Toil, though we can still change!

cswarth commented 7 years ago

To be candid, we are limping long with Toil, and particularly Toil/CWL/Slurm. We can navigate one path that seems to work, provided you're running my patched version of Toil. (My patch was submitted in a bug report but the fix was assigned to a member of their development team.)

The entire batch backend of Toil (the slurm, aws, gridengine support) is scheduled for refactoring, and CWL support is missing some significant features. As others have said, slurm support and CWL is particular are a WIP.