A nice feature would be to support different types of pipelines that make it easier for users to set up common kinds of pipelines. For instance, we could start by offering two types of builds:
Custom: This just lets users enter any sequence of (shell) commands and they are run in order during the processing step.
Snakemake: Asks the user to specify a directory where the snakefile is located (defaults to root of the repo) and automatically:
uses an image that has snakemake installed
creates a processing step that includes cding to the specified directory and runs the snakemake pipeline.
More advanced features can then be added to the second option (snakemake) for caching intermediate steps/files, figuring out dependencies, etc.
In the first option. all the "smart" features are disabled so the user can just run any script they want without caching or dependency management. It would all be up to the user to figure out.
achilleas-k
commented
5 years ago
A nice feature would be to support different types of pipelines that make it easier for users to set up common kinds of pipelines. For instance, we could start by offering two types of builds:
snakefileis located (defaults to root of the repo) and automatically:cding to the specified directory and runs the snakemake pipeline.More advanced features can then be added to the second option (snakemake) for caching intermediate steps/files, figuring out dependencies, etc.
In the first option. all the "smart" features are disabled so the user can just run any script they want without caching or dependency management. It would all be up to the user to figure out.