search

theislab / scib-pipeline

Snakemake pipeline that works with the scIB package to benchmark data integration methods.
MIT License
66 stars 28 forks source link

One tool = one environment? #3

Open ilibarra opened 4 years ago

ilibarra commented 4 years ago

Hi @LuckyMD and @danielStrobl,

I am interested in setting up a one-environment per R package set of environments (e.g. scIB-R-integration-liger, scIB-R-integration-fastmnn, etc). This is for avoiding multi-package conflicts with some tools that I think could be solved if isolating and treating manually each tool.

I think this is a reasonable idea to avoid multi-tool conflicts within the same environment, and as far as I remember @danielStrobl also agreed with me last week (perhaps not anymore). @LuckyMD might I start a branch for this? If you observe caveats or prefer more thoughts/discussion into this let me know.

Thank you!

LuckyMD commented 4 years ago

Hey!

This is a good idea. It would however require at minimum a bit of shuffling around in the Snakemake pipeline. There are 2 ways to do this:

  1. Create a separate environment per tool in conda then assign the name of this env to a variable in the config.yaml that is parsed and then used to run the environment.
  2. Create a separate rule for each tool in the Snakefile, and then use the conda: keyword to let Snakemake create the environments for you for this tool.

I would probably go for option 1. Option 2 is cleaner as you have version control inbuilt into the environments directly and you can't ruin the environments accidentally (the same instructions are run every time), but it also requires a large restructuring of the Snakemake pipeline. With option 1, you would just need to read in the environment name from the config per method and then slightly adapt the command that is run in the integration rules.

Maybe @mumichae has something to add here, and may be able to help?

mumichae commented 3 years ago

I would go with option 1, as this fits well with the Snakemake's integrated environment feature

Also, development for the pipeline has moved to https://github.com/theislab/scib-pipeline, so I think we should move the discussion there