search

dib-lab / elvers

(formerly eelpond) an automated RNA-Seq workflow system
https://dib-lab.github.io/elvers/
Other
28 stars 3 forks source link

update conda envs #119

Open bluegenes opened 5 years ago

bluegenes commented 5 years ago

Add a param, --update_conda_envs to update the conda environments when desired. Don't do this automatically, because reproducibility? Do we want to enable choice of list, or just do all at once?

Note that if the environment.yml file changes, snakemake will automatically update the environment. But in cases where the file stays the same but a new program version is released, we need a way of updating (motivation: salmon example below).

/Users/tessa/miniconda3/envs/test_elvers_install/bin/python /Users/tessa/Dropbox/dib-lab/eelpond/.snakemake/scripts/tmp_60kx_7d.quant-wrapper.py
Activating conda environment: /Users/tessa/Dropbox/dib-lab/eelpond/.snakemake/conda/1d46b4fe
salmon quant -i /Users/tessa/Dropbox/dib-lab/eelpond/yeast_out/quant/yeast_input.salmonindex --validateMappings  -l A  -r /Users/tessa/Dropbox/dib-lab/eelpond/yeast_out/preprocess/ERR458501_2_se.trim.fq.gz -o /Users/tessa/Dropbox/dib-lab/eelpond/yeast_out/quant/ERR458501_x_yeast_input.salmon  -p 1   2> /Users/tessa/Dropbox/dib-lab/eelpond/yeast_out/logs/salmon/ERR458501_x_yeast_input.log
Version Info: ### PLEASE UPGRADE SALMON ###
### A newer version of Salmon with important bug fixes and improvements is available. ####
###
The newest version, available at https://github.com/COMBINE-lab/salmon/releases
contains new features, improvements, and bug fixes; please upgrade at your
earliest convenience.
###
ctb commented 5 years ago

Do we not version pin salmon in the rule-specific yml file? If so, then wouldn't updating it basically involve updating the rule file?

Separate question is whether we want to keep software versions pinned in the files for each eelpond/elvars version?

bluegenes commented 5 years ago

Not all software is pinned rn, but it's a good idea. Then updating conda envs would just involve updating the environment.yml file for each rule.

If we wanted to, we could make the snakemake rule conda: environment.yml pull from the program parameters (e.g. conda: params.get('conda_env', environment.yml), so that anyone who wanted to specify a specific version could provide their own environment.yml files or parameters. That way users are not limited to when/how we update the pinned versions. Taking that a step further, we could keep environment.yml files for versions (forward from now) as they come out, like salmon_v10.0.1.yml, and scrape the rule dirs to produce a list of all program versions available. Not much extra work, if we'll be updating pinned versions anyway.