Snakemake --conda-frontend conda error (mamba package not available)

[ilikvlad]

Hello,

together with Erich Schwarz we would like to run charcoal on some genomic assemblies, however when I tried to run charcoal, it ran for a while and then kept giving me this error message:

Error: mamba package manager is not available. The mamba package manager (https://github.com/mamba-org/mamba) is an extremely fast and robust conda replacement. It is the recommended way of using Snakemake's conda integration. It can be installed with conda install -n base -c conda-forge mamba. If you still prefer to use conda, you can enforce that by setting --conda-frontend conda.

I do not want to replace my biconda by mamba, so I understood that I have to change the snakemake --conda-frontend to "conda" instead of "mamba" (default) for charcoal, but I was not able to figure out how, could you please help me to solve this issue?

Thank you,

Vladislav

[ctb]

hi @ilikvlad,

you should be able to append --conda-frontend conda to the charcoal command line and have it work.

Alternatively, you can install mamba in its own conda environment and just use charcoal from there. It will not be installed outside that conda environment and will not interfere with your base conda environment unless you install it in base. Note that mamba is much, much faster than conda :).

Digging around a bit, I'm... actually a little surprised you're running into this, since snakemake for charcoal is pinned at v5.8.1. (I should look at upgrading that!) Are you following the instructions at the quickstart when you get this message?

[ctb]

Right, just verified - the version of snakemake installed in the environment.yml for charcoal specifies an older version of snakemake that does not use mamba.

(I thought I recalled having to update the continuous integration to install mamba when I updated snakemake over in spacegraphcats, for exactly this reason; and I was right, I did have to! https://github.com/spacegraphcats/spacegraphcats/pull/407/files#diff-faff1af3d8ff408964a57b2e475f69a6b7c7b71c9978cccc8f471798caac2c88R33)

[ctb]

oops! I spoke too soon. The pip install -e . uses setup.py, which specifies snakemake >= 5.10, which then installs the latest snakemake v6, which has mamba integration. Sigh.

[ctb]

Further update:

python -m charcoal run demo/demo.conf -j 4 --conda-frontend conda

works for me!

[ilikvlad]

Hello @ctb,

thank you for working on this, I carefully followed all the instructions and also used the command you proposed:

python -m charcoal run demo/demo.conf -j 4 --conda-frontend conda

However, the result of running this command was this:

read 2 provided lineages config file checks PASSED! ** from here on out, it's all snakemake... Building DAG of jobs... Updating job make_index. Using shell: /usr/bin/bash Provided cores: 4 Rules claiming more threads will be scaled down. Job counts: count jobs 1 all 8 download_matching_genomes_one_by_one 4 make_genbank_info_csv 1 make_hitlist_matches_info_csv 1 make_html_alignment 1 make_index 1 make_notebook_alignment 8 mashmap_compare 25

[Tue Jun 1 12:00:33 2021] rule download_matching_genomes_one_by_one: input: genbank_info/GCA_900066645.info.csv output: genbank_genomes/GCA_900066645_genomic.fna.gz jobid: 47 wildcards: acc=GCA_900066645

[Tue Jun 1 12:00:33 2021] rule download_matching_genomes_one_by_one: input: genbank_info/GCF_002159845.info.csv output: genbank_genomes/GCF_002159845_genomic.fna.gz jobid: 35 wildcards: acc=GCF_002159845

[Tue Jun 1 12:00:33 2021] rule download_matching_genomes_one_by_one: input: genbank_info/GCF_900199515.info.csv output: genbank_genomes/GCF_900199515_genomic.fna.gz jobid: 38 wildcards: acc=GCF_900199515

[Tue Jun 1 12:00:33 2021] rule download_matching_genomes_one_by_one: input: genbank_info/GCF_002160955.info.csv output: genbank_genomes/GCF_002160955_genomic.fna.gz jobid: 41 wildcards: acc=GCF_002160955

Error: mamba package manager is not available. The mamba package manager (https://github.com/mamba-org/mamba) is an extremely fast and robust conda replacement. It is the recommended way of using Snakemake's conda integration. It can be installed with conda install -n base -c conda-forge mamba.If you still prefer to use conda, you can enforce that by setting--conda-frontend conda. Error: mamba package manager is not available. The mamba package manager (https://github.com/mamba-org/mamba) is an extremely fast and robust conda replacement. It is the recommended way of using Snakemake's conda integration. It can be installed withconda install -n base -c conda-forge mamba.If you still prefer to use conda, you can enforce that by setting --conda-frontend conda. Error: mamba package manager is not available. The mamba package manager (https://github.com/mamba-org/mamba) is an extremely fast and robust conda replacement. It is the recommended way of using Snakemake's conda integration. It can be installed with conda install -n base -c conda-forge mamba.If you still prefer to use conda, you can enforce that by setting--conda-frontend conda. Error: mamba package manager is not available. The mamba package manager (https://github.com/mamba-org/mamba) is an extremely fast and robust conda replacement. It is the recommended way of using Snakemake's conda integration. It can be installed withconda install -n base -c conda-forge mamba.If you still prefer to use conda, you can enforce that by setting --conda-frontend conda. Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: /ocean/projects/mcb190015p/ilikvlad/charcoal/.snakemake/log/2021-06-01T120023.310789.snakemake.log Error in snakemake invocation: Command '['snakemake', '-s', '/ocean/projects/mcb190015p/ilikvlad/charcoal/charcoal/Snakefile', '--use-conda', '-j', '1', '-j', '4', '--conda-frontend', 'conda', '--configfile', '/ocean/projects/mcb190015p/ilikvlad/charcoal/charcoal/conf/defaults.conf', '/ocean/projects/mcb190015p/ilikvlad/charcoal/charcoal/conf/system.conf', 'demo/demo.conf']' returned non-zero exit status 1.

Still forcing me to install "mamba", but as you pointed out, it could be efficient to install mamba independently on bioconda,

Vladislav

[ctb]

Error: mamba package manager is not available. The mamba package manager (https://github.com/mamba-org/mamba) is an extremely fast and robust conda replacement. It is the recommended way of using Snakemake's conda integration. It can be installed with conda install -n base -c conda-forge mamba.If you still prefer to use conda, you can enforce that by setting--conda-frontend conda. Error: mamba package manager is not available. The mamba package manager (https://github.com/mamba-org/mamba) is an extremely fast and robust conda replacement. It is the recommended way of using Snakemake's conda integration. It can be installed withconda install -n base -c conda-forge mamba.If you still prefer to use conda, you can enforce that by setting --conda-frontend conda. Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: /ocean/projects/mcb190015p/ilikvlad/charcoal/.snakemake/log/2021-06-01T120023.310789.snakemake.log Error in snakemake invocation: Command '['snakemake', '-s', '/ocean/projects/mcb190015p/ilikvlad/charcoal/charcoal/Snakefile', '--use-conda', '-j', '1', '-j', '4', '--conda-frontend', 'conda', '--configfile', '/ocean/projects/mcb190015p/ilikvlad/charcoal/charcoal/conf/defaults.conf', '/ocean/projects/mcb190015p/ilikvlad/charcoal/charcoal/conf/system.conf', 'demo/demo.conf']' returned non-zero exit status 1.

Still forcing me to install "mamba", but as you pointed out, it could be efficient to install mamba independently on bioconda,

thanks, vladislav! how annoying - you can see that the --conda-frontend option is actually getting passed into snakemake just fine, and it's still producing that error!