Closed jemma-nelson closed 2 weeks ago
bentsherman
commented
1 year ago Fair enough, should be a simple fix. Fwiw, I've been using Mambaforge which provides mamba as a drop-in replacement for conda rather than a separate command, so that I don't have to bother with these settings.
pditommaso
commented
1 year ago Do you mean miniforge? Not sure it's the same it says
conda-forge set as the default (and only) channel.
In any case the the support for micromamba should be honoured as described in the docs
bentsherman
commented
1 year ago Mambaforge is miniforge with mamba as the default conda backend. Not sure why it says conda-forge is the only channel because it allows any channels.
pditommaso
commented
1 year ago Unable to replicate. Nextflow uses micromamba with the provided test case
Caused by:
Failed to create Conda environment
command: micromamba env create --prefix /Users/pditommaso/Projects/921e3a8aa32efe22bd73fbe1cbdfd09b/work/conda/base-bda312ce1312db01eb3e5e99160dbdb1 --file /Users/pditommaso/Projects/921e3a8aa32efe22bd73fbe1cbdfd09b/env.yaml
status : 127
message:
bash: micromamba: command not foundI think there is still a bug here though:
The problem was never that the conda builder doesn't use micromamba, but that the bash wrapper uses conda info to activate the environment. If you happen to have conda installed it may work, but if not, I wouldn't expect it to work. This error would happen when the task is executed.
shenker
commented
1 year ago I can reproduce the two bugs discussed above:
1) --yes should be added to the micromamba create command. Otherwise one needs to manually set the envvar MAMBA_ALWAYS_YES=true when running nextflow.
2) The use of conda in the bash wrapper (as identified by @bentsherman above) needs to be replaced with micromamba. (A temporary workaround is to micromamba install conda, but nextflow should handle pure-micromamba installs, because pure-micromamba is probably the best practice for managing conda envs going forward.) To fix this, in line 434 of the above snippet, the conda/awk invocation can probably be replaced more simply by something like micromamba info|grep "env location :"|cut -d : -f 2.
Would be great to get a nextflow release that fixes these!
bentsherman
commented
1 year ago Thanks @shenker . Regarding the -y option, it looks like it is already added in most cases:
I think it is not added for conda env create because that command does not have a -y option. Is micromamba different?
shenker
commented
1 year ago Yes, micromamba env create accepts -y (it's confusing but the micromamba CLI is subtly different from the mamba/conda interfaces, the latter two being near-identical).
ewels
commented
4 days ago Should be solved in #4302 and released in 24.10.0, however the following error + log was reported in the nf-core Slack. Looks like Mamba is requesting confirmation for the mamba env create command?
Caused by:
Failed to create Conda environment
command: mamba env create --prefix /path/to/conda/environment/cache/directory/environment-9b0544c1a5024cf208947cbaff805d06 --file /root/.nextflow/assets/nf-core/rnaseq/./subworkflows/loc
al/prepare_genome/../../../modules/local/gtf_filter/environment.yml
status : 143
message:
Transaction
Prefix: /path/to/conda/environment/cache/directory/environment-9b0544c1a5024cf208947cbaff805d06
Updating specs:
- conda-forge::python=3.9.5
Package Version Build Channel Size
────────────────────────────────────────────────────────────────────────────
Install:
────────────────────────────────────────────────────────────────────────────
+ _libgcc_mutex 0.1 conda_forge conda-forge 3kB
+ _openmp_mutex 4.5 2_gnu conda-forge 24kB
+ ca-certificates 2024.8.30 hbcca054_0 conda-forge 159kB
+ ld_impl_linux-64 2.43 h712a8e2_2 conda-forge 669kB
+ libffi 3.3 h58526e2_2 conda-forge 53kB
+ libgcc 14.2.0 h77fa898_1 conda-forge 849kB
+ libgcc-ng 14.2.0 h69a702a_1 conda-forge 54kB
+ libgomp 14.2.0 h77fa898_1 conda-forge 461kB
+ libsqlite 3.46.0 hde9e2c9_0 conda-forge 865kB
+ libstdcxx 14.2.0 hc0a3c3a_1 conda-forge 4MB
+ libstdcxx-ng 14.2.0 h4852527_1 conda-forge 54kB
+ libzlib 1.2.13 h4ab18f5_6 conda-forge 62kB
+ ncurses 6.5 he02047a_1 conda-forge 889kB
+ openssl 1.1.1w hd590300_0 conda-forge 2MB
+ pip 24.3.1 pyh8b19718_0 conda-forge 1MB
+ python 3.9.5 h49503c6_0_cpython conda-forge 29MB
+ readline 8.2 h8228510_1 conda-forge 281kB
+ setuptools 75.3.0 pyhd8ed1ab_0 conda-forge 780kB
+ sqlite 3.46.0 h6d4b2fc_0 conda-forge 860kB
+ tk 8.6.13 noxft_h4845f30_101 conda-forge 3MB
+ tzdata 2024b hc8b5060_0 conda-forge 122kB
+ wheel 0.44.0 pyhd8ed1ab_0 conda-forge 59kB
+ xz 5.2.6 h166bdaf_0 conda-forge 418kB
+ zlib 1.2.13 h4ab18f5_6 conda-forge 93kB
[293/695]
Summary:
Install: 24 packages
Total download: 46MB
────────────────────────────────────────────────────────────────────────────
Confirm changes: [Y/n]mamba version is 2.0.3
Rocky Linux 8.7 x86_64
jemma-nelson
commented
2 days ago You may be able to use the workaround described here: https://github.com/nextflow-io/nextflow/issues/4161#issuecomment-1709028041 - set export MAMBA_ALWAYS_YES=true in your shell environment. (if you are not using bash, the process for defining/exporting a environment variable may be different).
Bug report
Expected behavior and actual behavior
According to the docs, the useMicromamba directive should cause Nextflow to use Micromamba instead of Conda.
When using this option, the nextflow process fails to run, because the generated
.command.runfile still usesconda.Steps to reproduce the problem
On a machine with micromamba installed, but not conda:
Program output
The .nextflow.log file, as well as the generated .command.* files are in the gist referenced above. (e.g: https://gist.github.com/jemma-nelson/921e3a8aa32efe22bd73fbe1cbdfd09b#file-nextflow-log)
Program output:
Environment
Additional context
I mentioned this in slack last week, but did not get any attention. https://nextflow.slack.com/archives/C02T98A23U7/p1690325743801519
As near as I can tell from examining the source code, this configuration option cannot work, nor can the
useMambaoption, as nextflow always emitsconda info --jsonfor conda activation.You may also want to add
--yesto themicromamba createcommand, because ifexport MAMBA_ALWAYS_YES=trueis not set, environment creation will hang indefinitely.