Sporadic failures using `mamba-version: "*"`

victorlin commented 1 year ago

(originally from https://github.com/nextstrain/augur/issues/1125)

Description

I'm aware that using mamba-version is an experimental feature, and some instability can be expected.

However, I've noticed that it's been failing more often than usual on at least two repos managed by my team @nextstrain.

The error is not very useful:

##[error]The process '/usr/share/miniconda/condabin/conda' failed with exit code 1

I've compared a failing run with a successful re-run of the same workflow. There are some notable differences in the output of conda config --show:

75d74
< allowlist_channels: []
110c109
< default_python: 3.10
---
> default_python: 3.9
123a123
> experimental_solver: classic
125d124
< fetch_threads: 5
137d135
< number_channel_notices: 5
165d162
< solver: classic
181a179
> whitelist_channels: []
197c195
< no change     /usr/share/miniconda/lib/python3.10/site-packages/xontrib/conda.xsh
---
> no change     /usr/share/miniconda/lib/python3.9/site-packages/xontrib/conda.xsh

I do not understand why there are differences in Python versions and other conda config between two runs of the same workflow.

Known Workarounds

1. Use Mambaforge

Remove mamba-version.
Add miniforge-variant: Mambaforge (and optionally miniforge-version).

Example: https://github.com/nextstrain/cli/commit/4a764976519ca5c540c745463548a9d883eae079

2. Use the new libmamba solver

Note that this workaround can introduce a separate issue when used on windows-2022: https://github.com/conda/conda/issues/12161.

Remove mamba-version.

Run this after the setup-miniconda step:

- run: |
    conda install -n base conda-libmamba-solver
    conda config --set solver libmamba

Replace mamba install with conda install.

Example: https://github.com/nextstrain/augur/commit/344ed514dc7e70a6ce70307f470ea2745eb82923

joamatab commented 1 year ago

observing the same today

https://github.com/gdsfactory/gdsfactory/actions/runs/3906659862/jobs/6675080754

joshuailevy commented 1 year ago

Also observing this. https://github.com/andersen-lab/Freyja/actions/runs/3907050506. Thanks @victorlin for posting the workaround.

jongyoul commented 1 year ago

I also found the same error on https://github.com/apache/zeppelin/actions/runs/3901061183/jobs/6665542729

traversaro commented 1 year ago

If you are not interested in using defaults, a possible strategy is to use mambaforge instead of miniconda, see for example:

victorlin commented 1 year ago

@traversaro thanks, I've updated the list of workarounds in the issue description and put that one first since it is the more popular one.

victorlin commented 1 year ago

If you are not interested in using defaults, a possible strategy is to use mambaforge instead of miniconda

Note that defaults is included in the generated .condarc regardless of miniforge-variant. To truly get away from it, you'll have to use nodefaults, --override-channels, or conda config --remove channels defaults. See #207 for more info.

kthyng commented 1 year ago

I have been having the following error with ubuntu-latest:

PackagesNotFoundError: The following packages are missing from the target environment:
    - defaults/linux-64::conda==22.11.1=py310h06a4308_4[md5=07e2e8a698cf1e7f580b9efcae72184e]

with this in my test.yaml

      - uses: conda-incubator/setup-miniconda@v2
        with:
          mamba-version: "*" # activate this to build with mamba.
          channels: conda-forge, defaults # These need to be specified to use mamba
          channel-priority: true
          environment-file: ci/environment-py${{ matrix.python-version }}.yml

          activate-environment: test_env_cmocean
          use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly!

(example PR with errors earlier: https://github.com/matplotlib/cmocean/pull/88)

and workaround 1 above worked for me!

rosepearson commented 1 year ago

This is affecting me too on https://github.com/rosepearson/GeoFabrics/actions/runs/3925803401/jobs/6710960393. I'll look at work around 1 thanks,

Zeitsperre commented 1 year ago

This appears to be affecting me as well: https://github.com/CSHS-CWRA/RavenPy/actions/runs/3933458237/jobs/6727178753

lewisgross1296 commented 1 year ago

Solution 3 from the work arounds fixed our CI in this case. Thanks @victorlin

padix-key commented 1 year ago

I created a minimum working example for this issue using different OS and Python versions: https://github.com/padix-key/gha-mamba

In this setup this issue only affects Ubuntu builds: https://github.com/padix-key/gha-mamba/actions/runs/4044454081

goanpeca commented 1 year ago

Hi @padix-key thanks for the example!

padix-key commented 5 months ago

In an updated version of https://github.com/padix-key/gha-mamba (updated setup-miniconda and platform version), this issue did not appear anymore. Can this issue be closed?

conda-incubator / setup-miniconda