conda / conda-lock

Lightweight lockfile for conda environments
https://conda.github.io/conda-lock/
Other
459 stars 102 forks source link

Tests using micromamba are flaky #416

Closed maresb closed 1 year ago

maresb commented 1 year ago

Checklist

What happened?

Most test suite runs now fail on tests which have conda_exe="micromamba".

Example

___ test_run_lock_with_pip_environment_different_names_same_deps[micromamba] ___
...
self = CompletedProcess(args=['/home/runner/.local/share/ensure-conda/micromamba', 'create', '--prefix', '/home/runner/work/_...dout='', stderr="critical libmamba Unable to read repo solv file 'conda-forge/linux-64', error was: not a SOLV file\n")

Example

_______________ test_run_lock_with_input_hash_check[micromamba] ________________
...
self = CompletedProcess(args=['/home/runner/.local/share/ensure-conda/micromamba', 'create', '--prefix', '/home/runner/work/_...tderr="critical libmamba Unable to read repo solv file 'conda-forge/linux-64', error was: unexpected EOF, depth = 3\n")

Example

 ____________________ test_virtual_packages[micromamba-env] _____________________
...
self = CompletedProcess(args=['/home/runner/.local/share/ensure-conda/micromamba', 'create', '--prefix', '/home/runner/work/_...dout='', stderr="critical libmamba Unable to read repo solv file 'conda-forge/linux-64', error was: not a SOLV file\n")

Example

___________________ test_run_lock_uppercase_pip[micromamba] ____________________
...
self = CompletedProcess(args=['/Users/runner/Library/Application Support/ensure-conda/micromamba', 'create', '--prefix', '/Us...stdout='', stderr="critical libmamba Unable to read repo solv file 'conda-forge/noarch', error was: not a SOLV file\n")

Example

__________________________ test_run_lock[micromamba] __________________________
...
Could not lock the environment for platform osx-64
    Command: ['C:\\Users\\runneradmin\\AppData\\Local\\ensure-conda\\ensure-conda\\micromamba.exe', 'create', '--prefix', 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmp1mcect_2\\prefix', '--dry-run', '--json', '-v', '--override-channels', '--channel', 'conda-forge', '--channel', 'defaults', '--channel', 'D:\\a\\_temp\\tmpaa9x5b_d', 'zlib']
    STDERR:
critical libmamba Unable to read repo solv file 'conda-forge/noarch', error was: unexpected EOF, depth = 3

UPDATE

I switched to the setup-micromamba action, and now I'm unable to reproduce the previous issues. Instead, with v1.4.3 I'm consistently seeing:

Example

menuinst called from non-root env C:\Users\runneradmin\micromamba\envs\test-env
mariusvniekerk commented 1 year ago

cc @wolfv These seem to occur rather commonly when testing conda-lock in conjunction with micromamba

maresb commented 1 year ago

I wasn't seeing this previously, so I have a suspicion that something changed in a recent release.

maresb commented 1 year ago

I'm going to close this and try to summarize the findings.

At least one mode of failure was the No package record found! warning as described in #448. We implemented several mitigations. One in #448 was to add extract_json_object in our json.loads command in order to ignore this warning. Another was to add fn to the conda-meta/ spec. Finally there was https://github.com/mamba-org/mamba/pull/2662 to print the warning to stderr instead of stdout.

We have not done a detailed analysis to determine which mitigations were effective, but we noticed that a pin was no longer necessary after adding extract_json_object.

Please don't be shy to comment in case I missed anything.

CC @riccardoporreca