search

mamba-org / setup-micromamba

GitHub Action to set up micromamba
MIT License
97 stars 15 forks source link

`micromamba list` runs indefinitely on Micromamba v2.0.0 #225

Open mrvisscher opened 6 days ago

mrvisscher commented 6 days ago

Our Github runners timed out last night running micromamba list. Fix for us right now is pinning the last Micromamba release before v2.0.0 but something is definitely up. Runner logs

pavelzw commented 6 days ago

i think this is an issue for the mamba repository, right @jjerphan @JohanMabille? could you transfer it?

mikemhenry commented 6 days ago

Related to https://github.com/mamba-org/micromamba-releases/issues/58 probably, see if explicitly pulling in 2.0 fixes it

      - name: Setup micromamba
        uses: mamba-org/setup-micromamba@v1.9.0
        with:
          micromamba-version: '2.0.0-0'
bhperry commented 5 days ago

pinning 2.0.0-0 did not fix for me. It seems like it's always using the same micromamba version no matter what I pin.

I see this in the logs Will use pre-installed micromamba at /home/runner/micromamba-bin/micromamba

How do I get it to not use the pre-installed version?

bhperry commented 5 days ago

Looks like there is a pre-installed version on the runner, and setup-micromamba will completely ignore your pinned version if that exists.

Also had to got through and delete all of the github actions caches for micromamba because otherwise even with the correct binary, it would still fail to list when pulling the previous cache.

bhperry commented 5 days ago

This is what fixed it for me

      - name: Set up micromamba
        uses: mamba-org/setup-micromamba@v1.9.0
        with:
          # https://github.com/mamba-org/setup-micromamba/issues/225
          micromamba-version: 1.5.10-0
          micromamba-binary-path: /home/runner/micromamba-bin-versioned/micromamba

And then delete every previous micromamba cache from github actions cache

jjerphan commented 2 days ago

I have not been able to reproduce the error yet, can you open an issue upstream with a specification of your environment? Thank you a lot.

maresb commented 2 days ago

@jjerphan, please find a minimal example in https://github.com/maresb/setup-micromamba-gh225

jjerphan commented 2 days ago

I still cannot reproduce the error with micromamba 2.0.0.

maresb commented 2 days ago

@jjerphan are you trying to reproduce it locally or on GHA? If GHA then is my example deficient in any way to reproduce?

jjerphan commented 2 days ago

We observe errors with GHA, as if it picks release candidates builds instead of the actual release builds.

pavelzw commented 2 days ago

are you sure it uses rcs? it should just use the latest version from micromamba-releases

jjerphan commented 2 days ago

The latest build are not the actual latest one (we do not understand why).

pavelzw commented 2 days ago

are you sure?

❯ curl -Ls https://github.com/mamba-org/micromamba-releases/releases/latest/download/micromamba-linux-64 | sha256sum
b1b8b7a445b5bac0d7417c45f2eb3a29280567975d0185e7d9f4c5a94e2ef784  -
~
❯ curl -Ls https://github.com/mamba-org/micromamba-releases/releases/download/2.0.0-1/micromamba-linux-64 | sha256sum
b1b8b7a445b5bac0d7417c45f2eb3a29280567975d0185e7d9f4c5a94e2ef784  -
~
❯ curl -Ls https://github.com/mamba-org/micromamba-releases/releases/download/2.0.0rc6-0/micromamba-linux-64 | sha256sum
34354f0feebbb450905fc0fe55d4227c2c7a69eab3587a5b905d1d1dfc2d3ca7  -

this looks fine to me 🤔

or is the 2.0.0-1 release already faulty?

jjerphan commented 2 days ago

We have had observed problem with the "latest" builds when the version of micromamba was not explicitly specified (hence our recommendation to pin 2.0.0-0).

pavelzw commented 2 days ago

pinning 2.0.0-0 did not fix for me

https://github.com/mamba-org/setup-micromamba/issues/225#issuecomment-2377465914

bhperry commented 2 days ago

Explicitly specifying a version is not enough, because GHA seems to provide pre-installed micromamba binaries which will always be used if present at micromamba-binary-path regardless of version.

I also observed that a binary with a fixed version will still hang indefinitely when listing a cached environment created by a faulty binary, so that can lead to further confusion over which versions are fixed vs. not.

pavelzw commented 2 days ago

the "latest" build always uses the one with the latest badge.

image

as long as this is correctly specified in micromamba-releases, this shouldn't be the problem

pavelzw commented 2 days ago

because GHA seems to provide pre-installed micromamba binaries which will always be used if present

this is news to me 🤔

bhperry commented 2 days ago

Me as well, was not expecting it. And it is not listed under the software for the ubuntu-24.04 runner image, so I'm not 100% sure where it's coming from. But the logs shows that setup-micromamba was using a pre-installed binary, and I could not get it to use a specific version until I changed the bin path.

jmccreight commented 1 day ago

micromamba-version: 1.5.10-0 works for me, but 2.0.0-0 does not. I didnt need micromamba-binary-path which is nice because I'm testing all platforms and the path looks linux specific.

Thanks for all the notes!

orbeckst commented 17 hours ago

micromamba-version: 1.5.10-0 worked for me after deleting all GH caches.

jjerphan commented 7 hours ago

I can reproduce the issue with 2.0.1 and the following environment:

name: repro
channels:
  - conda-forge
dependencies:
  - python=3.10
  - httpx>=0.25.0
  - pandas[version=">=0.25.2,<3"]
  - pytz
  - cartopy>=0.21
  - cartopy_offlinedata
  - check-manifest
  - geopandas
  - interrogate
  - joblib
  - jupyter
  - nbclassic
  - nbconvert
  - nbsphinx
  - netcdf4
  - pendulum>=2.0.1
  - pooch
  - pre-commit
  - pyarrow
  - pytest
  - pytest-cov
  - pytest-sugar
  - pytest-vcr
  - regionmask>=0.9
  - scitools-iris>=3.3.0
  - setuptools_scm
  - sphinx
  - twine
  - types-pytz
  - types-requests
  - wheel
  - xarray

from https://github.com/ioos/erddapy/actions/runs/11124942088/job/30911568849?pr=353.

ricardoV94 commented 4 hours ago

Does not seem to be specific to v2, also failed with v1 here for us: https://github.com/pymc-devs/pytensor/actions/runs/11141222416/job/30961688938?pr=1008

pavelzw commented 4 hours ago

might be because of caching? https://github.com/mamba-org/setup-micromamba/issues/225#issuecomment-2387213400

maresb commented 4 hours ago

Does not seem to be specific to v2, also failed with v1 here for us: https://github.com/pymc-devs/pytensor/actions/runs/11141222416/job/30961688938?pr=1008

False alarm, I just chatted with @ricardoV94, and he was referring to v1 of the setup-micromamba GHA rather than v2.0.1 of micromamba. So the problem is indeed related in his case to the v2 release of micromamba since he has micromamba-version: "latest".

ricardoV94 commented 3 hours ago

Thanks for clarifying. And to answer the other comment if still relevant I tried deleting the caches related to micromamba and that still hanged

bhperry commented 1 hour ago

Thanks for clarifying. And to answer the other comment if still relevant I tried deleting the caches related to micromamba and that still hanged

Try 1.5.10-0 from a clean cache

ricardoV94 commented 1 hour ago

Thanks for clarifying. And to answer the other comment if still relevant I tried deleting the caches related to micromamba and that still hanged

Try 1.5.10-0 from a clean cache

Yes that works