Update to conda 22.9 with tests for boa compatibility

hmaarrfk commented 1 year ago

Explicit test for conda/mamba/boa compatibility for Mambaforge
Reduced test matrix for Mambaforge-pypy3 on emulated architectures (aarch64 and ppc64le) -- ensures CI pass in time?
Updated micromamba on osx

Checklist

[ ] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase code>@<space/conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

hmaarrfk commented 1 year ago

I think anaconda is just blocking us from downloading the package index:

23:02 $ wc -l /home/mark/mambaforge/pkgs/cache/a08d06e9.json
2311247 /home/mark/mambaforge/pkgs/cache/a08d06e9.json
(base) ✔ ~/git/feedstocks/miniforge [fixup_2209|✔]
23:02 $ wc -l /home/mark/mambaforge/pkgs/cache/568c2e3d.json
7067307 /home/mark/mambaforge/pkgs/cache/568c2e3d.json

about 7 million lines, not 3000.

hmaarrfk commented 1 year ago

@wolfv are you able to help look into why when I try to install boa, it seems to be unable to find packages.

For reference, this is a similar PR that simply removes to boa install tests.

isuruf commented 1 year ago

The package cache is the one with only the dependencies of the packages in the installer (in this case python, conda and mamba) so that the packages inside the installer can be installed. These caches should have an old timestamp so that when new packages are installed, this cache is invalidated. Not sure why that is.

isuruf commented 1 year ago

You need https://github.com/conda/constructor/commit/85bca74e0179c94beb9c58438e42730dde05797e

wolfv commented 1 year ago

@hmaarrfk is there still something that I could do to help here?

hmaarrfk commented 1 year ago

If you can help us understand why the following index cache isn't being invalidated that would be great:

cat /root/miniforge/pkgs/cache/09cdf8bf.json
{"_mod": "Mon, 07 Jan 2019 15:22:15 GMT", "_url": "https://conda.anaconda.org/conda-forge/noarch",
  "_etag": "W/\"ded6fb8204936e8acedca121bd4b42e3\"",
  "_cache_control": "public, max-age=1200",

I'm not sure what "max-age" refers to.

hmaarrfk commented 1 year ago

@wolfv My latest modification shows that in fact, this seems to be a peculiarity that is observed with mamba and not conda.

It seems at least that mamba is holding on to the cache for too long, or conda ignores it entirely. I'm not too sure.

isuruf commented 1 year ago

Looks like mamba doesn't look at _mod timestamp in the json and instead looks at the timestamp in the file attributes. See https://github.com/mamba-org/mamba/blob/e3ab841b2c1349afb75af156eee88fc3a1bc125e/libmamba/src/core/subdirdata.cpp#L269

hmaarrfk commented 1 year ago

wow. excellent work!

isuruf commented 1 year ago

See https://github.com/conda/constructor/pull/579

hmaarrfk commented 1 year ago

I updated my branch. lets see.

hmaarrfk commented 1 year ago

We might have to trim down some of the test matrix.

hmaarrfk commented 1 year ago

Thank you for your help Isuru!

hmaarrfk commented 1 year ago

Seems like the thorough check is just a little too much on these emulated architectures.

hmaarrfk commented 1 year ago

Maybe with your fix i can remove the heavy handed skipping of the tests.

hmaarrfk commented 1 year ago

Thanks again!

jakirkham commented 1 year ago

Thank you both! 🙏

Are we comfortable publishing this release as final? Or what still needs to be done to get to that point?

hmaarrfk commented 1 year ago

Honestly, i'm lost in the number of assets generated. So if somebody can confirm that 74 i the correct count, we can publish.

isuruf commented 1 year ago

Yes, 74 is correct.

jakirkham commented 1 year ago

Noticed 2 releases showed up:

https://github.com/conda-forge/miniforge/releases/tag/22.9.0-0 (marked pre-release)
https://github.com/conda-forge/miniforge/releases/tag/4.14.0-2 (marked latest)

Which one should we be using?

hmaarrfk commented 1 year ago

I guess 22.9.0. They both point to the same sha. I think the version was always meant to follow the conda version, and as such, should be 22.9.0

jakirkham commented 1 year ago

Sounds good. Updating the Docker images with PR ( https://github.com/conda-forge/docker-images/pull/224 )

isuruf commented 1 year ago

You can use either. Miniforge3-4.14.0-2 gives you conda=4.14.0 and Miniforge3-22.9.0-0 gives you conda=22.9.0.

nyngwang commented 1 year ago

@isuruf Out of curious why the version "jumps" from 4.x to 22.x on the conda side. (I thought I was a time traveler when I saw the version jumped like that)

Appreciate your reading and I'm sorry for commenting on an old issue.

hmaarrfk commented 1 year ago

Jus following conda https://github.com/conda/conda/releases/tag/22.9.0

conda-forge / miniforge

Update to conda 22.9 with tests for boa compatibility #361