Open jakirkham opened 1 year ago
Just so it doesn't get lost in the notifications, something seems to be off with the python 3.12 pinning. I know it is confusing to do dual migrations at the same time so I'm not sure there is an easy answer https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/5790#discussion_r1590007898
Yeah, thanks for the note @hmaarrfk & @xhochy. In previous numpy bumps, we always did them for the open python 3.x & pypy migrations as well, and this wasn't an issue. Here however, bumping the 3.12 migrator gets immediately picked up by a given feedstock rerendering and so runs into breakage if that package isn't yet numpy 2.0-compatible (basically all of them).
I undid the changes to the 312 migrator to unbreak this, but I'm kind of unsure how to square this circle now. I think for now the least bad option is to just accept that we don't get numpy 2.0 builds for 3.12. Alternatives might be:
python:
as being updated.With 1., the numpy2 migrator would look like
numpy:
- 1.22 # no py38 support for numpy 2.0
- 2.0
- 2.0
- 2.0
- 2.0
python:
- 3.8.* *_cpython
- 3.9.* *_cpython
- 3.10.* *_cpython
- 3.11.* *_cpython
- 3.12.* *_cpython
python_impl:
- cpython
- cpython
- cpython
- cpython
- cpython
while the python312 one would stay untouched.
Next snag: bot isn't opening PRs ๐
We are 87% of the easy through the Python 3.12 migration. Maybe we are close to closing this out? Perhaps with a little effort?
Wonder if there are other configuration options available to us if we make the NumPy 2 migration depend on the Python 3.12 migration? IOW the "no ice cream until you eat your vegetables" approach ๐
Wonder if there are other configuration options available to us if we make the NumPy 2 migration depend on the Python 3.12 migration?
Yeah, there's wait_for_migrators
, but this is not exactly well tested and it would still run into the issue that we'd have to override the whole zip, which would then open PRs for all non-noarch python feedstocks.
We are 87% of the easy through the Python 3.12 migration.
The migrator status page is somewhat misleading on this - it reports the percentage of "PRs opened", not "PRs closed". So we're actually at ~67% with the 3.12 migration, with about ~900 feedstocks still to go. That's IMO quite a bit too early to close?
I guess the only nice alternative to points 1. & 2. above would be to implement something in cf-scripts like skip_unless_present:
, where we could say that a feedstock should only be migrated if it depends on numpy
, in order to force it to ignore the feedstocks that wrongly get picked up due to the presence of the python:
key (that we need to override the zip) if we do point 1.
I guess there aren't really any other zips that are affected in this way, so it'd be a bit tailor-made for the numpy/python situation. From that POV, maybe we could also just deal with the hassle of closing the superfluous PRs this one time, because once we're on numpy 2.0 (and have dropped 3.8 around October), we can completely remove numpy from the python zip[^1] going forward.
[^1]: except PyPy, but that's a separate discussion for various reasons.
FWIW, I think we need to migrate 3.12 as well, as we won't simply be able to switch this one later (due to lots of missing dependencies then).
So if we don't do it as part of the overall numpy 2.0 migration, we'd need a separate migrator (now or later) that catches up 3.12 specifically. I feel that a second such migration wastes more time than it would do to deal with the overzealous PRs to python-but-not-numpy-related feedstocks (which by themselves aren't harmful even if merged).
After some discussion, we decided to soft-close the Python 3.12 migration. Meaning that the Python 3.12 migration is still running, but we now also include Python 3.12 in the global pinning. As a result, re-rendering a feedstock could add Python 3.12 (even if its dependencies are not available on Python 3.12 yet). Meaning the feedstock would get Python 3.12 builds that would fail. These could simply be ignored for now. Based on the Python 3.12 migration status, this is only relevant for ~11% of feedstocks (admittedly that is ~500 feedstocks). The rest have either already been migrated or have a migration PR
With Python 3.12 now addressed, we restarted the NumPy 2 migration today ( https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/5851 ). Here is the status of NumPy 2 migration. It may take it a bit to start adding PRs to all the feedstocks that can now take a PR. So please be patient ๐
Thanks all! ๐
statsmodels still seeling some NumPy 1.22 builds on my NumPy 2 migration. Have these not been migrated yet? Or will they never be migrated (and so need to be skipped (and if so, any pointers would be helpful)).
These all fail. All of the NumPy 2 targeted build pass (as do a few NumPy 1.22, although most of these should be disabled when the recipe is rerendered).
These all fail.
Those are all the PyPy builds, which is not participating in the numpy 2.0 migration (IOW, they should run just as before, but have broken for unrelated reasons). I haven't opened an issue yet, but PyPy support is (unfortunately!) being wound down, so don't hesitate to skip those. I did just that in the statsmodels PR even before seeing your comment here... :)
JFYI NumPy is planning to release 2.0.0 on June 16th
xref: https://github.com/numpy/numpy/issues/24300#issuecomment-2125643448
More updates:
NumPy is currently working on a NumPy 2.0 release, which is planned to come out later this year. Here are the current (draft) release notes. Also here's the upstream tracking issue ( https://github.com/numpy/numpy/issues/24300 ), and ecosystem compatibility tracker.
Some questions worth discussing:
numpy
pins in packages ( https://github.com/conda-forge/conda-forge-repodata-patches-feedstock/issues/516 )?Todos:
cc @conda-forge/core