ComputeCanada / wheels_builder

10 stars 2 forks source link

Update/default numpy #77

Closed ccoulombe closed 1 year ago

ccoulombe commented 1 year ago

Remove the pinned numpy and rely on the oldest-supported-numpy module we now have.

ccoulombe commented 1 year ago

@ostueker Any comments on this as well?

mboisson commented 1 year ago

That will break whenever oldest-supported-numpy is not up to date with the newest version of Python (like right now). I think if we do this, we need to filter the versions of python that we build for based on what oldest-supported-numpy supports.

Also, could that create issue with existing wheels ? Lots of existing wheels have been built with 1.21.2, but that version is more recent than the oldest supported version for python <3.10

mboisson commented 1 year ago

If we load python-build-bundle, we should not install/upgrade pip/setuptools&co in the setup function

ostueker commented 1 year ago

According to NEP 29 numpy 1.21 should now be the oldest supported version.

What's the benefit of this over setting NUMPY_DEFAULT_VERSION=1.22.4 on Jun 23, 2023 (the date to drop support for NumPy 1.21 according to NEP 29)?

mboisson commented 1 year ago

According to NEP 29 numpy 1.21 should now be the oldest supported version.

What's the benefit of this over setting NUMPY_DEFAULT_VERSION=1.22.4 on Jun 23, 2023 (the date to drop support for NumPy 1.21 according to NEP 29)?

for what it's wort, 1.22.4 is not support for Python 3.11, the oldest supported numpy for python 3.11 is 1.23.2, so that would not work @ostueker

ccoulombe commented 1 year ago

Also, could that create issue with existing wheels ? Lots of existing wheels have been built with 1.21.2, but that version is more recent than the oldest supported version for python <3.10

Yes it could but I think the risk is limited. This applies to newly built wheels from now on. It will also install whatever numpy version other packages may need.

mboisson commented 1 year ago

I think this is ready for merging. I have been using this for about a week, rebuilding tons of wheels without major issues

ccoulombe commented 1 year ago

Installing the latest scipy wheel by default has sneakily been building wheels with the current most recent numpy (1.24).

Since the latest scipy needs numpy<1.27.0,>=1.19.5, pip installs a numpy that is greater than the oldest-supported-numpy for python 3.8 and 3.9 (1.17.3, 1.19.3), resulting in the wheel's numpy requirement being adjusted to 1.24.

Removing the scipy installation from PYTHON_DEPS_DEFAULT effectively fixes this and use the oldest-supported-numpy for the current python.

This might create installation issues for some users/wheels.

I've found 17 wheels with a requirements numpy>=1.24.

Fixed in https://github.com/ComputeCanada/wheels_builder/commit/4fbf7db5c48fa888c62a082d22ace1f4ae923ec6