ComputeCanada / software-stack

Repository to host issues relative to the Compute Canada software stack
12 stars 0 forks source link

Support Numpy 2.0 #155

Closed ccoulombe closed 2 months ago

ccoulombe commented 2 months ago

Roadmap to support and deploy Numpy 2.0

By adding numpy 2.0, we risk of :

  1. breaking modules whichs used oldest-supported-numpy and uses a scipy-stack for its numpy
  2. breaking of users environment where they install any version of numpy and use scipy-stack (or any module that uses it, eg arrow or opencv).
  3. breaking current wheels that were built with 1.x and not constrained to numpy<2.0
  4. breaking current jobs of users that do not specify versions or use a requirements file
ccoulombe commented 2 months ago

Numpy 2.1.1 + matplotlib 3.9.1 (built with oldest-supported-numpy), no 1.x compat

>>> import matplotlib

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.1.1 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "<stdin>", line 1, in <module>
....
AttributeError: _ARRAY_API not found
ImportError: numpy.core.multiarray failed to import

Numpy 2.1.1 + matplotlib 3.9.2 (built with numpy 2.1.1), 1.x compat

(32603) [coulombc@node2 wheels_builder]$ pip install numpy==1.26.4 matplotlib-3.9.2+computecanada-cp311-cp311-linux_x86_64.whl 
(32603) [coulombc@node2 wheels_builder]$ python -c 'import matplotlib'
(32603) [coulombc@node2 wheels_builder]$ 
mboisson commented 2 months ago

Cool, au moins numpy donne une erreur compréhensible à l'usager.

ccoulombe commented 2 months ago

numpy 2.1.1 and the wheelhouse are now synced to the systems.