Support Numpy v2.0

[gnought]

Describe the issue

I am using Numpy v2.0. onnxruntime gives an incompatible error

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Is it possible to upgrade pybind 11>=2.12 in https://github.com/microsoft/onnxruntime/blob/0babc337259470a41038826954a1504505a3d5bf/cmake/deps.txt#L51

to

pybind11;https://github.com/pybind/pybind11/archive/refs/tags/v2.12.0.zip;8482f57ed55c7b100672815a311d5450858723fb

To reproduce

a simple python imports

import onnxruntime as ort
import numpy as np

Urgency

No response

Platform

Mac

OS Version

14.5

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.18.0

ONNX Runtime API

Python

Architecture

ARM64

Execution Provider

Default CPU

Execution Provider Library Version

No response

[aquitzia]

I got the same error:

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

The error is from this line. The compiler never evaluates the code. from onnxruntime import InferenceSession I'm not using numpy.

I'm running a Docker container built on an AWS Lambda image with --platform linux/amd64. I'm running it on MacOS 13.4 and requirements.txt lists: onnxruntime>=1.18.0

I'm guessing things are being changed right now because I could not build for a while, but now it's building and I'm getting the same error.

[snnn]

It is not enough. After this change we still see the error:

To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0

I am trying to understand the sentence. How to support NumPy 1.x while we use NumPy 2.0 at compile time? It sounds like there is a compile time macro we should use.

[snnn]

@mszhanyi, please help take a look

[aquitzia]

I'm also still getting the same error. Here's the stack trace:

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

from onnxruntime import InferenceSession
File "/var/task/onnxruntime/__init__.py", line 23, in <module>
from onnxruntime.capi._pybind_state import ExecutionMode  # noqa: F401
File "/var/task/onnxruntime/capi/_pybind_state.py", line 32, in <module>
from .onnxruntime_pybind11_state import *  # noqa
AttributeError: _ARRAY_API not found

[snnn]

I published the new packages to our nightly feed. It would be great if you can help me do some tests. You may install the package by running the following commands:

python -m pip install coloredlogs flatbuffers numpy packaging protobuf sympy
python -m pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ ort_nightly==1.19.0.dev20240619002

Please note the version number should be 1.19.0.dev20240619002

[johnnynunez]

how is it going?

[snnn]

We are discussing whether the upcoming 1.18.1 release should be built with Numpy 2.0 or staying with numpy 1.x and mark the package as incompatible with numpy 2.0 in its requirements.txt.

[snnn]

I anticipate the problem will be addressed in our nightly build in a few days. There is an open PR for this: https://github.com/microsoft/onnxruntime/pull/21085

[johnnynunez]

I anticipate the problem will be addressed in our nightly build in a few days. There is an open PR for this: #21085

But if I compile it manually, is it compatible?

[snnn]

onnxruntime 1.18.1 is published. If you need to use it with NumPy 2.0, please get a nightly package by using the following commands:

python -m pip install coloredlogs flatbuffers numpy packaging protobuf sympy
python -m pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ ort_nightly

[snnn]

But if I compile it manually, is it compatible?

Yes.

[maximilian22x]

Why this problem still apear, if I use numpy in the sql server maschine Learning.

[johnnynunez]

But if I compile it manually, is it compatible?

Yes.

done! Compiled and working, thank you

[snnn]

Why this problem still apear, if I use numpy in the sql server maschine Learning.

If you need to use ONNX Runtime with NumPy 2.0, please get a nightly package by using the following commands:

python -m pip install coloredlogs flatbuffers numpy packaging protobuf sympy
python -m pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ ort_nightly

[maximilian22x]

I use Python and numpy inside the SQL Server. Put I got this error if I use the newest Numpy

[vlasov01]

Hi, I've got the same issue. I've installed the nightly package. And now I'm getting the following error: C:\Users\serge\anaconda3\envs\ryzenai-1.1-20240702-230249\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'VitisAIExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'

[dkbarn]

It's a little confusing that this issue was closed as complete before the version of ONNX Runtime that supports NumPy 2.0 has actually made its way to PyPI. Currently it is only available as a nightly package. How are we to track the progress of it being released officially, and what is the expected ETA?

[johnnynunez]

It's a little confusing that this issue was closed as complete before the version of ONNX Runtime that supports NumPy 2.0 has actually made its way to PyPI. Currently it is only available as a nightly package. How are we to track the progress of it being released officially, and what is the expected ETA?

you can compile from the main using the documents on the page web

[maximilian22x]

I used this and still get the error on the sql server if i try to use Python on it. python -m pip install coloredlogs flatbuffers numpy packaging protobuf sympy python -m pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ ort_nightly

[snnn]

Which ONNX Runtime did you install? Can you run "python3 -m pip list | grep ^o" and give us the output?

[maximilian22x]

No it works, thanks man.

[ogencoglu]

Any updates here? Having numpy==2.0.1 and onnxruntime==1.18.1 in requirements throws a pip install error.