search

jax-ml / ml_dtypes

A stand-alone implementation of several NumPy dtype extensions used in machine learning.
Apache License 2.0
191 stars 25 forks source link

Package Requirements Looks Inconsistent #155

Closed daskol closed 3 months ago

daskol commented 3 months ago

Package distribution dependencies project.dependencies requires NumPy 1.x while build stage requires NumPy 2.x in build-system.requires. Is it a typo? Are actual build deps the following?

[build-system]
requires = [
    # TODO(phawkins): update this to 2.0.0 after its release.
    "numpy>=1.26.0",
    "setuptools~=68.1.0",
]
jakevdp commented 3 months ago

This is deliberate. In order to be compatible with both NumPy 1.X and 2.X, a package must be built against the NumPy 2.X ABI. However at runtime, we still support NumPy 1.X.

daskol commented 3 months ago

Can we still build it with NumPy v1 for vendored packages (see details)?

jakevdp commented 3 months ago

You can build with an older NumPy version, but then the built artifacts will not be compatible with NumPy 2.0.

This is a very similar issue to #51: we need to build with particular NumPy versions in order for the built artifacts to satisfy the declared runtime dependency requirements. If you are working in a context where you don't need to satisfy the declared runtime version requirements, you can build with a different NumPy version.

jakevdp commented 3 months ago

I suspect you'll find something similar in the coming year or two for virtually every Python package that depends on NumPy's ABI. For example, here's scipy pinning NumPy 2.0 during build: https://github.com/scipy/scipy/blob/c109bb1c4d1560e01058307260032d639da9feb6/pyproject.toml#L25-L31

daskol commented 3 months ago

Yeah, I get it. It turns out that NumPy made a note on migration and binary compatibility.

Suspect you'll find something similar in the coming year or two for virtually every Python package

Totally agreed. Many thanks.