search

tskit-dev / tskit

Population-scale genomics
MIT License
147 stars 70 forks source link

Numpy 2.0 Python #2894

Closed jeromekelleher closed 1 month ago

jeromekelleher commented 5 months ago

Numpy 2.0 is about to drop: https://github.com/numpy/numpy/issues/24300

There is a ruff plugin which can check this as lint

This might be a motivation to switch to ruff

Would anyone like to try installing numpy 2.0 and see how the experience goes with running the test suite? Hopefully the Python level changes are small. See also #2893

jeromekelleher commented 1 month ago

This is going to become an issue soon - we need to either verify that we work OK with numpy v2 or release a version that pins to numpy<2.

I've had a quick look using numpy 2.0.0rc2 and hit some issues.

jeromekelleher commented 1 month ago

I've had a bit of a fiddle with this, and I've not been able to successfully build and import _tskit or the lwt_module (in the example_c_module). There's some funkiness going on with the build-time requirements and the updated numpy C API.

I don' think there's any fundamental problem here, but until we have numpy 2.0 packages easily installable across a range of platforms I don't think there's any point in trying to nail it down.

So, I vote that we push out an updated version of tskit that specify numpy<2 that should at least avoid some initial breakage when the new version drops. I guess we should do the same for msprime, but hopefully the tskit pin should be enough?

Any thoughts @benjeffery?

benjeffery commented 1 month ago

I've had a go and got the module to import - the test suite is a whole other problem though with the chicken-and-egg msprime issues.

I am concerned that when we do switch our wheels built on numpy2 won't work if the user has numpy1 - and with many packages specifying numpy<2 we're going to cause conflicts.

As you suggest, best to just pin for now.

molpopgen commented 1 month ago

This transition will be painful for a while. I know that fwdpy11 and moments are both pinning to <2 for now. It is hard to coordinate relaxing that, and there will also be issues with scipy/numba/etc..

jeromekelleher commented 1 month ago

Conda forge packages seem to have gone through ok, so that might be a way forward in terms of CI anyway