search

desihub / desiutil

General DESI utilities, shell scripts, desiInstall, etc.
BSD 3-Clause "New" or "Revised" License
3 stars 9 forks source link

support a more recent (>=1.23) version of numpy? #205

Closed moustakas closed 6 months ago

moustakas commented 8 months ago

I am working with a student and @jdbuhler on building a JAX version of FastSpecFit. JAX requires numpy>=1.23 but desiutil (a FastSpecFit dependency) requires numpy<1.23-- https://github.com/desihub/desiutil/blob/main/setup.cfg#L36

Would it be possible to relax this numpy version cap via the appropriate code changes (which I would be willing to help with, since this is a blocking factor for our work)?

moustakas commented 8 months ago

This ticket is possibly related to #198, since it looks like distutils was deprecated in 1.23.0-- https://numpy.org/doc/stable/release/1.23.0-notes.html#deprecations

weaverba137 commented 8 months ago

I think #198 arose from unit tests of numpy>=1.23, but we can test again on a branch.

sbailey commented 8 months ago

I also recently tripped across this. #187 has the context for why we pinned to numpy<1.23. The issue is that astropy 5.0.x uses numpy.asscalar, which was dropped by numpy 1.23. So if you are using an old-ish astropy (5.0.x) then you also need an old-ish number (<1.23). But desiutil likely also works with the combination of a newer astropy + newer numpy, but I don't know how to express that double-dependency constraint for pip install / setup.cfg purposes.

It would likely work to install the astropy/numpy dependencies yourself to the versions you need, and then install desiutil with pip install --no-deps .... I don't know the version of astropy 5.x that is compatible with numpy >=1.23.

weaverba137 commented 8 months ago

I agree with Stephen's suggestion. I don't think there's anything intrinsic to desiutil that requires numpy<1.23, except that the pipeline was using astropy==5.0 for a while and that does not work with 1.23. However astropy==5.0.8 does seem to work.

Basically desiutil tests a narrow range of versions around what the pipeline is using. Outside of that, the test matrix starts to explode.

We already know that numpy 2.0 will cause issues, but we're also way behind on numpy<2.0. We're orbiting around numpy 1.22-1.23, meanwhile, numpy 1.26 is the current version.

And of course, we haven't really done much testing on Python 3.11, which is already not the most recent version.

weaverba137 commented 6 months ago

Note that #207 tests more recent versions of numpy, so we can now say that desiutil supports numpy >= 1.23. HOWEVER we are not supporting Python >= 3.11 at this time. Since Jura was run with Python 3.10, that's OK for now.

If there are still concerns about the range of versions supported, please comment. Otherwise I will consider #207 as having closed this issue.

weaverba137 commented 6 months ago

Closed by #207.