search

esmf-org / esmf

The Earth System Modeling Framework (ESMF) is a suite of software tools for developing high-performance, multi-component Earth science modeling applications.
https://earthsystemmodeling.org/
Other
149 stars 70 forks source link

ESMF shared library fails to load properly with `setuptools>65.5` #105

Open valeriupredoi opened 1 year ago

valeriupredoi commented 1 year ago

Hi guys, apologies in advance if this is a duplicate of another issue (I looked through the current ones though) and/or if this is already one someone's radar screen, and if this is the wrong way to approach you! We have started seeing issues with the ESMF shared library with setuptools>65.5: see our internal issue here https://github.com/ESMValGroup/ESMValCore/issues/1824, the test fail and the root cause pointed out to me by my colleague @bouweandela at https://github.com/numpy/numpy/issues/22623 - with a TL;DR being numpy's distutils if used, it'll lead to breakages since setuptools will not be supported anymore (they're moving to Meson). I was wondering what you guys have in mind to avoid this - we, having esmf as a dependency, could always pin setuptools but we thought we'd reach out to you for a more permament solution (actual solution vs a workaround that'd be the pin). Cheers muchly for your good work :beer:

rokuingh commented 1 year ago

@valeriupredoi my apologies for the late reply, I have been on leave for the last couple months. I just looked into this a bit and I cannot reproduce the error on my local system (linux). From your test output it seems like you may have an older version of the code because the line number of your exception doesn't match what I am seeing for 8.4.0 and develop. Would you please verify that you are using a version of the code that is at version 8.4.0 or later?

esmpy doesn't use numpy.distutils, but based on the comments from the ticket you linked, the setuptools >65 seems to also break numpy.ctypes.load_library. This function is used for the OSX build, and unfortunately I don't have a mac for development to verify this. Some of the other ESMF group members have macs, and once I can remind myself how reestablish myself on our mac server I will try to find a workaround.

I am only working a couple hours a day right now so this may not happen quickly. If you need to move forward in a more stable manner then I would recommend submitting a pr to esmpy-feedstock that pins setuptools<65. (We may also decide to pin esmpy to setuptools <65, but i don't think it would cause a version clash if do go that route).

Thank you so much for bringing this to our attention! We really appreciate helpful and thorough user code reviews and bug reports like you have provided here. Cheers to that and happy new year!

valeriupredoi commented 1 year ago

Hi @rokuingh cheers for your reply and Happy New Year! Indeed, we are using a slightly older version of esmpy since we can't yet use 8.4 (for internal and other dependency conflicts reasons, but we'll get there eventually). I might go the pin route in the feedstock indeed, but I'll wait until we're fully up to date at our end of the environment (ie use 8.4 or higher, as it comes online). Cheers and feel free to close this issue if you think it served its purpose :beers: