search

desihub / desisurvey

Code for desi survey planning and implementation
BSD 3-Clause "New" or "Revised" License
2 stars 7 forks source link

desisurvey broken with astropy 4.2 #134

Closed sbailey closed 3 years ago

sbailey commented 3 years ago

Multiple desisurvey unit tests fail with astropy 4.2 due to "ModuleNotFoundError: No module named 'astropy._erfa.core'" due to "import astropy._erfa.core" in utils.py.

Whatever solution is found, it should also be backwards compatible with 4.0.1.post1 (what we currently use at NERSC) and whatever version of astropy is in the ICS environment for operations (it's not in the default desiobserver environment and I don't see a eups package to load, so I'm not sure what that config is).

======================================================================
ERROR: desisurvey.test.test_utils (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: desisurvey.test.test_utils
Traceback (most recent call last):
  File "/global/common/software/desi/cori/desiconda/20210326-1.5.0-spec/conda/lib/python3.9/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/global/common/software/desi/cori/desiconda/20210326-1.5.0-spec/conda/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/global/common/software/desi/users/sjbailey/desisurvey/py/desisurvey/test/test_utils.py", line 18, in <module>
    from desisurvey.test.base import Tester
  File "/global/common/software/desi/users/sjbailey/desisurvey/py/desisurvey/test/base.py", line 12, in <module>
    import desisurvey.ephem
  File "/global/common/software/desi/users/sjbailey/desisurvey/py/desisurvey/ephem.py", line 22, in <module>
    import desisurvey.utils
  File "/global/common/software/desi/users/sjbailey/desisurvey/py/desisurvey/utils.py", line 18, in <module>
    import astropy._erfa.core
ModuleNotFoundError: No module named 'astropy._erfa.core'

Mentioning @dkirkby @weaverba137 as original erfa workaround gurus. This is a blocking factor for upgrading astropy in our next desiconda release.

sbailey commented 3 years ago

earlier versions of astropy are unavailable in conda for python 3.9, thus requiring us to stick to python 3.8 until this issue is fixed

sbailey commented 3 years ago

@weaverba137 could you take a look at this astropy erfa issue and suggest a workaround that works with astropy 4.2 and 4.0.x? This is a blocking factor for upgrading to astropy 4.2 for the next software release. Although that isn't strictly required, it would be preferable to catch up with that instead of staying on 4.0.x and we'll need to address this at some point. In the meantime, maintaining compatibility with 4.0.x is important to support the KPNO operations environment which uses the the next tile selector in desisurvey.

weaverba137 commented 3 years ago

I'll take a look, but I'm wiped out for the day. I don't think this is a hard problem. I believe there is now a pyerfa module, no longer part of astropy itself, that has to be imported instead. There shouldn't be any API changes, just an import change.

weaverba137 commented 3 years ago

For reference: https://docs.astropy.org/en/stable/whatsnew/4.2.html#removed-erfa-module

I'll start testing this momentarily.

weaverba137 commented 3 years ago

@dkirkby, what does astropy._erfa.core actually do in desisurvey.utils? As far as I can tell, it is imported but not used! If it is completely unused, then this could be a particularly easy fix.

dkirkby commented 3 years ago

@weaverba137 This is the commit where I added this import: https://github.com/desihub/desisurvey/commit/32fdde1beef3c8d2b8bf52f52154418c3a924797

It was needed for the definition of astropy._erfa.core.ErfaWarning used in freeze_iers but that code now lives in desiutils.iers.

weaverba137 commented 3 years ago

Aha, so this may need to be fixed in desiutil then. Good to know.

weaverba137 commented 3 years ago

In fact, desiutil already handles this correctly: https://github.com/desihub/desiutil/blob/9dd77fe24f620d9282d4211fa881763c307ed890/py/desiutil/iers.py#L126

So I think the solution is just to remove the unused import statement.