aiapy.calibrate.degradation() runs with warnings and output is now different from aia_bp_get_corrections

[nabobalis]

In GitLab by @ebuchlin on May 9, 2021, 06:41

Hello,

Given the following code:

import astropy.units as u
from astropy.time import Time
from aiapy.calibrate import degradation

t = Time(58817.5, format="mjd", scale="utc")   # 2019-11-30T12:00:00
d = degradation(94 * u.AA, t)
print(d)

before I did an upgrade of aiapy, astropy, pyerfa, and related packages, the output was 0.983675, a bit different than that I got (0.9880996) from aia_bp_get_corrections.pro (which is normally using aia_V9_20200706_215452_response_table.txt).

Now, after the upgrade (still with python 3.8.6; system: Ubuntu 20.10), I have:

• aiapy==0.5.0
• astropy==4.2.1
• sunpy==2.1.4
• pyerfa==1.7.3

and the output value is 0.90317732, which is really far off from the other values. The output including warnings is:

/usr/local/lib/python3.8/dist-packages/erfa/core.py:154: ErfaWarning: ERFA function "taiutc" yielded 1 of "dubious year (Note 4)"
  warnings.warn('ERFA function "{}" yielded {}'.format(func_name, wmsg),
WARNING: SunpyDeprecationWarning: specify desired keywords as arguments to JSOCResponse.show() [aiapy.calibrate.util]
WARNING: SunpyDeprecationWarning: use JSOCClient.search() instead [aiapy.calibrate.util]
/usr/local/lib/python3.8/dist-packages/erfa/core.py:154: ErfaWarning: ERFA function "utctai" yielded 1 of "dubious year (Note 3)"
  warnings.warn('ERFA function "{}" yielded {}'.format(func_name, wmsg),
/usr/local/lib/python3.8/dist-packages/erfa/core.py:154: ErfaWarning: ERFA function "dtf2d" yielded 100 of "dubious year (Note 6)"
  warnings.warn('ERFA function "{}" yielded {}'.format(func_name, wmsg),
WARNING: AiapyUserWarning: Multiple valid epochs for 58817.5. Using the most recent one [aiapy.calibrate.util]
[0.90317732]

I think that the ErfaWarnings were already there before; they could be related to astropy issue 9603, although I don't understand what I'm doing wrong with this date (UTC is well defined in 2019), but I'm not very familiar with astropy.time. I don't understand the SunpyDeprecationWarning and AiapyUserWarning.

I have some plots of all degradation factors before and after (plain lines) the upgrade, compared to aia_bp_get_corrections.pro (dots). The date is number of days since 2010-05-01. But please disregard the last month of the dotted lines because our SSW is currently broken (that's one more reason to migrate from AIA SSW to aiapy).

Before:

After:

[nabobalis]

In GitLab by @wtbarnes on May 9, 2021, 09:41

Thank you for the detailed issue @ebuchlin. I believe the differences you’re showing here are related to the default calibration version. In v0.4, the default version was switched to V10. From your comment, it looks like the SSW version is using the v9 table. Note that you can pass in a specific version and even a specific calibration table into the degradation function: https://aiapy.readthedocs.io/en/stable/api/aiapy.calibrate.degradation.html#aiapy.calibrate.degradation. What does your comparison look like if you explicitly pass in version 9 or your specific calibration table?

@pboerner and @markcheung may be able to comment more on the reasons for the differences between the two calibration versions.

[nabobalis]

In GitLab by @ebuchlin on May 9, 2021, 11:13

Thanks, it was indeed a matter of versions. Our aia_bp_get_corrections.pro was outdated, meaning that max(allowedversions) was 9. By passing calibration_version=9 to degradation(), I recover the old value. The comparison between V9 and V10 looks very similar to the above comparison between before and after the upgrade (dotted is V9, plain is V10):

I am still a little bit worried by the warnings.

[nabobalis]

In GitLab by @wtbarnes on May 9, 2021, 13:53

Glad that helped clarify things!

I agree the ERFA warnings are somewhat worrisome. Thanks for the link to that astropy issue. We should really track down where exactly these warnings are being emitted. Would you mind creating a separate issue for this or editing this existing issue to be more explicitly focused on those warnings?

[nabobalis]

In GitLab by @ebuchlin on May 10, 2021, 02:32

Sure, I agree that warnings should be different issues. This is now issue #102 (ErfaWarning and AiapyUserWarning) and issue #103 (SunpyDeprecationWarning).