Update calibration coefficients to version 2021

[sfinkens]

Proposal by @kgkarl: Update calibration coefficients to PATMOS-X version 2021. Note that the coefficients in this PR deviate from the original NOAA coefficients. I think this should be discussed in the community. For comparison I uploaded the original v2021 coefficients here.

Explanation by @kgkarl:

2. Observe that I did not follow the instrument file information strictly (!). The reason is that calibration coefficients for some satellites are simply unrealistic, which apparently is due to an insufficient amount of samples/matchups from reference data (MODIS). The lack of matchups normally leads to quite unrealistic coefficients for the time-dependent corrections. This concerns in particular Metop-C (with observations starting as late as in 2018). (In fact, the 2021 update for Metop-C was actually delivered to us and implemented but, as you know it lead to big problems). Thus, for Metop-C, I have now only kept the constant term S0 and set the other two (S1 and S2) to zero for the three visible channels. This corresponds to using exclusively the pre-launch calibration without time corrections. In my opinion, it must be better to exclusively use pre-launch calibration in this case compared to using completely unrealistic coefficients for the time corrections. It will at least allow ICDR production for a couple of years without too large deviations. But on a longer term (within one year or two from now) new time-dependent corrections must be implemented also for Metop-C. Notice, however, that the updated calibration most likely will solve our previous problems with Metop-B. Here we got new coefficients which appear to be much more reasonable in comparison to the 2017 calibration.

3. Calibration information for AVHRR channel 3a at 1.6 micron is problematic, and not only for Metop-C. Also for satellites NOAA-15, NOAA-18 and NOAA-19 the information in the new instrument files appear quite unrealistic. Notice that instrument files set all three coefficients (S0, S1 and S2) to zero for satellites NOAA-15 and NOAA-19 while coefficients for NOAA-18 are non-zero but completely unrealistic when compared to corresponding values for other AVHRRs with active switching (e.g. NOAA-16 and NOAA-17). If we follow this we will run into problems. Thus, I have here just given the S0 coefficient the value 0.1 to at least be able to produce something useful in this channel. You could ask why this is needed since these satellites are not normally switching 3a and 3b channels. It is true but it has happened! So, to avoid missing data completely for these few switching periods it is better to use this default value. Remember that we had a two-month period in March to May 2009 for NOAA-19 when switching was active. Furthermore, NOAA has activated switching for NOAA- 18 and NOAA-19 over the Alaska region for some months from 2016-2019 (maybe even additional years thereafter). However, I noticed that in the latest Pygac version the default value of 0.1 for the S0 coefficient is already used in this channel for the mentioned satellites. This explains why we did not see this problem of missing data in CLARA-A3 and in ICDR products. So, I was obviously not the first one to observe these weaknesses of the instrument files and taking action. The weaknesses themselves are quite understandable: NOAA could not find enough with useful matchups for the channel 3a calibration process here (no homogenous targets over Alaska and no matchup data at high latitude ocean areas which is normally used).

TODO:

• [ ] Add new entry to coefficient history

[sfinkens]

cc @oliversus

[kgkarl]

As I wrote in point 3 above, the calibration coefficients for channel 3a for satellites NOAA-18 and NOAA-19 are still not properly defined in the latest update of the calibration files from NOAA (avhrr_v2021r1.tar). However, Mike Foster sent an update on 5th of January 2023 which provides the following default (prelaunch) values for the S0 coefficient for channel 3a for these two satellites: NOAA-18: S0 = 0.10566857 NOAA-19: S0 = 0.10742857

I have prepared a new version of the calibration.json file where these values are updated. I hope this is the final version. Next update will give us a completely new version based on analysis of additional data for 2021-2022 period. This new version will be delivered sometime in 2023. Hopefully, this new version will give us useful time-dependent corrections for Metop-C. Until then we have to use prelaunch coefficients for all visible channels (only S0 being non-zero) for Metop-C.

[sfinkens]

@abhaydd @kgkarl Did you receive an update from Mike in the meantime? Or how should we proceed on this? I think it would be important to properly test the coefficients that deviate from the NOAA version.

[kgkarl]

We have not yet received the new 2023 update but it should be available very soon. There is at least a new place holder for it at https://cimss.ssec.wisc.edu/patmosx/calibration/ (“2023 instrument files”) but still without content. Let’s wait another week or two before making our next action towards Mike.

/Karl-Göran

Från: Stephan Finkensieper @.> Skickat: den 26 juni 2023 13:32 Till: pytroll/pygac @.> Kopia: Karlsson Karl-Göran @.>; Mention @.> Ämne: Re: [pytroll/pygac] Update calibration coefficients to version 2021 (PR #121)

@abhayddhttps://github.com/abhaydd @kgkarlhttps://github.com/kgkarl Did you receive an update from Mike in the meantime? Or how should we proceed on this? I think it would be important to properly test the coefficients that deviate from the NOAA version.

— Reply to this email directly, view it on GitHubhttps://github.com/pytroll/pygac/pull/121#issuecomment-1607277277, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AD736B2QIVND7UQRZ37VDSDXNFXKPANCNFSM6AAAAAATQSNQ5E. You are receiving this because you were mentioned.Message ID: @.**@.>>

[kgkarl]

I noticed that the new calibration version from 2023 is now posted at the PATMOS-x calibration site (“AVHRR instrument files with latest calibration coefficients [2023]”). Please try to update the calibration in Pygac as soon as possible.

/Karl-Göran

Från: Karlsson Karl-Göran Skickat: den 11 juli 2023 15:35 Till: 'pytroll/pygac' @.***> Ämne: SV: [pytroll/pygac] Update calibration coefficients to version 2021 (PR #121)

We have not yet received the new 2023 update but it should be available very soon. There is at least a new place holder for it at https://cimss.ssec.wisc.edu/patmosx/calibration/ (“2023 instrument files”) but still without content. Let’s wait another week or two before making our next action towards Mike.

/Karl-Göran

Från: Stephan Finkensieper @.**@.>> Skickat: den 26 juni 2023 13:32 Till: pytroll/pygac @.**@.>> Kopia: Karlsson Karl-Göran @.**@.>>; Mention @.**@.>> Ämne: Re: [pytroll/pygac] Update calibration coefficients to version 2021 (PR #121)

@abhayddhttps://github.com/abhaydd @kgkarlhttps://github.com/kgkarl Did you receive an update from Mike in the meantime? Or how should we proceed on this? I think it would be important to properly test the coefficients that deviate from the NOAA version.

— Reply to this email directly, view it on GitHubhttps://github.com/pytroll/pygac/pull/121#issuecomment-1607277277, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AD736B2QIVND7UQRZ37VDSDXNFXKPANCNFSM6AAAAAATQSNQ5E. You are receiving this because you were mentioned.Message ID: @.**@.>>

[sfinkens]

👍 Thanks for checking @kgkarl! I'll put it on my list, but it might take a while 🙈

[sfinkens]

Closing in favor of https://github.com/pytroll/pygac/pull/124