pytroll / satpy

Python package for earth-observing satellite data processing
http://satpy.readthedocs.org/en/latest/
GNU General Public License v3.0
1.04k stars 287 forks source link

Adapt reader mviri_l1b_fiduceo_nc #2802

Open bkremmli opened 1 month ago

bkremmli commented 1 month ago

[INFO] on hold until a new xarray release with this contribution is out which fixes the duplicate dimension problem: https://github.com/pydata/xarray/pull/9099


This PR fixes the mviri_l1b_fiduceo_nc reader when being used with a new xarray version (2024.3.0). When using the original reader, a ValueError about not being able to decode the times is thrown. The file is now opened without decoding. The decoding is now done in DatasetWrapper()._decode_cf(). The time is decoded separatly from the other data values. FillValues for the time are recognized to replace time values with NaT and time is decoded using the offset values included within the attributes.

Also, opening the dataset using chunks is deactivated because the input files contains dimensions of the same name which cannot be processed by xarray at the moment. The chunking as well as renaming the dimensions is also performed in DatsetWrapper().

sfinkens commented 1 month ago

Also I noticed that space pixels now have some finite values (instead of NaN), because decode_cf=False. You can use decode_cf=True together with time.encoding["add_offset/_FillValue"], see https://github.com/sfinkens/satpy/commit/7045a87cc5e8066712df153ba4e9998c06df6265

codecov[bot] commented 1 month ago

Codecov Report

Attention: Patch coverage is 94.91525% with 3 lines in your changes missing coverage. Please review.

Project coverage is 95.90%. Comparing base (f33c3e4) to head (73acfb7). Report is 202 commits behind head on main.

Files Patch % Lines
...py/tests/reader_tests/test_mviri_l1b_fiduceo_nc.py 94.59% 2 Missing :warning:
satpy/readers/mviri_l1b_fiduceo_nc.py 95.45% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2802 +/- ## ========================================== - Coverage 95.95% 95.90% -0.06% ========================================== Files 379 366 -13 Lines 53888 53524 -364 ========================================== - Hits 51708 51330 -378 - Misses 2180 2194 +14 ``` | [Flag](https://app.codecov.io/gh/pytroll/satpy/pull/2802/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll) | Coverage Δ | | |---|---|---| | [behaviourtests](https://app.codecov.io/gh/pytroll/satpy/pull/2802/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll) | `4.04% <0.00%> (-0.05%)` | :arrow_down: | | [unittests](https://app.codecov.io/gh/pytroll/satpy/pull/2802/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll) | `95.99% <94.91%> (-0.06%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

bkremmli commented 3 weeks ago

The test for the duplicate dimensions still needs to be added.