pytroll / satpy

Python package for earth-observing satellite data processing
http://satpy.readthedocs.org/en/latest/
GNU General Public License v3.0
1.06k stars 292 forks source link

Drop Python 3.9 support #2741

Open djhoese opened 7 months ago

djhoese commented 7 months ago

Following the guidance of:

https://scientific-python.org/specs/spec-0000/

This PR drops Python 3.9 support and adjusts CI, pre-commit, and readthedocs to use newer versions of Python. This means the currently supported versions of Python will be 3.10, 3.11, and 3.12.

CC @pytroll/satpy-core I've asked for reviews from some of you, but I think as long as there are no complaints this should be a simple agreement and merge.

djhoese commented 7 months ago

The remaining unstable environment breakage has been reported here: https://github.com/Unidata/cftime/issues/318

codecov[bot] commented 7 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (4877082) 95.88% compared to head (4c73bd6) 95.88%. Report is 7 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2741 +/- ## ========================================== - Coverage 95.88% 95.88% -0.01% ========================================== Files 371 371 Lines 52825 52824 -1 ========================================== - Hits 50653 50651 -2 - Misses 2172 2173 +1 ``` | [Flag](https://app.codecov.io/gh/pytroll/satpy/pull/2741/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll) | Coverage Δ | | |---|---|---| | [behaviourtests](https://app.codecov.io/gh/pytroll/satpy/pull/2741/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll) | `4.15% <20.00%> (-0.01%)` | :arrow_down: | | [unittests](https://app.codecov.io/gh/pytroll/satpy/pull/2741/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll) | `95.98% <100.00%> (-0.01%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pytroll#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

coveralls commented 7 months ago

Pull Request Test Coverage Report for Build 7802697177


Totals Coverage Status
Change from base Build 7801990864: 96.0%
Covered Lines: 50523
Relevant Lines: 52648

💛 - Coveralls
ameraner commented 7 months ago

I understand that the fact that EUMETSAT's fcidecomp package (https://anaconda.org/Eumetsat/fcidecomp, and https://gitlab.eumetsat.int/open-source/data-tailor-plugins/fcidecomp/-/tree/2.0.1?ref_type=heads, see also threads on slack) only currently supports only up to py3.9 is going to be a problem after this PR.

Considering that FCI will go pre-operational soon, and satpy will get a lot of attention in the next months because of this, is it an option to wait a bit more for dropping 3.9? EUM "promised" an update in May, but maybe with a little more pressure (e.g. at the OPSWG in March) this could be accelerated.

I fully understand that it's highly annoying to wait for updates for a package that is not maintained properly, but considering this special phase with an update of a major met satellite, maybe it's worth considering.

If this is not possible/too painful, maybe we should consider writing a docs page for possible workarounds (manual installation from Pytroll's fcidecomp fork, or hdf5plugin)?

simonrp84 commented 7 months ago

I agree with @ameraner, if possible it'd be good to delay the dropping of 3.9 until FCI can be easily supported.

djhoese commented 7 months ago

I don't have a problem waiting a little longer, but don't all the readers use hdf5plugin right now? I think @gerritholl had shown that fcidecomp doesn't even work with modern numpy versions.

pnuu commented 7 months ago

I don't have a problem waiting a little longer, but don't all the readers use hdf5plugin right now?

We don't import hdf5plugins in Satpy. It was decided that we shouldn't force the user to one solution, so either import hdf5lugins before importing satpy/xarray/netcdf4/h5netcdf, or installing fcidecomp, by the user is always required.

djhoese commented 7 months ago

Ah ok. So not as strict/severe of a case then. Still, if fcidecomp doesn't work with modern numpy then users won't be able to create a new python environment that works with fcidecomp anyway. How do we anticipate people wanting to process FCI data will create their python environments? If they don't specify a specific Python version then they're going to get Python 3.11 or 3.12 as it is. If they are told to install Python 3.9 (or less) then they'll get an older version of Satpy that is compatible with that. So maybe the question is if the newest FCI changes (composites, L2, etc) are requested...eh OK we'll wait.

gerritholl commented 7 months ago

I think @gerritholl had shown that fcidecomp doesn't even work with modern numpy versions.

That problem was fixed by EUMETSAT (or their contractor). fcidecomp now works with modern numpy.