calliope-project / calliope

A multi-scale energy systems modelling framework
https://www.callio.pe
Apache License 2.0
287 stars 93 forks source link

Bump to py3.12 support #519

Closed brynpickering closed 10 months ago

brynpickering commented 11 months ago

Move from py3.9-3.11 to py3.10-3.12.

This is made possible by the recent release of a new pandas patch (v2.1.2) that should have dealt with the regressions introduced in v2.1.1 (see #498), according to their changelog.

EDIT: I've run some tests (national scale model over one year) and it seems that the pandas patch hasn't solved the issues entirely. The run has gone from approx 3min50 to 6min. This is offset by a ~30% reduction in peak "resident" memory consumption (no change in "heap" consumption between the two).

Reviewer checklist:

brynpickering commented 11 months ago

The primary benefit of killing py3.9 support is that cleaner type hints are possible without needing to retain backwards compatibility (Union[A, B] in <=py3.9 vs A | B in >=py3.10).

The only issue I see is that it keeps us quite constrained (currently) to one version of pandas. Once more v2.1 patches are released, we'll have a bit more breathing room.

codecov[bot] commented 11 months ago

Codecov Report

Merging #519 (dd9a839) into main (3779e0d) will increase coverage by 0.21%. Report is 2 commits behind head on main. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #519      +/-   ##
==========================================
+ Coverage   93.21%   93.42%   +0.21%     
==========================================
  Files          28       28              
  Lines        3817     3804      -13     
  Branches      877      800      -77     
==========================================
- Hits         3558     3554       -4     
+ Misses        168      162       -6     
+ Partials       91       88       -3     
Files Coverage Δ
src/calliope/backend/backend_model.py 97.47% <100.00%> (+0.01%) :arrow_up:
src/calliope/backend/where_parser.py 97.67% <100.00%> (ø)
src/calliope/core/model.py 93.75% <100.00%> (+0.03%) :arrow_up:
src/calliope/preprocess/checks.py 85.16% <100.00%> (ø)
src/calliope/preprocess/model_run.py 91.82% <ø> (ø)
src/calliope/preprocess/nodes.py 93.75% <100.00%> (+3.17%) :arrow_up:

... and 1 file with indirect coverage changes