ENH: large refactoring - Githubissues

bzah commented 7 months ago

Pull Request to resolve #184

[ ] Unit tests cover the changes.
[ ] These changes were tested on real data.
[x] The relevant documentation has been added or updated.
[x] A short description of the changes has been added to doc/source/references/release_notes.rst.

Describe the changes you made

Goal 1: Add DCSC indices

This pull request add the indices required for DRIAS in a new StandardIndexRegistry DcscIndexRegistry. Additionally, a new parameter offset has been added to Threshold class and build_threshold function. This allows to easily compute indices such as count days where tasmax is 5 deg_C above normal :

thresh = build_threshold(operator=">", value="normal_tasmax.nc", offset="5 delta_deg_C")
result = icclim.count_occurrences("tasmax.nc", threshold=thresh)

Todo:

[x] Finish architecture refactoring
[ ] Add automated tests
[x] Update CI to generate the new indices.
[x] Add documentation
[ ] Add example notebook
[x] Remove icclim.dcsc.dcsc_indices_functions.py and autogenerate functions from the indices definition (similarly to ecad indices)
[ ] Some indices details need to be checked with MF to ensure their correctness.
[x] cleanup code (todos, commented code)

[ ] Add missing indices:

vents:
    FF3   Écart du cumul du nombre de jours sans vent
          (par rapport à la référence 1976-2005)1
humidité
    HUSAV   Humidité spécifique moyenne journalière

icclim will perform a unit conversion if needed.

The following indices were added:

TAV
TXAV
TRAV
TX10
TX90
TN10
TN90
TNFD
TXFD
SD
TX35
TR
TXND
TNHT
TNND
TNCWD
TXHWD
HDD
CDD
PAV
PINT
RR
RR1MM
PN20MM
PXCDD
PXCWD
R99
PFL90
FFAV
FF98
PQ90
PQ99
FFQ98

Goal 2: Refactor to improve maintainability

context: by adding the DCSC indices many issues within the code base were exposed thus this PR includes to improve this situation.

Issues

Lack of docstrings for the public methods and for modules --> Use of github copilot to help generating those.
On our doc website, no access to the existing docstring --> Use autoapi to generate it once
Lack of type hinting in some functions and classes --> Enable additional ruff rules and add type hints
Mutable state of the Indicators when computing the metadata, corrupting the subsequent calls --> Add a clone method to the relevant classes
extract_icclim_funs script is far from easy to augment when creating new StandardIndex registries (like the DCSC one) --> refactor the script to ease maintenance of it.
The architecture does not separate a public API, which should not be updated to often to avoid breaking changes and a private API --> Inspired by numpy 2.0 _core module, segregate the private API in icclim._core and expose the public API in icclim module

review-notebook-app[bot] commented 7 months ago

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

github-actions[bot] commented 6 months ago

Report

File	Stmts	Miss	Cover	Missing
/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/icclim
logger.py	86	29	66%	24, 44, 66, 71–96, 106, 112–143, 151–156
main.py	270	32	88%	60–70, 131, 140, 491–492, 658–659, 727, 753–754, 756–757, 759, 761–762, 764–765, 789–796, 866–871
rechunk.py	90	7	92%	29–32, 126, 228–229, 233
/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/icclim/_core
climate_variable.py	99	20	80%	31–42, 156–162, 264–265, 367–368, 377, 411, 421–422
frequency.py	180	13	93%	39–41, 258–261, 449–453, 470, 553–557, 579, 584
input_parsing.py	263	63	76%	28–37, 92–93, 97–98, 103–108, 140–141, 269, 274–278, 281, 301–305, 307–308, 349–350, 402–406, 409, 463, 534–539, 543, 545, 635, 686, 692–700, 731, 735–739, 754–758, 785, 799, 804–809, 816–821, 832–833, 837–844, 860
utils.py	14	2	86%	30–35
/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/icclim/_core/generic
functions.py	319	83	74%	54–63, 154, 238–239, 980–981, 987, 1021, 1027, 1081–1085, 1087–1088, 1106–1112, 1116, 1169, 1171, 1234–1235, 1244–1270, 1298–1318, 1334, 1342–1374, 1380–1383
indicator.py	150	17	89%	36–41, 158–161, 165, 408, 416–420, 422–426, 461, 507–508, 530
/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/icclim/_core/generic/threshold
basic.py	104	8	92%	31–34, 72, 172–173, 198, 324
bounded.py	55	13	76%	23–29, 54–56, 84–88, 92–96, 227
percentile.py	134	13	90%	40–47, 215–216, 339–344, 458–460
/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/icclim/_core/legacy/user_index
model.py	20	20	0%	3–43
parse.py	85	15	82%	21–26, 60–61, 79, 86, 90–94, 204, 229, 235–236
/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/icclim/_core/model
cf_calendar.py	35	1	97%	63
global_metadata.py	6	6	0%	3–23
icclim_types.py	14	14	0%	3–25
in_file_dictionary.py	9	9	0%	3–46
index_config.py	31	6	81%	13–18
indicator.py	36	8	78%	10–11, 23, 28, 67, 72, 77, 82
logical_link.py	21	1	95%	14
operator.py	32	3	91%	13, 18–22
standard_index.py	36	9	75%	9–16, 68, 72, 77
standard_variable.py	48	1	98%	48
threshold.py	54	10	81%	15–25, 95, 100, 149
/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/icclim/_generated
_dcsc.py	101	55	46%	20–32, 136, 217, 298, 407, 527, 647, 767, 859, 943, 1027, 1111, 1195, 1413–1419, 1524–1530, 1635–1641, 1746–1752, 1834, 1918, 2002, 2083, 2167, 2248, 2332, 2416, 2500, 2612, 2733, 2854, 2975, 3083, 3193
_ecad.py	144	8	94%	19–31
_generic.py	59	10	83%	18–33
/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/icclim/ecad
binding.py	58	8	86%	15–17, 45, 49, 86, 90, 127, 131
/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/icclim/generic
registry.py	41	4	90%	39, 46–47, 69
/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/icclim/threshold
factory.py	169	12	93%	38, 177–178, 255, 279–280, 285, 355–356, 359–363, 403
TOTAL	3101	500	84%

Test results

Tests	Skipped	Failures	Errors	Time
196	0 :zzz:	0 :x:	0 :fire:	2m 3s :stopwatch:

bzah commented 6 months ago

pre-commit.ci autofix

cerfacs-globc / icclim

ENH: large refactoring #303