IAMconsortium / pyam

Analysis & visualization of energy & climate scenarios
https://pyam-iamc.readthedocs.io/
Apache License 2.0
221 stars 115 forks source link

Refactor initialization for simpler maintenance #730

Closed coroa closed 1 year ago

coroa commented 1 year ago

Please confirm that this PR has done the following:

Description of PR

~On-top of #729.~ Rebased to main.

Splits utils.format_data into 6 different functions:

  1. _convert_r_columns(df) - Check and convert R-style year columns
  2. _knead_data(df, **kwargs) - Replace, rename and concat according to user arguments
  3. _format_from_database(df) - Post-process database results
  4. _intuit_column_groups(df, index) - Check and categorise columns in dataframe
  5. _format_data_to_series(df, index) - Convert a long or wide pandas dataframe to a series with the required columns
  6. _validate_complete_index(df)

Functionally it should be neutral. Make sure to look at individual commits to follow the refactoring trail.

codecov[bot] commented 1 year ago

Codecov Report

Merging #730 (58eb241) into main (7a97516) will increase coverage by 0.0%. The diff coverage is 100.0%.

@@          Coverage Diff          @@
##            main    #730   +/-   ##
=====================================
  Coverage   95.0%   95.0%           
=====================================
  Files         59      59           
  Lines       6014    6020    +6     
=====================================
+ Hits        5717    5725    +8     
+ Misses       297     295    -2     
Impacted Files Coverage Δ
pyam/core.py 95.4% <100.0%> (ø)
pyam/utils.py 92.6% <100.0%> (+0.7%) :arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

gidden commented 1 year ago

This LGTM but best if @danielhuppmann approves!

gidden commented 1 year ago

LGTM - thanks @coroa ! Will merge after tests pass.