IAMconsortium / pyam

Analysis & visualization of energy & climate scenarios
https://pyam-iamc.readthedocs.io/
Apache License 2.0
221 stars 115 forks source link

Pandas 2.0 breaks some tests #738

Closed phackstock closed 1 year ago

phackstock commented 1 year ago

As I was updating https://github.com/IAMconsortium/nomenclature to be pandas 2.0 compatible I checked if pandas 2.0 would break anything in pyam. Turns out it does:

================================================================ short test summary info =================================================================
FAILED tests/test_core.py::test_init_df_with_duplicates_raises[test_df0] - AttributeError: 'DataFrame' object has no attribute 'append'
FAILED tests/test_core.py::test_init_df_with_duplicates_raises[test_df1] - AttributeError: 'DataFrame' object has no attribute 'append'
FAILED tests/test_core.py::test_init_df_with_duplicates_raises[test_df2] - AttributeError: 'DataFrame' object has no attribute 'append'
FAILED tests/test_core.py::test_init_df_with_duplicates_raises[test_df3] - AttributeError: 'DataFrame' object has no attribute 'append'
FAILED tests/test_core.py::test_interpolate_datetimes[test_df1] - TypeError: assert_series_equal() got an unexpected keyword argument 'check_less_precise'
FAILED tests/test_core.py::test_interpolate_datetimes[test_df2] - TypeError: assert_series_equal() got an unexpected keyword argument 'check_less_precise'
FAILED tests/test_core.py::test_interpolate_datetimes[test_df3] - TypeError: assert_series_equal() got an unexpected keyword argument 'check_less_precise'
FAILED tests/test_filter.py::test_filter_mixed_time_domain[arg_year0-arg_time0] - AttributeError: module 'pandas' has no attribute 'Int64Index'
FAILED tests/test_filter.py::test_filter_mixed_time_domain[arg_year1-arg_time1] - AttributeError: module 'pandas' has no attribute 'Int64Index'
FAILED tests/test_filter.py::test_filter_with_numpy_64_date_vals[test_df1] - AssertionError: assert False
FAILED tests/test_filter.py::test_filter_with_numpy_64_date_vals[test_df2] - AssertionError: assert False
FAILED tests/test_filter.py::test_filter_with_numpy_64_date_vals[test_df3] - AssertionError: assert False
FAILED tests/test_iiasa.py::test_query_with_meta_arg[kwargs0-meta0] - TypeError: Passing a set as an indexer is not supported. Use a list instead.
FAILED tests/test_iiasa.py::test_query_with_meta_arg[kwargs0-meta1] - TypeError: Passing a set as an indexer is not supported. Use a list instead.
FAILED tests/test_iiasa.py::test_query_with_meta_arg[kwargs1-meta0] - TypeError: Passing a set as an indexer is not supported. Use a list instead.
FAILED tests/test_iiasa.py::test_query_with_meta_arg[kwargs1-meta1] - TypeError: Passing a set as an indexer is not supported. Use a list instead.
FAILED tests/test_iiasa.py::test_query_with_meta_arg[kwargs2-meta0] - TypeError: Passing a set as an indexer is not supported. Use a list instead.
FAILED tests/test_iiasa.py::test_query_with_meta_arg[kwargs2-meta1] - TypeError: Passing a set as an indexer is not supported. Use a list instead.
FAILED tests/test_statistics.py::test_statistics - AttributeError: 'DataFrame' object has no attribute 'append'
FAILED tests/test_statistics.py::test_statistics_by_filter - AssertionError: MultiIndex level [1] are different
FAILED tests/test_statistics.py::test_statistics_with_rows - AssertionError: MultiIndex level [1] are different
FAILED tests/test_tutorials.py::test_iiasa_dbs - subprocess.CalledProcessError: Command '['jupyter', 'nbconvert', '--to', 'notebook', '--execute', '--E...

@danielhuppmann as this does not look like a 5 minute fix I would propose a version pin to pandas<2 until it's resolved.

danielhuppmann commented 1 year ago

Working on it... I'll add the pin if I don't manage to fix it before the Easter weekend.

phackstock commented 1 year ago

Perfect. I think as opposed to nomenclature which works with both pandas 1.5.2 and pandas 2.0, pyam will need to introduce breaking changes.