search

ec-jrc / pyPoseidon

Framework for Hydrodynamic simulations
https://pyposeidon.readthedocs.io/
European Union Public License 1.2
20 stars 8 forks source link

Support pandas 2.0 #134

Closed pmav99 closed 10 months ago

pmav99 commented 1 year ago

As mentioned by @uturuncoglu here 

Traceback (most recent call last):
  File "merge.py", line 12, in <module>
    model.results()
  File "/Users/turuncu/miniconda3/envs/schism/lib/python3.8/site-packages/pyposeidon/schism.py", line 1476, in results
    sdate = pd.Timestamp(
  File "pandas/_libs/tslibs/timestamps.pyx", line 1656, in pandas._libs.tslibs.timestamps.Timestamp.__new__
  File "pandas/_libs/tslibs/timezones.pyx", line 172, in pandas._libs.tslibs.timezones.maybe_get_tz
TypeError: unsupported type for timedelta seconds component: numpy.int64

This is also causing a test to fail when running under pandas 2.0:

  pyposeidon/d3d.py:306: FutureWarning: In a future version of pandas all arguments of StringMethods.split except for the argument 'pat' will be keyword-only.
    d2 = df.loc[tlines, "data"].str.split("=", 2, expand=True)
brey commented 10 months ago

@pmav99 Is this still an issue?

pmav99 commented 10 months ago

We are still pinning pandas < 2.

I released the constrained, updated the packages and run the tests. They are passing but there are a bunch of FutureWarnings:

tests/test_dem_fix.py: 8 warnings
tests/test_mesh.py: 4 warnings
tests/test_mesh_global.py: 2 warnings
  /home/panos/Prog/poseidon/pyPoseidon/pyposeidon/mjigsaw.py:146: FutureWarning: Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
    npoints = int(grid.loc[2].str.split("=")[0][1])

tests/test_dem_fix.py: 8 warnings
tests/test_mesh.py: 4 warnings
tests/test_mesh_global.py: 2 warnings
  /home/panos/Prog/poseidon/pyPoseidon/pyposeidon/mjigsaw.py:154: FutureWarning: Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
    nedges = int(grid.loc[ie].str.split("=")[0][1])

tests/test_dem_fix.py: 8 warnings
tests/test_mesh.py: 4 warnings
tests/test_mesh_global.py: 2 warnings
  /home/panos/Prog/poseidon/pyPoseidon/pyposeidon/mjigsaw.py:161: FutureWarning: Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
    ntria = int(grid.loc[i3].str.split("=")[0][1])

tests/test_dem_io.py: 4 warnings
tests/test_execution_control.py: 2 warnings
tests/test_schism_mesh.py: 2 warnings
tests/test_mesh_global.py: 4 warnings
tests/test_update.py: 4 warnings
tests/test_schism.py: 1 warning
  /home/panos/Prog/poseidon/pyPoseidon/pyposeidon/mesh.py:232: FutureWarning: Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
    ni, nj = df.iloc[0].str.split()[0]

Do you wanna take a look? You need to upgrade pandas to 2.1 and run the tests.

tomsail commented 10 months ago

confirming also for pandas<2, it conflicts when having analysea in the same conda env

tomsail commented 10 months ago

Just did it. All tests run fine. creating PR

tomsail commented 10 months ago

Actually not that straightforward. I am puzzled that I can't reproduced the errors on the CI though.. On my machine everything was running fine. See #170 for reference