search

UMEP-dev / SuPy

SUEWS that speaks Python
https://supy.readthedocs.io/
GNU General Public License v3.0
13 stars 7 forks source link

ValueError: Columns must be same length as key #31

Closed feigenwinter closed 3 years ago

feigenwinter commented 3 years ago

Describe the Issue Hi Ting, The above error occurs when running supy in a Linux environment. The example is for loading SampleData, but it also occurs with my own data, e.g. with df_state_init = sp.init_supy(rctrl). supy runs fine with the same input data in a Windows environment. Location

Screenshots feige@met-ws1:~/py$ python3 Python 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information.

import supy as sp
df_state_init, df_forcing = sp.load_SampleData()

error message:

/users/staff/geogra/feige/.local/lib/python3.8/site-packages/supy/_load.py:1361: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider using pd.concat instead.  To get a de-fragmented frame, use `newframe = frame.copy()`
  df_init[(k, "0")] = dict_InitCond_default[k]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/users/staff/geogra/feige/.local/lib/python3.8/site-packages/supy/_supy_module.py", line 261, in load_SampleData
    df_state_init = init_supy(path_runcontrol, force_reload=False)
  File "/users/staff/geogra/feige/.local/lib/python3.8/site-packages/supy/_supy_module.py", line 103, in init_supy
    df_state_init = load_InitialCond_grid_df(
  File "/users/staff/geogra/feige/.local/lib/python3.8/site-packages/supy/_load.py", line 1591, in load_InitialCond_grid_df
    df_init = load_SUEWS_InitialCond_df(path_runcontrol)
  File "/users/staff/geogra/feige/.local/lib/python3.8/site-packages/supy/_load.py", line 1369, in load_SUEWS_InitialCond_df
    df_init[("waterstate", "0")] = df_init["waterdepth"]
  File "/users/staff/geogra/feige/.local/lib/python3.8/site-packages/pandas/core/frame.py", line 3599, in __setitem__
    self._set_item_frame_value(key, value)
  File "/users/staff/geogra/feige/.local/lib/python3.8/site-packages/pandas/core/frame.py", line 3724, in _set_item_frame_value
    raise ValueError("Columns must be same length as key")
ValueError: Columns must be same length as key

Additional context Thanks for checking Christian

sunt05 commented 3 years ago

Hi Christian, thanks for reporting the issue.

Could you please provide more diagnostic info by running the following command:

sp.show_version()
feigenwinter commented 3 years ago

Ubuntu 20.04.2 LTS (GNU/Linux 5.4.0-74-generic x86_64)

import supy supy.version '2021.7.1 ' import pandas pandas.version '1.3.0' import numpy numpy.version '1.21.0'

feigenwinter commented 3 years ago

Here is the full output...

>>> sp.show_version()
SuPy versions
-------------
supy: 2021.7.1
supy_driver: 2021a2

=================
SYSTEM DEPENDENCY

INSTALLED VERSIONS
------------------
commit           : f00ed8f47020034e752baf0250483053340971b0
python           : 3.8.10.final.0
python-bits      : 64
OS               : Linux
OS-release       : 5.4.0-74-generic
Version          : #83-Ubuntu SMP Sat May 8 02:35:39 UTC 2021
machine          : x86_64
processor        : x86_64
byteorder        : little
LC_ALL           : None
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 1.3.0
numpy            : 1.21.0
pytz             : 2021.1
dateutil         : 2.8.1
pip              : 21.1.3
setuptools       : 57.1.0
Cython           : None
pytest           : 6.2.4
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : 4.6.3
html5lib         : 1.1
pymysql          : None
psycopg2         : None
jinja2           : None
IPython          : None
pandas_datareader: None
bs4              : 4.9.3
bottleneck       : None
fsspec           : 2021.06.1
fastparquet      : None
gcsfs            : None
matplotlib       : 3.4.2
numexpr          : 2.7.3
odfpy            : None
openpyxl         : 3.0.7
pandas_gbq       : None
pyarrow          : None
pyxlsb           : None
s3fs             : None
scipy            : 1.7.0
sqlalchemy       : None
tables           : 3.6.1
tabulate         : None
xarray           : 0.18.2
xlrd             : 2.0.1
xlwt             : 1.3.0
numba            : None
sunt05 commented 3 years ago

thanks for the info. Could you please double-check on your windows machine if such info is similar? in particular, is your pandas on windows also 1.3.0? or a 1.2.x version? I noticed you have pandas 1.3.0 on your linux machine.

sunt05 commented 3 years ago

just confirmed on macOS pandas 1.3.0 would cause the same issue.

feigenwinter commented 3 years ago

on my windows PCs I have pandas 1.1.5 and 1.2.5, both work fine with supy. So it seems that I have to downgrade pandas on the Linux machine?

sunt05 commented 3 years ago

for the time being, please do so: downgrade pandas to get things working. however, I'll fix this soon: ~this weekend. sorry about the inconvenience.

feigenwinter commented 3 years ago

works now with pandas 1.2.5 thanks a lot! Best regards from rainy Switzerland Christian

sunt05 commented 3 years ago

this should be fixed now: a minor issue in assigning values. please wait for an hour or so before you can update supy to the latest version 2021.7.15: a fixed version was just submitted.

sunt05 commented 3 years ago

fixed in https://github.com/UMEP-dev/SuPy/commit/92de2eac813b83f32c316543b22a34b965a0613d