Error when running SuPy

[emd01]

Continuously getting a KeyError: 'Level var not found' when running the code:

import supy as sp
import matplotlib.pyplot as plt
import pandas as pd
from pathlib import Path
path_runcontrol = Path('C:/Users/emili/Downloads/SuPy_document') / 'RunControl.nml'
df_state_init = sp.init_supy(path_runcontrol)
grid = df_state_init.index[0]
df_forcing = sp.load_forcing_grid(path_runcontrol, grid)
df_output, df_state_final = sp.run_supy(df_forcing, df_state_init)
list_path_save = supy.save_supy(df_output, df_state_final, freq_s=3600, site='', path_dir_save='C:/Users/emili/Downloads/SuPy_document/output')

see full error in the following doc: SuPy ValueError.docx

Also get an error when running in QGIS, debugging file is too big to attach

[sunt05]

Please provide the following:

• version info: sp.show_version()
• input files that others may reproduce your error.

Thanks.

[sunt05]

Hi @emd01, your last post is unreadable. Could you please provide additional information using the proper GitHub interface?

[emd01]

Input.zip

[sunt05]

RunControl.nml is missing.

[emd01]

Input.zip

[emd01]

Apologies, the runcontrol file should be included in the zipped file above now

[sunt05]

Thanks - I had a quick look at the files upload but didn't spot any apparent issue. However, it is noted a relatively old supy is being used - please do the following:

1. update to a recent dev version: pip install supy==2023.7.3.dev0
2. update your input data frame df_state as your input files are based on a previous version of SUEWS:

   # p_runcontrol is the path to your `RunControl.nml`
   df_state_init = sp.init_supy(p_runcontrol)
   df_state_init[("faimethod", "0")] = 0

3. run supy:

grid = df_state_init.index[0]
df_forcing = sp.load_forcing_grid(path_runcontrol, grid)
df_output, df_state_final = sp.run_supy(df_forcing, df_state_init)

Should further issues arise, please attach the complete error log in a plain text file or simply post it here – no worries about the length. Thank you.

[emd01]

This RunTime error is now coming up, I've tried it in both jupyter notebook and command terminal

2023-12-19 15:11:08,948 - SuPy - INFO - All cache cleared.
2023-12-19 15:11:24,803 - SuPy - INFO - All cache cleared.
2023-12-19 15:11:40,976 - SuPy - INFO - All cache cleared.
2023-12-19 15:11:56,321 - SuPy - INFO - ====================
2023-12-19 15:11:56,322 - SuPy - INFO - Simulation period:
2023-12-19 15:11:56,322 - SuPy - INFO -   Start: 2018-06-09 23:05:00
2023-12-19 15:11:56,322 - SuPy - INFO -   End: 2018-06-15 23:00:00
2023-12-19 15:11:56,323 - SuPy - INFO -
2023-12-19 15:11:56,323 - SuPy - INFO - No. of grids: 11097
2023-12-19 15:11:56,324 - SuPy - INFO - SuPy is running in serial mode
Traceback (most recent call last):
  File "C:\Users\emili\anaconda3\Lib\site-packages\supy\_run.py", line 380, in run_supy_ser
    list_res_grid = [
                    ^
  File "C:\Users\emili\anaconda3\Lib\site-packages\supy\_run.py", line 381, in <listcomp>
    suews_cal_tstep_multi(dict_state_input, df_forcing)
  File "C:\Users\emili\anaconda3\Lib\site-packages\supy\_run.py", line 140, in suews_cal_tstep_multi
    dict_input = {k: dict_input[k] for k in list_var_input_multitsteps}
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\emili\anaconda3\Lib\site-packages\supy\_run.py", line 140, in <dictcomp>
    dict_input = {k: dict_input[k] for k in list_var_input_multitsteps}
                     ~~~~~~~~~~^^^
KeyError: 'faimethod'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\emili\Downloads\SuPy_document\myscript.py", line 13, in <module>
    df_output, df_state_final = sp.run_supy(df_forcing, df_state_init)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\emili\anaconda3\Lib\site-packages\supy\_supy_module.py", line 372, in run_supy
    df_output, df_state_final = run_supy_ser(
                                ^^^^^^^^^^^^^
  File "C:\Users\emili\anaconda3\Lib\site-packages\supy\_run.py", line 388, in run_supy_ser
    raise RuntimeError(
RuntimeError:
====================
SUEWS kernel error!
A zip file for debugging has been saved as `C:/Users/emili/supy_debug-9a6e5d05.zip`:Please report this issue with the above zip file to the developer at https://github.com/UMEP-dev/SuPy/issues/new?assignees=&labels=&template=issue-report.md.
====================

[sunt05]

did you use this piece of code provided above?

# p_runcontrol is the path to your `RunControl.nml`
df_state_init = sp.init_supy(p_runcontrol)
df_state_init[("faimethod", "0")] = 0

[emd01]

Yes I did

[sunt05]

please upload the zip file as indicated in your error message above for further debugging.

====================
SUEWS kernel error!
A zip file for debugging has been saved as `C:/Users/emili/supy_debug-9a6e5d05.zip`:Please report this issue with the above zip file to the developer at https://github.com/UMEP-dev/SuPy/issues/new?assignees=&labels=&template=issue-report.md.
====================

[emd01]

The zipped file seems to be too large to upload as its 108MB, is there any other way to get the folder to you?

[sunt05]

The zipped file seems to be too large to upload as its 108MB, is there any other way to get the folder to you?

Please upload it here: https://www.dropbox.com/request/X9BJ5mYHjkFmYc81Enz9

[emd01]

Thank you, just uploaded it there!

[sunt05]

Just checked the zip file uploaded - apparently the modification was not done properly.

After the modification, there should be a new column faimethod in the modified df_state - use this code to check it:

df_state_init.filter(like='faimethod')

If you're unable to see it, then this line of code likely wasn't run correctly.

BTW, here I upload my code (in .ipynb file) and your input files in a zip, which should be working - please give it a go:

GH241.zip

[emd01]

Thank you, strange when I use that code to check it there is a faimethod column, it's full of zeros but I think it's supposed to be?

[sunt05]

Thank you, strange when I use that code to check it there is a faimethod column, it's full of zeros but I think it's supposed to be?

That's right.

Anyway, please make sure you are using the the modified df_state in the simulation, which needs to include a column faimethod as you noticed above.

[emd01]

I've ran your code and it runs with 10 grids, however when I increase it to 100 grids the kernel dies, the data I am attempting to test has 11,097 grids. Could it be just an issue with the amount of grids being used?

[sunt05]

That implies some of your grids contain invalid values that may trigger runtime errors. You will need to carefully determine which grids are causing the issue and find out the invalid values.

[sunt05]

Hopefully, my answer above addresses this issue, so I'll close it for now. Otherwise, feel free to re-open and provide more details to help resolve it.