RSL issue

[sunt05]

I did the first and i have the problem with temperature again this is the calculated from suews values

the map to see in some areas the problem my data SUEWS_Soil.zip

Originally posted by @Empanag in https://github.com/UMEP-dev/UMEP/issues/415#issuecomment-1126806464

[Empanag]

any news about the issue?

[sunt05]

Hi @Empanag, sorry about the delay - recently my development focus is on the SPARTACUS related issues. I'll come back to this later next week.

[sunt05]

any news about the issue?

Hi @Empanag,

Please update supy: pip install supy==2022.7.1.dev0, these issues should be resolved in this version; otherwise, please post here with your testing files. Also, you can just include those grids with issues; other grids are not needed - they actually make our debugging much harder - thanks!

[Empanag]

any news about the issue?

Hi @Empanag,

Please update supy: pip install supy==2022.7.1.dev0, these issues should be resolved in this version; otherwise, please post here with your testing files. Also, you can just include those grids with issues; other grids are not needed - they actually make our debugging much harder - thanks!

Noted. i need SPARTACUS.nml. What is this?

[sunt05]

Sorry to see this - the dev version of supy involves some changes in the input files - hence the SPARTACUS.nml file you noted. What is the setting for your netradiationmethod? if larger than 1000, please deduct it by 1000 (e.g. 1003 -> 3) and see if it works. otherwise, please see here for example input files.

[Empanag]

Sorry to see this - the dev version of supy involves some changes in the input files - hence the SPARTACUS.nml file you noted. What is the setting for your netradiationmethod? if larger than 1000, please deduct it by 1000 (e.g. 1003 -> 3) and see if it works. otherwise, please see here for example input files.

okey i will try to reproduce the input files from UMEP and contact you again if i have a problem. The netradiationmethod is 1

[suegrimmond]

you shouldn't be using Spartacus at this time - it is not ready to be used yet

[Empanag]

I understand but the new version produce this error. 2020-08-01 23:00:00 2022-07-07 17:24:27,334 - SuPy - INFO - All cache cleared. 2022-07-07 17:24:27,338 - SuPy - ERROR - E:\SUEWSapp\testirrig\7-ir-25sm10-11-12-13\SUEWS_SPARTACUS.nml does not exists! Traceback (most recent call last): File "c:\users\rslab20emm\appdata\local\programs\python\python39\lib\site-packages\supy_load.py", line 245, in load_SUEWS_nml df_nml = pd.DataFrame(parser.read(p_nml)) File "C:\Users\RSLab20Emm\AppData\Roaming\Python\Python39\site-packages\f90nml\parser.py", line 279, in read nml_file = open(nml_fname, 'r') if nml_is_path else nml_fname FileNotFoundError: [Errno 2] No such file or directory: 'E:\SUEWSapp\testirrig\7-ir-25sm10-11-12-13\SUEWS_SPARTACUS.nml'

[sunt05]

based on the error above, you just need to download the SUEWS_SPARTACUS.nml from this place and place it in your input location (E:\SUEWSapp\testirrig\7-ir-25sm10-11-12-13). hope it will be resolved.

[Empanag]

Okey i downloaded the 'SUEWS_SPARTACUS.nml

based on the error above, you just need to download the SUEWS_SPARTACUS.nml from this place and place it in your input location (E:\SUEWSapp\testirrig\7-ir-25sm10-11-12-13). hope it will be resolved.

Okey i downloaded SUEWS_SPARTACUS.nml and also GridLayout.nml needed so i downloaded.

i have the following message but seems to run

2020-08-01 23:00:00
2022-07-07 17:37:51,589 - SuPy - INFO - All cache cleared.
n_vegetation_region_urban 1 <class 'numpy.int32'> is not included: this should not happen! please report to the developer: https://github.com/UMEP-dev/SuPy/issues/new
n_stream_sw_urban 4 <class 'numpy.int32'> is not included: this should not happen! please report to the developer: https://github.com/UMEP-dev/SuPy/issues/new
n_stream_lw_urban 4 <class 'numpy.int32'> is not included: this should not happen! please report to the developer: https://github.com/UMEP-dev/SuPy/issues/new
2022-07-07 17:40:06,322 - SuPy - INFO - All cache cleared.
n_vegetation_region_urban 1 <class 'numpy.int32'> is not included: this should not happen! please report to the developer: https://github.com/UMEP-dev/SuPy/issues/new
n_stream_sw_urban 4 <class 'numpy.int32'> is not included: this should not happen! please report to the developer: https://github.com/UMEP-dev/SuPy/issues/new
n_stream_lw_urban 4 <class 'numpy.int32'> is not included: this should not happen! please report to the developer: https://github.com/UMEP-dev/SuPy/issues/new
2022-07-07 17:42:20,762 - SuPy - INFO - ====================
2022-07-07 17:42:20,763 - SuPy - INFO - Simulation period:
2022-07-07 17:42:20,764 - SuPy - INFO -   Start: 2020-08-01 02:05:00
2022-07-07 17:42:20,764 - SuPy - INFO -   End: 2020-08-02 02:00:00
2022-07-07 17:42:20,765 - SuPy - INFO - 
2022-07-07 17:42:20,766 - SuPy - INFO - No. of grids: 8976
2022-07-07 17:42:20,766 - SuPy - INFO - SuPy is running in serial mode
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File c:\users\rslab20emm\appdata\local\programs\python\python39\lib\site-packages\supy\_run.py:368, in run_supy_ser(df_forcing, df_state_init, save_state, chunk_day)
    367 try:
--> 368     list_res = [
    369         suews_cal_tstep_multi(dict_input_grid, df_forcing)
    370         for dict_input_grid in list_dict_input
    371     ]
    372     list_state_end, list_output_array = zip(*list_res)

File c:\users\rslab20emm\appdata\local\programs\python\python39\lib\site-packages\supy\_run.py:369, in <listcomp>(.0)
    367 try:
    368     list_res = [
--> 369         suews_cal_tstep_multi(dict_input_grid, df_forcing)
    370         for dict_input_grid in list_dict_input
    371     ]
    372     list_state_end, list_output_array = zip(*list_res)

File c:\users\rslab20emm\appdata\local\programs\python\python39\lib\site-packages\supy\_run.py:116, in suews_cal_tstep_multi(dict_state_start_grid, df_met_forcing_block)
    100 dict_input.update(
    101     {
    102         "metforcingblock": np.array(
   (...)
    114     }
    115 )
--> 116 dict_input = {k: dict_input[k] for k in list_var_input_multitsteps}
    118 # main calculation:

File c:\users\rslab20emm\appdata\local\programs\python\python39\lib\site-packages\supy\_run.py:116, in <dictcomp>(.0)
    100 dict_input.update(
    101     {
    102         "metforcingblock": np.array(
   (...)
    114     }
    115 )
--> 116 dict_input = {k: dict_input[k] for k in list_var_input_multitsteps}
    118 # main calculation:

KeyError: 'n_vegetation_region_urban'

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
Input In [10], in <cell line: 22>()
     25 print(start_date)
     26 dataf(start_date,filefolder,dfmetdata)
---> 27 suews_out=suews(filefolder)
     28 tiffs(filefolder,suews_out)
     29 del suews_out

Input In [6], in suews(filefolder)
    458 grid = df_state_init.index[0]
    459 df_forcing = sp.load_forcing_grid(path_runcontrol, grid)  
--> 460 df_output,df_state_final=sp.run_supy(df_forcing,df_state_init,chunk_day=1)
    462 #df_output,df_state_final=sp.run_supy(df_forcing,df_state_init,chunk_day=10)
    463 #print("ok")
    464 path_dir_save=filefolder+"resultstxt"

File c:\users\rslab20emm\appdata\local\programs\python\python39\lib\site-packages\supy\_supy_module.py:370, in run_supy(df_forcing, df_state_init, save_state, chunk_day, logging_level, check_input, serial_mode)
    368 else:
    369     logger_supy.info(f"SuPy is running in serial mode")
--> 370     df_output, df_state_final = run_supy_ser(
    371         df_forcing, df_state_init, save_state, chunk_day
    372     )
    374 # show simulation time
    375 end = time.time()

File c:\users\rslab20emm\appdata\local\programs\python\python39\lib\site-packages\supy\_run.py:375, in run_supy_ser(df_forcing, df_state_init, save_state, chunk_day)
    372     list_state_end, list_output_array = zip(*list_res)
    374 except:
--> 375     raise RuntimeError("SUEWS kernel error")
    377 # collect output arrays
    378 dict_output = {
    379     grid: dict_output_array
    380     for grid, dict_output_array in zip(list_grid, list_output_array)
    381 }

RuntimeError: SUEWS kernel error

[sunt05]

@Empanag have you got the new version running? if not, please leave your input dataset here and I'll have a look recently.

[Empanag]

Hi Ting, i tried some runs and i have this error

2022-07-29 11:34:06,148 - SuPy - INFO - All cache cleared.
2022-07-29 11:34:06,148 - SuPy - ERROR - E:\SUEWSapp\testirrig\7-ir-25sm9-10-11-12\SUEWS_SPARTACUS.nml does not exists!
Traceback (most recent call last):
  File "C:\Users\RSLab20Emm\AppData\Local\Programs\Python\Python39\lib\site-packages\supy\_load.py", line 245, in load_SUEWS_nml
    df_nml = pd.DataFrame(parser.read(p_nml))
  File "C:\Users\RSLab20Emm\AppData\Roaming\Python\Python39\site-packages\f90nml\parser.py", line 279, in read
    nml_file = open(nml_fname, 'r') if nml_is_path else nml_fname
FileNotFoundError: [Errno 2] No such file or directory: 'E:\\SUEWSapp\\testirrig\\7-ir-25sm9-10-11-12\\SUEWS_SPARTACUS.nml'
Traceback (most recent call last):
  File "E:\SUEWSapp\testirrig\7-ir-25sm9-10-11-12\suews.py", line 639, in <module>
    suews_out=suews(filefolder)
  File "E:\SUEWSapp\testirrig\7-ir-25sm9-10-11-12\suews.py", line 445, in suews
    df_state_init = sp.init_supy(path_runcontrol)
  File "C:\Users\RSLab20Emm\AppData\Local\Programs\Python\Python39\lib\site-packages\supy\_supy_module.py", line 103, in init_supy
    df_state_init = load_InitialCond_grid_df(
  File "C:\Users\RSLab20Emm\AppData\Local\Programs\Python\Python39\lib\site-packages\supy\_load.py", line 1751, in load_InitialCond_grid_df
    df_init = load_SUEWS_InitialCond_df(path_runcontrol)
  File "C:\Users\RSLab20Emm\AppData\Local\Programs\Python\Python39\lib\site-packages\supy\_load.py", line 1365, in load_SUEWS_InitialCond_df
    dict_ModConfig = load_SUEWS_dict_ModConfig(path_runcontrol)
  File "C:\Users\RSLab20Emm\AppData\Local\Programs\Python\Python39\lib\site-packages\supy\_load.py", line 1281, in load_SUEWS_dict_ModConfig
    dict_RunControl_x = {k[0]: v for k, v in load_SUEWS_nml(path_spartacus).items()}
AttributeError: 'NoneType' object has no attribute 'items'

[sunt05]

Did you place the Spartacus.nml file in that directory?

[Empanag]

yes i placed it, the previous comment didn't have the right error this is the real one with SUEWS_spartacus.nml and GridLayout.nml

2020-06-05 23:00:00                                                      ]   0%
2022-07-29 14:30:27,886 - SuPy - INFO - All cache cleared.
2022-07-29 14:30:27,886 - SuPy - ERROR - E:\SUEWSapp\testirrig\7-ir-25sm9-10-11-12\SUEWS_SPARTACUS.nml does not exists!
Traceback (most recent call last):
  File "C:\Users\RSLab20Emm\AppData\Local\Programs\Python\Python39\lib\site-packages\supy\_load.py", line 245, in load_SUEWS_nml
    df_nml = pd.DataFrame(parser.read(p_nml))
  File "C:\Users\RSLab20Emm\AppData\Roaming\Python\Python39\site-packages\f90nml\parser.py", line 279, in read
    nml_file = open(nml_fname, 'r') if nml_is_path else nml_fname
FileNotFoundError: [Errno 2] No such file or directory: 'E:\\SUEWSapp\\testirrig\\7-ir-25sm9-10-11-12\\SUEWS_SPARTACUS.nml'
Traceback (most recent call last):
  File "E:\SUEWSapp\testirrig\7-ir-25sm10-11-12-13\suews.py", line 639, in <module>
    suews_out=suews(filefolder)
  File "E:\SUEWSapp\testirrig\7-ir-25sm10-11-12-13\suews.py", line 445, in suews
    df_state_init = sp.init_supy(path_runcontrol)
  File "C:\Users\RSLab20Emm\AppData\Local\Programs\Python\Python39\lib\site-packages\supy\_supy_module.py", line 103, in init_supy
    df_state_init = load_InitialCond_grid_df(
  File "C:\Users\RSLab20Emm\AppData\Local\Programs\Python\Python39\lib\site-packages\supy\_load.py", line 1751, in load_InitialCond_grid_df
    df_init = load_SUEWS_InitialCond_df(path_runcontrol)
  File "C:\Users\RSLab20Emm\AppData\Local\Programs\Python\Python39\lib\site-packages\supy\_load.py", line 1365, in load_SUEWS_InitialCond_df
    dict_ModConfig = load_SUEWS_dict_ModConfig(path_runcontrol)
  File "C:\Users\RSLab20Emm\AppData\Local\Programs\Python\Python39\lib\site-packages\supy\_load.py", line 1281, in load_SUEWS_dict_ModConfig
    dict_RunControl_x = {k[0]: v for k, v in load_SUEWS_nml(path_spartacus).items()}
AttributeError: 'NoneType' object has no attribute 'items'

the files inside the folder are

07-Jul-22 05:37 PM 4,189 ftod_2020_data_60.txt 07-Jul-22 05:36 PM 4,017 GridLayoutftod.nml 07-May-22 04:10 PM 482 InitialConditionsftod_2020.nml 07-Jul-22 05:37 PM 902 RunControl.nml 09-May-22 01:07 AM 887 RunControl.nml.bak 29-Jul-22 02:30 PM 26,643 suews.py 12-May-22 02:12 AM 26,622 suews.py.bak 07-May-22 04:10 PM 6,296 SUEWS_AnthropogenicEmission.txt 07-May-22 04:10 PM 835 SUEWS_BiogenCO2.txt 07-May-22 04:10 PM 450 SUEWS_Conductance.txt 07-May-22 04:10 PM 12,715 SUEWS_ESTMCoefficients.txt 07-May-22 04:10 PM 2,154 SUEWS_Irrigation.txt 07-May-22 04:10 PM 3,652 SUEWS_NonVeg.txt 07-May-22 04:10 PM 6,589 SUEWS_OHMCoefficients.txt 07-May-22 04:10 PM 5,386 SUEWS_Profiles.txt 07-May-22 04:10 PM 3,880,118 SUEWS_SiteSelect.txt 07-May-22 04:10 PM 1,664 SUEWS_Snow.txt 07-May-22 04:10 PM 1,837 SUEWS_Soil.txt 07-Jul-22 05:36 PM 423 SUEWS_SPARTACUS.nml 07-Jul-22 05:36 PM 0 SUEWS_SPARTACUS.nml.bak 07-May-22 04:10 PM 6,816 SUEWS_Veg.txt 07-May-22 04:10 PM 1,519 SUEWS_Water.txt 07-May-22 04:10 PM 2,419 SUEWS_WithinGridWaterDist.txt

[sunt05]

Please zip the whole folder and post it here - I'll have a look later on today.

[Empanag]

7-ir-25sm10-11-12-13.zip

you can run the code inside by changing the line 591 to the folder that all of these are exported.

[sunt05]

I just had a try - the issue is in your suews.py file, in which the filefolder in line 624 overrides the same variable set in line 591.

And if you notice the line in the error message above -

FileNotFoundError: [Errno 2] No such file or directory: 'E:\\SUEWSapp\\testirrig\\7-ir-25sm9-10-11-12\\SUEWS_SPARTACUS.nml'

you might have got the clue.

[Empanag]

yes you are right for this. I change it and now i have this error. (everything is the same except the 624 line that i erased it)

Traceback (most recent call last):
  File "C:\Users\RSLab20Emm\AppData\Local\Programs\Python\Python39\lib\site-packages\supy\_run.py", line 368, in run_supy_ser
    list_res = [
  File "C:\Users\RSLab20Emm\AppData\Local\Programs\Python\Python39\lib\site-packages\supy\_run.py", line 369, in <listcomp>
    suews_cal_tstep_multi(dict_input_grid, df_forcing)
  File "C:\Users\RSLab20Emm\AppData\Local\Programs\Python\Python39\lib\site-packages\supy\_run.py", line 116, in suews_cal_tstep_multi
    dict_input = {k: dict_input[k] for k in list_var_input_multitsteps}
  File "C:\Users\RSLab20Emm\AppData\Local\Programs\Python\Python39\lib\site-packages\supy\_run.py", line 116, in <dictcomp>
    dict_input = {k: dict_input[k] for k in list_var_input_multitsteps}
KeyError: 'n_vegetation_region_urban'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\SUEWSapp\testirrig\7-ir-25sm10-11-12-13\suews.py", line 639, in <module>
    suews_out=suews(filefolder)
  File "E:\SUEWSapp\testirrig\7-ir-25sm10-11-12-13\suews.py", line 453, in suews
    df_output,df_state_final=sp.run_supy(df_forcing,df_state_init,chunk_day=1)
  File "C:\Users\RSLab20Emm\AppData\Local\Programs\Python\Python39\lib\site-packages\supy\_supy_module.py", line 370, in run_supy
    df_output, df_state_final = run_supy_ser(
  File "C:\Users\RSLab20Emm\AppData\Local\Programs\Python\Python39\lib\site-packages\supy\_run.py", line 375, in run_supy_ser
    raise RuntimeError("SUEWS kernel error")
RuntimeError: SUEWS kernel error

[sunt05]

Hopefully, my answer above addresses this issue, so I'll close it for now. Otherwise, feel free to re-open and provide more details to help resolve it.