[SuPy Testing] Impact Studies Using SuPy - NameError: name 'df_res_alb_test' is not defined

[rarygit]

Running Tutorial "Impact Studies Using SuPy" using same Ubuntu 20.04 OS etc as described in earlier posts. Asciidoc output attached. https://supy.readthedocs.io/en/latest/tutorial/impact-studies.html

Choose results of July 2012 for analysis:

df_res_alb_test_july = df_res_alb_test.SUEWS.unstack(0).loc["2012 7"]
df_res_alb_T2_stat = df_res_alb_test_july.T2.describe()
df_res_alb_T2_diff = df_res_alb_T2_stat.transform(
    lambda x: x - df_res_alb_T2_stat.iloc[:, 0]
)
df_res_alb_T2_diff.columns = list_alb_test - list_alb_test[0]

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Input In [11], in <module>
      1 # choose results of July 2012 for analysis
----> 2 df_res_alb_test_july = df_res_alb_test.SUEWS.unstack(0).loc["2012 7"]
      3 df_res_alb_T2_stat = df_res_alb_test_july.T2.describe()
      4 df_res_alb_T2_diff = df_res_alb_T2_stat.transform(
      5     lambda x: x - df_res_alb_T2_stat.iloc[:, 0]
      6 )

NameError: name 'df_res_alb_test' is not defined

Error_Impact Studies Using SuPy.zip

[sunt05]

Hi @rarygit , it seems you might somehow forget to run cell 7 as shown in the figure above.

[rarygit]

Thanks for replying Ting!

I now looked the Terminal while I ran the Jupyter notebook. It has reported an Error: ImportError: Missing optional dependency 'tables'. Use pip or conda to install tables. Now trying "pip3 install tables"

+++++++++++++++++++++++++++++++++++++ As below:

[I 11:58:39.230 NotebookApp] Saving file at /Untitled28.ipynb
[IPKernelApp] ERROR | Exception in message handler:
multiprocess.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/l1nux/.local/lib/python3.9/site-packages/multiprocess/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/l1nux/.local/lib/python3.9/site-packages/multiprocess/pool.py", line 51, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "/home/l1nux/.local/lib/python3.9/site-packages/supy/_run.py", line 422, in run_save_supy
    df_output.to_hdf(path_out, f"out_{ind}", mode="w")
  File "/home/l1nux/.local/lib/python3.9/site-packages/pandas/core/generic.py", line 2703, in to_hdf
    pytables.to_hdf(
  File "/home/l1nux/.local/lib/python3.9/site-packages/pandas/io/pytables.py", line 311, in to_hdf
    with HDFStore(
  File "/home/l1nux/.local/lib/python3.9/site-packages/pandas/io/pytables.py", line 572, in __init__
    tables = import_optional_dependency("tables")
  File "/home/l1nux/.local/lib/python3.9/site-packages/pandas/compat/_optional.py", line 118, in import_optional_dependency
    raise ImportError(msg) from None
ImportError: Missing optional dependency 'tables'.  Use pip or conda to install tables.

The above exception was the direct cause of the following exception:

... etc etc

[sunt05]

I see: you need to install pytables: conda install pytables

[rarygit]

Hi Ting, I have installed tables and pytables and then ran Cells 1 to 7 in the Tutorial. This would be Cells 2 to ... "8" in my notebook on Python 3 The jupyter kernel crashes and does not complete Cell 8.

Here is the function: Conduct simulations with supy: df_forcing_part = df_forcing.loc['2012 01':'2012 07'] df_res_alb_test,df_state_final_x = sp.run_supy(df_forcing_part, df_state_init_x,logging_level=90)

Running this Cell in the notebook immediately creates a folder in the Filesystem: /tmp/tmpux1fmnts

Folder 'tmpux1fmnts' grows rapidly in size until 9.8 Gb; why so big? Is this expected? And then the kernel crashes with 3.6 Gb free space in the Filesystem.

I haven't checked the example to understand why the tmp files should expand to 9.6 Gb. Before that, the /tmp folder was 2.2 Mb in size.

Any suggestions? Attached is the exported asciidoc And a screenshot.

Untitled28.asciidoc.zip

[sunt05]

Hi @rarygit sorry to learn the trouble. to be honest, I've no much idea why the tmp folder would expand so heavily: perhaps this simulation involves a suite of multiple settings? try to reduce the timespan for a shorter period and see if it can run through: e.g. df_forcing_part = df_forcing.loc['2012 01':'2012 03']

[rarygit]

Hi Ting, that worked by shortening the timespan to 3 i.e. replacing '2012 07' and '2012 7' with '2012 03' and '2012 3' in five Cells, [7], [8], [11], [17] & [19]

/tmp folder increases to 2 Mb to 4 Gb and then reduces to 2 Mb after running "Conduct simulations with supy", Cell [7].

With those changes the whole Tutorial was completed with plotted charts, which are attached below.

[sunt05]

Glad to know all is working now 🎉

[rarygit]

We can close this.