Feature/#532 update to plotpy

[Alonsoju95]

changes in plot.py in calc_network_expansion to plot only DC links and in v_nom

[CarlosEpia]

When I try etrago.plot_grid(line_colors= "v_nom" , bus_sizes= 0.001), the result looks like the figure below. I can see that there are some buses which are nor connected. Are they not electrical? in that case, they should not be plotted.

[CarlosEpia]

The option line_colors= "line_loading" is not working for me. Does it work in your machine?

[CarlosEpia]

The default value fo bus_sizes produces plots like this. I would suggest finding a better one. Additionally, the message "WARNING:etrago.tools.plot:bus_color grey undefined" appears.

[CarlosEpia]

The network expansion looks like the figure below. Since you are showing just electrical network expansion, the other sectors are not supposed to be displayed. This makes the extension of the map to go until Rusia, but would be better to stay around Germany.

[Alonsoju95]

Hi @CarlosEpia

I made some changes and now its only plotting AC buses in expansion_abs plot I will continue working on the other comments

[Alonsoju95]

Hi @CarlosEpia

This is how etrago.plot_grid(line_colors= "v_nom" , bus_sizes= 0.001) looks like now. I have just pushed the changes

[Alonsoju95]

Hi @CarlosEpia I have made the required changes. Can you take a look again? Thanks Alonso

[CarlosEpia]

Hi @CarlosEpia I have made the required changes. Can you take a look again? Thanks Alonso

Hi Alonso. Please send me by email plots where we can see the current state of each plot available in the function plot_grid(). Then I will proceed to check the code.

[CarlosEpia]

Hi @ClaraBuettner, you can find below the commands I used to test the function in case it is useful for the review: `#line_colors etrago.plot_grid(line_colors= "line_loading")

etrago.plot_grid(line_colors= "dlr")

etrago.plot_grid(line_colors= "grey")

etrago.plot_grid(line_colors= "v_nom")

etrago.plot_grid(line_colors= "expansion_abs")

etrago.plot_grid(line_colors= "expansion_rel")

etrago.plot_grid(line_colors= "q_flow_max")

bus_colors

etrago.plot_grid("expansion_abs", bus_colors= "nodal_production_balance", bus_sizes= 0.0000001, scaling_store_expansion= None, timesteps=range(50),)

etrago.plot_grid("expansion_abs", bus_colors= "storage_distribution", scaling_store_expansion= None, bus_sizes= 0.0001)

etrago.plot_grid("expansion_abs", bus_colors= "storage_expansion", bus_sizes= 0.000001)

etrago.plot_grid("expansion_abs", bus_colors= "h2_battery_storage_expansion", bus_sizes= 0.00001, scaling_store_expansion = {"H2_overground": 1,"H2_underground": 1,"battery": 1,}, legend_entries = ["battery", "H2_overground", "H2_underground"])

etrago.plot_grid("expansion_abs", bus_colors= "gen_dist", scaling_store_expansion= None, bus_sizes= 0.000001, timesteps=range(50))

etrago.plot_grid("expansion_abs", bus_colors= "PowerToH2", scaling_store_expansion= None)

etrago.plot_grid("expansion_abs", bus_colors= "flexibility_usage", scaling_store_expansion= None, bus_sizes= 0.0000001)`

[ClaraBuettner]

I tried out some of your update plots but run into different kind of problems

`#line_colors etrago.plot_grid(line_colors= "line_loading")

  link_load[i] = load.abs().max(axis=1)
Traceback (most recent call last):

  File "/tmp/ipykernel_72289/979504549.py", line 1, in <cell line: 1>
    etrago.plot_grid(line_colors= "line_loading")

  File "/home/clara/etrago-dev/eTraGo/etrago/tools/plot.py", line 2570, in plot_grid
    link_colors = calc_dc_loading(network, timesteps).abs() / rep_snapshots

  File "/home/clara/etrago-dev/eTraGo/etrago/tools/plot.py", line 1778, in calc_dc_loading
    link_load[i] = load.abs().max(axis=1)

  File "/home/clara/etrago-dev/venv-new/lib/python3.8/site-packages/pandas/core/frame.py", line 3960, in __setitem__
    return self._setitem_slice(indexer, value)

  File "/home/clara/etrago-dev/venv-new/lib/python3.8/site-packages/pandas/core/frame.py", line 3984, in _setitem_slice
    self.iloc[key] = value

  File "/home/clara/etrago-dev/venv-new/lib/python3.8/site-packages/pandas/core/indexing.py", line 818, in __setitem__
    iloc._setitem_with_indexer(indexer, value, self.name)

  File "/home/clara/etrago-dev/venv-new/lib/python3.8/site-packages/pandas/core/indexing.py", line 1795, in _setitem_with_indexer
    self._setitem_with_indexer_split_path(indexer, value, name)

  File "/home/clara/etrago-dev/venv-new/lib/python3.8/site-packages/pandas/core/indexing.py", line 1879, in _setitem_with_indexer_split_path
    raise ValueError(

ValueError: Must have equal len keys and value when setting with an iterable

etrago.plot_grid(line_colors= "dlr")

Fatal Python error: Aborted

Main thread:
Current thread 0x00007f8cb761d740 (most recent call first):
  File "/home/clara/etrago-dev/venv-new/lib/python3.8/site-packages/cartopy/crs.py", line 814 in _project_line_string
  File "/home/clara/etrago-dev/venv-new/lib/python3.8/site-packages/cartopy/crs.py", line 808 in project_geometry
  File "/home/clara/etrago-dev/venv-new/lib/python3.8/site-packages/cartopy/mpl/geoaxes.py", line 860 in set_extent
  File "/home/clara/etrago-dev/venv-new/lib/python3.8/site-packages/pypsa/plot.py", line 211 in plot
  File "/home/clara/etrago-dev/eTraGo/etrago/tools/plot.py", line 1928 in plot_background_grid
  File "/home/clara/etrago-dev/eTraGo/etrago/tools/plot.py", line 2649 in plot_grid
  File "/tmp/ipykernel_72289/2744401509.py", line 1 in <cell line: 1>

Restarting kernel...

etrago.plot_grid(line_colors= "grey")

etrago.plot_grid(line_colors= "grey")

WARNING:etrago.tools.plot:line_color grey undefined
Traceback (most recent call last):

  File "/tmp/ipykernel_75586/2250661082.py", line 1, in <cell line: 1>
    etrago.plot_grid(line_colors= "grey")

  File "/home/clara/etrago-dev/eTraGo/etrago/tools/plot.py", line 2807, in plot_grid
    link_colors=link_colors,

UnboundLocalError: local variable 'link_colors' referenced before assignment

etrago.plot_grid(line_colors= "v_nom")

see dlr plot

[ClaraBuettner]

Sorry, it looks like a had to pull again. Now I seem to run into the cartopy-restarting-kernal-problem for all settings.

[CarlosEpia]

Sorry, it looks like a had to pull again. Now I seem to run into the cartopy-restarting-kernal-problem for all settings.

I created a new virtual environment to try to reproduce the described error but it also worked with and without Cartopy. I used the command "pip cache purge" before installing eTraGo as we discussed. Could you try the same?

[ClaraBuettner]

It is working in a new environment. It would still be good to know which package version created the problem to adjust the installation of eTraGo.

The plots you mentioned above are working, I didn't tested all of them, bus some. But when I use other settings, e.g. this one:

etrago.plot_grid(
    "expansion_abs",
    ext_width=4000,
    filename="test.png",
)

it fails:

/home/clara/etrago-dev/venv-plots/lib/python3.8/site-packages/pypsa/plot.py:259: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead.
  for i, ratio in ratios.iteritems():
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/clara/etrago-dev/eTraGo/etrago/tools/plot.py", line 2627, in plot_grid
    ll = network.plot(
  File "/home/clara/etrago-dev/venv-plots/lib/python3.8/site-packages/pypsa/plot.py", line 347, in plot
    b_widths = as_branch_series(branch_widths[c.name], "width", c.name, n)
  File "/home/clara/etrago-dev/venv-plots/lib/python3.8/site-packages/pypsa/plot.py", line 420, in as_branch_series
    assert not ser.isnull().any(), (
AssertionError: Link_widths does not specify all entries. Missing values for Link: ['0', '524', '525', '526', '527', '528', '529', '530', '531', '532', '533', '534', '535', '536', '537', '538', '539', '540', '541', '542', '543', '544', '545', '546', '547', '548', '549', '550',  ....

Have you ever tried out this "ext_width"option?

Also, the legend entries are not completely visible:

[CarlosEpia]

It is working in a new environment. It would still be good to know which package version created the problem to adjust the installation of eTraGo.

The plots you mentioned above are working, I didn't tested all of them, bus some. But when I use other settings, e.g. this one:

etrago.plot_grid(
    "expansion_abs",
    ext_width=4000,
    filename="test.png",
)

it fails:

/home/clara/etrago-dev/venv-plots/lib/python3.8/site-packages/pypsa/plot.py:259: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead.
  for i, ratio in ratios.iteritems():
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/clara/etrago-dev/eTraGo/etrago/tools/plot.py", line 2627, in plot_grid
    ll = network.plot(
  File "/home/clara/etrago-dev/venv-plots/lib/python3.8/site-packages/pypsa/plot.py", line 347, in plot
    b_widths = as_branch_series(branch_widths[c.name], "width", c.name, n)
  File "/home/clara/etrago-dev/venv-plots/lib/python3.8/site-packages/pypsa/plot.py", line 420, in as_branch_series
    assert not ser.isnull().any(), (
AssertionError: Link_widths does not specify all entries. Missing values for Link: ['0', '524', '525', '526', '527', '528', '529', '530', '531', '532', '533', '534', '535', '536', '537', '538', '539', '540', '541', '542', '543', '544', '545', '546', '547', '548', '549', '550',  ....

Have you ever tried out this "ext_width"option?

Also, the legend entries are not completely visible:

The described problems were solved in the commits 569303d, 55bfbdddc941dfc9f04c8a1be73f9801b238e798 and 82eae3a16e9d755e962571dd0479cc30a13027e9

[CarlosEpia]

I just went through the different plots and they look OK. I think it is worth giving it a try @ClaraBuettner.

[ClaraBuettner]

When I try out etrago.plot_grid(line_colors= "expansion_abs") or etrago.plot_grid(line_colors= "expansion_rel") I run into this error:

Traceback (most recent call last):

  Cell In[7], line 1
    etrago.plot_grid(line_colors= "expansion_abs")

  File ~/etrago-powerd/eTraGo/etrago/tools/plot.py:2481 in plot_grid
    all_network, line_colors, link_colors = calc_network_expansion(

  File ~/etrago-powerd/eTraGo/etrago/tools/plot.py:1738 in calc_network_expansion
    extension_links[extension_links.carrier != "DC"] = 0

  File ~/etrago-powerd/vemv/lib/python3.8/site-packages/pandas/core/frame.py:3968 in __setitem__
    self._setitem_array(key, value)

  File ~/etrago-powerd/vemv/lib/python3.8/site-packages/pandas/core/frame.py:4003 in _setitem_array
    self.iloc[indexer] = value

  File ~/etrago-powerd/vemv/lib/python3.8/site-packages/pandas/core/indexing.py:818 in __setitem__
    iloc._setitem_with_indexer(indexer, value, self.name)

  File ~/etrago-powerd/vemv/lib/python3.8/site-packages/pandas/core/indexing.py:1795 in _setitem_with_indexer
    self._setitem_with_indexer_split_path(indexer, value, name)

  File ~/etrago-powerd/vemv/lib/python3.8/site-packages/pandas/core/indexing.py:1888 in _setitem_with_indexer_split_path
    self._setitem_single_column(loc, value, pi)

  File ~/etrago-powerd/vemv/lib/python3.8/site-packages/pandas/core/indexing.py:1992 in _setitem_single_column
    self.obj._mgr.column_setitem(loc, plane_indexer, value)

  File ~/etrago-powerd/vemv/lib/python3.8/site-packages/pandas/core/internals/managers.py:1391 in column_setitem
    new_mgr = col_mgr.setitem((idx,), value)

  File ~/etrago-powerd/vemv/lib/python3.8/site-packages/pandas/core/internals/managers.py:393 in setitem
    return self.apply("setitem", indexer=indexer, value=value)

  File ~/etrago-powerd/vemv/lib/python3.8/site-packages/pandas/core/internals/managers.py:352 in apply
    applied = getattr(b, f)(**kwargs)

  File ~/etrago-powerd/vemv/lib/python3.8/site-packages/pandas/core/internals/blocks.py:1417 in setitem
    values[indexer] = value

  File ~/etrago-powerd/vemv/lib/python3.8/site-packages/geopandas/array.py:421 in __setitem__
    raise TypeError(

TypeError: Value should be either a BaseGeometry or None, got 0

[ClaraBuettner]

etrago.plot_grid("v_nom", bus_colors= "PowerToH2", scaling_store_expansion= None) does not return any error message, but the limit for PowerToH2 expansion is nan:

[CarlosEpia]

The mentioned problems were already addressed.