incorporate into MARLlib

[Theohhhu]

Hi there!

I came across your project and wanted to say great job on the PDN implementation! As the author of MARLlib, I was wondering if you would be open to providing a simple example for PDN, similar to this. We are looking to include PDN in our framework and this would be really helpful to us.

Thanks. Theo

[hsvgbkhgbv]

Hi Theo,

I am very pleased to see that you are interested in our work. I will give a simple example like this soon. When I finished, I would notify you.

Best, Jianhong

[hsvgbkhgbv]

Hi Theo,

I have updated README, please see the section Interaction with Environment that shows the simple example you required.

[Theohhhu]

I have tried running your example code and get the following result:

This is the s_max: 
[2.20330932 2.20330932 2.14085598 3.5906096  0.33415149 0.33415149]
Total reward in epsiode 0 = -8.72
The power flow for the reactive power penetration cannot be solved.
This is the pv: 
0    0.229314
1    0.229792
2    0.129680
3    0.266771
4    0.036425
5    0.036186
Name: p_mw, dtype: float64
This is the q: 
0   -1.708888
1   -2.624564
2    0.710579
3    1.640819
4   -0.116186
5    0.126108
Name: q_mvar, dtype: float64
This is the active demand: 
0     0.043792
1     0.070328
2     0.084235
3     0.027576
4     0.018318
5     0.129076
6     0.110572
7     0.046629
8     0.045406
9     0.018839
10    0.036170
11    0.042453
12    0.011296
13    0.019775
14    0.046345
15    0.035680
16    0.032540
17    0.035347
18    0.043661
19    0.053613
20    0.057338
21    0.049478
22    0.288953
23    0.238046
24    0.042373
25    0.037630
26    0.037778
27    0.087373
28    0.150511
29    0.114435
30    0.076290
31    0.044779
Name: p_mw, dtype: float64
This is the reactive demand: 
0     0.022930
1     0.036108
2     0.211971
3     0.013835
4     0.006813
5     0.072876
6     0.057123
7     0.017509
8     0.017444
9     0.013085
10    0.023907
11    0.022003
12    0.007493
13    0.007508
14    0.016044
15    0.009594
16    0.012430
17    0.012977
18    0.022295
19    0.029663
20    0.023095
21    0.028877
22    0.166988
23    0.123631
24    0.022058
25    0.012105
26    0.017869
27    0.059874
28    0.472175
29    0.057778
30    0.033632
31    0.032090
Name: q_mvar, dtype: float64
This is the res_bus: 
    vm_pu  va_degree  p_mw  q_mvar
0     NaN        NaN   NaN     NaN
1     NaN        NaN   NaN     NaN
2     NaN        NaN   NaN     NaN
3     NaN        NaN   NaN     NaN
4     NaN        NaN   NaN     NaN
5     NaN        NaN   NaN     NaN
6     NaN        NaN   NaN     NaN
7     NaN        NaN   NaN     NaN
8     NaN        NaN   NaN     NaN
9     NaN        NaN   NaN     NaN
10    NaN        NaN   NaN     NaN
11    NaN        NaN   NaN     NaN
12    NaN        NaN   NaN     NaN
13    NaN        NaN   NaN     NaN
14    NaN        NaN   NaN     NaN
15    NaN        NaN   NaN     NaN
16    NaN        NaN   NaN     NaN
17    NaN        NaN   NaN     NaN
18    NaN        NaN   NaN     NaN
19    NaN        NaN   NaN     NaN
20    NaN        NaN   NaN     NaN
21    NaN        NaN   NaN     NaN
22    NaN        NaN   NaN     NaN
23    NaN        NaN   NaN     NaN
24    NaN        NaN   NaN     NaN
25    NaN        NaN   NaN     NaN
26    NaN        NaN   NaN     NaN
27    NaN        NaN   NaN     NaN
28    NaN        NaN   NaN     NaN
29    NaN        NaN   NaN     NaN
30    NaN        NaN   NaN     NaN
31    NaN        NaN   NaN     NaN
32    NaN        NaN   NaN     NaN
Episode terminated at time: 97 with return: -209.7751.
Total reward in epsiode 1 = -210.10
Total reward in epsiode 2 = -10.04

Please let me know if these results are accurate or if there are any issues that need to be addressed.

[hsvgbkhgbv]

Hi Theo,

This result is normal, since the random actions are out of the feasible region so as to incur the damage to the power system. Actually, it is one of the challeneges for this task, to estimate the boundary of capicity of the system. As you see, if the action is out of range, the agent would receive a penalty of -200 as reward.

Best, Jianhong

[Theohhhu]

Incorporated here. Thank you for your assistance. However, I found that 'case322_3min_final' is not working, while the other two scenarios are working just fine. Please let me know if you can help with a pull request to fix this issue for this part.

[hsvgbkhgbv]

Could you please share the error information here?

[Theohhhu]

Traceback (most recent call last):
  File "/home/hsy/anaconda3/envs/harl/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 3361, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas/_libs/index.pyx", line 76, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'type'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/hsy/PycharmProjects/MARLlib/examples/dev_new.py", line 5, in <module>
    env = marl.make_env(environment_name="voltage", map_name="case322_3min_final")
  File "/home/hsy/PycharmProjects/MARLlib/marllib/marl/__init__.py", line 150, in make_env
    env = ENV_REGISTRY[env_config["env"]](env_config["env_args"])
  File "/home/hsy/PycharmProjects/MARLlib/marllib/envs/base_env/voltage.py", line 70, in __init__
    self.env = VoltageControl(env_config)
  File "/home/hsy/PycharmProjects/MARLlib/marllib/patch/dpn/var_voltage_control/voltage_control_env.py", line 53, in __init__
    self.base_powergrid = self._load_network()
  File "/home/hsy/PycharmProjects/MARLlib/marllib/patch/dpn/var_voltage_control/voltage_control_env.py", line 402, in _load_network
    base_net = pp.from_pickle(network_path)
  File "/home/hsy/anaconda3/envs/harl/lib/python3.7/site-packages/pandapower/file_io.py", line 181, in from_pickle
    convert_format(net)
  File "/home/hsy/anaconda3/envs/harl/lib/python3.7/site-packages/pandapower/convert_format.py", line 35, in convert_format
    _add_missing_columns(net, elements_to_deserialize)
  File "/home/hsy/anaconda3/envs/harl/lib/python3.7/site-packages/pandapower/convert_format.py", line 281, in _add_missing_columns
    net.sgen["current_source"] = net.sgen["type"].apply(
  File "/home/hsy/anaconda3/envs/harl/lib/python3.7/site-packages/pandas/core/frame.py", line 3458, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/hsy/anaconda3/envs/harl/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 3363, in get_loc
    raise KeyError(key) from err
KeyError: 'type'

Like this. Only case322_3min_final trigger this error.

[hsvgbkhgbv]

Hi Theo,

Could you please try installing pandas==1.1.0 ? Let me know whether it works.

Thanks, Jianhong

[Theohhhu]

Still encountering the same error. Here is the package information for further analysis.

Package                  Version
------------------------ --------------
absl-py                  1.4.0
appdirs                  1.4.4
asttokens                2.2.1
async-timeout            4.0.2
attrs                    23.1.0
cachetools               5.3.0
certifi                  2022.12.7
cffi                     1.15.1
charset-normalizer       3.0.1
click                    8.1.3
cloudpickle              2.2.1
colorama                 0.4.6
cycler                   0.11.0
Cython                   0.29.33
deepdiff                 6.2.3
dm-env                   1.6
dm-env-rpc               1.1.4
dm-tree                  0.1.8
dnspython                2.3.0
docker-pycreds           0.4.0
docopt                   0.6.2
enum34                   1.1.10
executing                1.2.0
fasteners                0.18
filelock                 3.12.0
fonttools                4.38.0
future                   0.18.3
geographiclib            2.0
gitdb                    4.0.10
GitPython                3.1.31
glfw                     2.5.6
google-auth              2.16.0
google-auth-oauthlib     0.4.6
googleapis-common-protos 1.58.0
grpcio                   1.51.1
gym                      0.21.0
gym-notices              0.0.8
gymnasium                0.27.1
gymnasium-notices        0.0.1
icecream                 2.1.3
idna                     3.4
imageio                  2.25.1
immutabledict            2.2.3
importlib-metadata       4.13.0
importlib-resources      5.12.0
jax-jumpy                0.2.0
joblib                   1.2.0
JSBSim                   1.1.6
jsonpickle               1.5.2
jsonschema               4.17.3
kiwisolver               1.4.4
llvmlite                 0.39.1
lz4                      4.3.2
Markdown                 3.4.1
MarkupSafe               2.1.2
matplotlib               3.5.3
mock                     5.0.1
mpyq                     0.2.5
msgpack                  1.0.5
mujoco-py                2.0.2.8
munch                    2.5.0
networkx                 2.6.3
numba                    0.56.4
numpy                    1.19.0
nvidia-cublas-cu11       11.10.3.66
nvidia-cuda-nvrtc-cu11   11.7.99
nvidia-cuda-runtime-cu11 11.7.99
nvidia-cudnn-cu11        8.5.0.96
oauthlib                 3.2.2
ordered-set              4.1.0
orjson                   3.8.6
packaging                23.0
pandapower               2.11.1
pandas                   1.1.0
pathtools                0.1.2
PettingZoo               1.22.2
Pillow                   9.4.0
pip                      22.3.1
pkgutil_resolve_name     1.3.10
portpicker               1.5.2
protobuf                 3.19.5
psutil                   5.9.4
py-cpuinfo               9.0.0
pyasn1                   0.4.8
pyasn1-modules           0.2.8
pycparser                2.21
pygame                   2.1.2
pyglet                   1.5.0
Pygments                 2.15.1
pymap3d                  2.9.1
pymongo                  4.3.3
pymunk                   6.2.1
pyparsing                3.0.9
pyrsistent               0.19.3
PySC2                    4.0.0
python-dateutil          2.8.2
pytz                     2023.3
PyWavelets               1.3.0
PyYAML                   5.4.1
ray                      1.8.0
redis                    4.5.4
requests                 2.28.2
requests-oauthlib        1.3.1
rsa                      4.9
s2clientprotocol         5.0.11.89720.0
s2protocol               5.0.11.89720.0
sacred                   0.8.2
SciencePlots             2.0.1
scikit-image             0.19.3
scikit-learn             0.20.0
scipy                    1.7.3
sentry-sdk               1.16.0
setproctitle             1.3.2
setuptools               65.6.3
six                      1.16.0
sk-video                 1.1.10
sklearn                  0.0.post4
SMAC                     1.0.0
SMACv2                   1.0.0
smmap                    5.0.0
stable-baselines3        1.7.0
SuperSuit                3.7.0
tabulate                 0.9.0
tensorboard              2.11.2
tensorboard-data-server  0.6.1
tensorboard-logger       0.1.0
tensorboard-plugin-wit   1.8.1
tensorboardX             2.5.1
threadpoolctl            3.1.0
tifffile                 2021.11.2
tinyscaler               1.2.5
torch                    1.13.1
tqdm                     4.65.0
typing_extensions        4.4.0
urllib3                  1.26.14
wandb                    0.13.10
websocket-client         1.5.1
Werkzeug                 2.2.2
wheel                    0.37.1
wrapt                    1.15.0
zipp                     3.12.0

[hsvgbkhgbv]

Hi Theo,

Please try installing the packages of following versions:

• pandapower==2.7.0
• pandas==1.1.3

Let me know whether it works.

[Theohhhu]

Works. Appreciate for your assistance.