AttributeError: 'DataFrame' object has no attribute 'max_i_ka'

[Alwinator]

When I run:

import grid2op
from l2rpn_baselines.PandapowerOPFAgent import evaluate

env = grid2op.make()
evaluate(env, logs_path='logs-eval-opf-agent')

I get the following exception:

Traceback (most recent call last):
  File "/home/alwin/PycharmProjects/l2rpn-challenge/l2rpn_baseline/evaluate_opf_agent.py", line 7, in <module>
    evaluate(env, logs_path='logs-eval-opf-agent')
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/l2rpn_baselines/PandapowerOPFAgent/evaluate.py", line 78, in evaluate
    pbar=False)
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/grid2op/Runner/Runner.py", line 940, in run
    res = self.run_sequential(nb_episode, path_save=path_save, pbar=pbar, seeds=seeds, max_iter=max_iter)
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/grid2op/Runner/Runner.py", line 767, in run_sequential
    max_iter=max_iter)
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/grid2op/Runner/Runner.py", line 556, in run_one_episode
    pbar=pbar, seed=seed, max_iter=max_iter)
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/grid2op/Runner/Runner.py", line 648, in _run_one_episode
    act = agent.act(obs, reward, done)
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/l2rpn_baselines/PandapowerOPFAgent/PandapowerOPFAgent.py", line 152, in act
    self.opf_type.lower(), logger=self.logger)
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/l2rpn_baselines/PandapowerOPFAgent/pp_functions.py", line 48, in run_opf
    trafo_loading_before = (grid.res_trafo.i_hv_ka / grid.trafo.max_i_ka * 100.).max()
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/pandas/core/generic.py", line 5274, in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'max_i_ka'

[jhmenke]

Could you send me the current grid.json you are using? Some older versions of the IEEE14 grid did not have "max_i_ka" properly set, but i thought it was fixed nowadays.

As a workaround, you can set it manually:

self.grid.line["max_i_ka"] = obs._obs_env._thermal_limit_a[:len(self.grid.line)] / 1000. 
self.grid.trafo["max_i_ka"] = obs._obs_env._thermal_limit_a[len(self.grid.line):] / 1000.

These lines go into the if self.grid is None: block of the function parse_observation_to_grid. Let me know if that helps.

[Alwinator]

Thanks for your answer. I am using l2rpn_baselines==0.4.2 installed via pip.

This is my grid.json file: grid.json

(I uploaded a .txt file, because GitHub don't like .json files)

I also tried the workaround, but unfortunately there is no if self.grid is None: block. Instead there is a if self.timestep == 0: block. I added your two lines there. It now looks like:

def parse_observation_to_grid(self, obs: grid2op.Observation):
        if self.timestep == 0: # line 62 (PandapowerOPFAgent.py)
            self.grid.line["max_i_ka"] = obs._obs_env._thermal_limit_a[:len(self.grid.line)] / 1000. # added line
            self.grid.trafo["max_i_ka"] = obs._obs_env._thermal_limit_a[len(self.grid.line):] / 1000. # added line
            self.grid.trafo["max_loading_percent"] = self.acceptable_loading_pct
            self.grid.line["max_loading_percent"] = self.acceptable_loading_pct
            self.grid.poly_cost.drop(self.grid.poly_cost.index, inplace=True)
            assert len(self.grid.ext_grid) == 1 and len(obs.gen_type) == len(self.grid.gen) + len(self.grid.ext_grid)
            self.grid.bus.min_vm_pu = 0.9
            self.grid.bus.max_vm_pu = 1.2
            # ...

Now it takes a lot longer, but after some time I get the following output/error:

OVERLOADING: line @ 98.7 % / trafo @ 0.4 %
Losses avoided: -0.380 MW (max. loading: 98.0% [-0.7%] / Trafo: 93.3% [-0.3%])
OPF wants to disconnect: pp_[] / connect: pp_[] 
OVERLOADING: line @ 100.1 % / trafo @ 0.4 %
Losses avoided: -0.773 MW (max. loading: 98.0% [-2.1%] / Trafo: 94.9% [-1.9%])
OPF wants to disconnect: pp_[] / connect: pp_[] 
OVERLOADING: line @ 100.3 % / trafo @ 0.4 %
Losses avoided: -0.881 MW (max. loading: 98.0% [-2.3%] / Trafo: 95.3% [-2.0%])
OPF wants to disconnect: pp_[] / connect: pp_[] 
OVERLOADING: line @ 99.5 % / trafo @ 0.4 %
Losses avoided: -0.688 MW (max. loading: 98.0% [-1.5%] / Trafo: 96.3% [-1.1%])
OPF wants to disconnect: pp_[] / connect: pp_[] 
OVERLOADING: line @ 100.1 % / trafo @ 0.4 %
Losses avoided: -0.908 MW (max. loading: 98.0% [-2.1%] / Trafo: 97.4% [-1.5%])
OPF wants to disconnect: pp_[] / connect: pp_[] 
OVERLOADING: line @ 99.4 % / trafo @ 0.4 %
OPF failed 
OVERLOADING: line @ 99.2 % / trafo @ 0.4 %
OPF failed 
OVERLOADING: line @ 98.3 % / trafo @ 0.4 %
Losses avoided: -0.474 MW (max. loading: 98.0% [-0.2%] / Trafo: 99.9% [1.5%])
OPF wants to disconnect: pp_[] / connect: pp_[] 
OVERLOADING: line @ 101.7 % / trafo @ 0.4 %
OPF failed 
OVERLOADING: line @ 104.3 % / trafo @ 0.4 %
OPF failed 
/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/pandapower/pf/ppci_variables.py:31: RuntimeWarning: invalid value encountered in true_divide
  V0[gbus] = gen[on, VG] / abs(V0[gbus]) * V0[gbus]
/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/scipy/sparse/linalg/dsolve/linsolve.py:206: MatrixRankWarning: Matrix is exactly singular
  warn("Matrix is exactly singular", MatrixRankWarning)
Traceback (most recent call last):
  File "/home/alwin/PycharmProjects/l2rpn-challenge/l2rpn_baseline/evaluate.py", line 42, in <module>
    main()
  File "/home/alwin/PycharmProjects/l2rpn-challenge/l2rpn_baseline/evaluate.py", line 38, in main
    evaluate(env, load_path=args.load_path, logs_path=f'logs-train-{args.agent}-{args.env}')
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/l2rpn_baselines/PandapowerOPFAgent/evaluate.py", line 78, in evaluate
    pbar=False)
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/grid2op/Runner/Runner.py", line 940, in run
    res = self.run_sequential(nb_episode, path_save=path_save, pbar=pbar, seeds=seeds, max_iter=max_iter)
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/grid2op/Runner/Runner.py", line 767, in run_sequential
    max_iter=max_iter)
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/grid2op/Runner/Runner.py", line 556, in run_one_episode
    pbar=pbar, seed=seed, max_iter=max_iter)
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/grid2op/Runner/Runner.py", line 648, in _run_one_episode
    act = agent.act(obs, reward, done)
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/l2rpn_baselines/PandapowerOPFAgent/PandapowerOPFAgent.py", line 122, in act
    self.parse_observation_to_grid(observation)
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/l2rpn_baselines/PandapowerOPFAgent/PandapowerOPFAgent.py", line 95, in parse_observation_to_grid
    pp.runpp(self.grid, init="results")
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/pandapower/run.py", line 225, in runpp
    _powerflow(net, **kwargs)
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/pandapower/powerflow.py", line 73, in _powerflow
    _ppci_to_net(result, net)
  File "/home/alwin/miniconda3/envs/l2rpn-challenge/lib/python3.7/site-packages/pandapower/powerflow.py", line 166, in _ppci_to_net
    "{1} iterations!".format(algorithm, max_iteration))
pandapower.powerflow.LoadflowNotConverged: Power Flow nr did not converge after 10 iterations!

Thanks you in advance!

[jhmenke]

Yes you placed it at the correct position. It now looks like the OPF is running for some time steps until it cannot handle the overloading anymore. I cannot debug for you unfortunately and never had the LoadflowNotConverged while developing the baseline, but you can check why the OPF does not want to disconnect/connect anything after it successfully lowered the line loading:

OPF wants to disconnect: pp[] / connect: pp[]

Here the lists should not be empty [] usually. However, i would strongly recommend to use the PowerModels OPF, since at some point the pypower OPF will inevitably fail due to its limited capacity.

You can also try/except the pandapower.powerflow.LoadflowNotConverged and print the generator values which might help you debug further.

[Alwinator]

Thank you for your help. If I set opf_type="powermodels" and installed Julia including the following packages:

import Pkg
Pkg.add("PyCall")
Pkg.add("PowerModels")
Pkg.add("JSON")
Pkg.add("Cbc")
Pkg.add("Ipopt")
Pkg.add("Juniper")
Pkg.add("JuMP")

Now it works flawlessly. Thank you. Could you maybe add this two lines directly in the repository that the problem is fixed for future users?

[jhmenke]

That's good to hear!

I will ask Benjamin of RTE directly if they could add the max_i_ka values in the reference grids, otherwise i will add it here yes.

[BDonnot]

Hello everyone :-)

As i said by mail to Jan Hendrick, this is rather complicated for us to check everything is correct or not. We have litterally hundreds of check to make sure the part of the datasets we generated are usable (including names, proper data types etc.).

We don't really use this attribute of pandapower so I cannot commit (i would love to :-( ) to fill this data.

What i can say is that you can initialize the OPFAgent by adding, in its constructor the thermal limit considered by the environment (calling env.get_thermal_limits()) for exameple. I think that would allow you to update that:

self.grid.line["max_i_ka"] = obs._obs_env._thermal_limit_a[:len(self.grid.line)] / 1000. 
self.grid.trafo["max_i_ka"] = obs._obs_env._thermal_limit_a[len(self.grid.line):] / 1000.

directly without relying on us (me in this case) to do something i have 95% chance to forget at some point :sweat:

[jhmenke]

Alright will push a fix for this tomorrow. Thanks for commenting :)