Open Len02805 opened 1 month ago
Hi, I'm matpowercaseframes and matpower-pip developer. The easiest way to solve it is to run matpower-pip.
from matpower import start_instance
from matpowercaseframes import CaseFrames
m = start_instance()
CASE_NAME = "case16am.m"
cf_16am = CaseFrames(CASE_NAME)
Then, after the line 100, we should introduce something like
ppc = {key: mpc_frames.__getattribute__(key) if not isinstance(
mpc_frames.__getattribute__(key), pd.DataFrame) else mpc_frames.__getattribute__(
key).values for key in mpc_frames._attributes}
_adjust_ppc_indices(ppc)
See: https://github.com/UGM-EPSLab/matpowercaseframes/blob/master/notebooks/load_case16am.ipynb
Let's wait pandapower developer on the best way to integrate this to pandapower.
Hi Yasirroni,
I know the m = start_instance() should work as you mentioned in https://github.com/e2nIEE/pandapower/issues/1556.
However, as you also mentioned in the above link "Sadly, it need a third parties octave.cli." and actually the oct2py package is unstable as well. BTW, I had a bug with function start_instance(). But I do not have enough time and interest to investigate the reason as it doesn't affect my work.
error: graphics_toolkit: = toolkit is not available error: called from graphics_toolkit at line 84 column 5
Sadly, we fall in a loop between two fairly unstable dependency packages while using pandapower :(
Handling per name file like if mpc_file[-8:]=="case69.m":
is not wise and I think it should not be implemented.
In my case, oct2py
is pretty stable and reliable. If you got a problem in installing matpower-pip or oct2py, you should report the issue in their respective repo. You can mention me there with complete debugging or info if possible.
Yes, as I mentioned, "this is a customized method for this spcific .m file. It would be helpful if you guys could rewrite a more general method". I post this spcific method for insight the debug solution. Appreantly you cannot implement this into the pandapower public package.
pandapower
relies on matpowercaseframes
to read .m
file. Solution to read that file is already solved with matpowercaseframes
and matpower-pip
.
If you don't want to use matpowercaseframes
and matpower-pip
, run matlab
/octave
, iterate loop save as .mat
, then pandapower
will be able to use scipy.io
.
I think that is the best possible way here.
If you run the reproducible example above, you will get an error.
If you run the reproducible example above, you will get an error.
Which code? Which error?
from matpower import start_instance
from matpowercaseframes import CaseFrames
m = start_instance()
CASE_NAME = "case16am.m"
cf_16am = CaseFrames(CASE_NAME, load_case_engine=m)
import numpy as np
import pandas as pd
def _adjust_ppc_indices(ppc):
# adjust indices of ppc, since ppc must start at 0 rather than 1 (matlab)
ppc["bus"][:, 0] -= 1
ppc["branch"][:, 0] -= 1
ppc["branch"][:, 1] -= 1
# if in ppc is only one gen -> numpy initially uses one dim array -> change to two dim array
if len(ppc["gen"].shape) == 1:
ppc["gen"] = np.array(ppc["gen"], ndmin=2)
ppc["gen"][:, 0] -= 1
mpc_frames = cf_16am
ppc = {key: mpc_frames.__getattribute__(key) if not isinstance(
mpc_frames.__getattribute__(key), pd.DataFrame) else mpc_frames.__getattribute__(
key).values for key in mpc_frames._attributes}
_adjust_ppc_indices(ppc)
print(ppc)
{'version': '2',
'baseMVA': 10.0,
'bus': array([[ 0. , 3. , 0. , 0. , 0. , 0. , 1. , 1. , 0. ,
12.66, 1. , 1. , 1. ],
[ 1. , 1. , 0. , 0. , 0. , 0. , 1. , 1. , 0. ,
12.66, 1. , 1.1 , 0.9 ],
[ 2. , 1. , 2. , 1.6 , 0. , 0. , 1. , 1. , 0. ,
12.66, 1. , 1.1 , 0.9 ],
[ 3. , 1. , 3. , 0.4 , 0. , 0. , 1. , 1. , 0. ,
12.66, 1. , 1.1 , 0.9 ],
[ 4. , 1. , 2. , -0.4 , 0. , 0. , 1. , 1. , 0. ,
12.66, 1. , 1.1 , 0.9 ],
[ 5. , 1. , 1.5 , 1.2 , 0. , 0. , 1. , 1. , 0. ,
12.66, 1. , 1.1 , 0.9 ],
[ 6. , 1. , 4. , 2.7 , 0. , 0. , 1. , 1. , 0. ,
12.66, 1. , 1.1 , 0.9 ],
[ 7. , 1. , 5. , 1.8 , 0. , 0. , 1. , 1. , 0. ,
12.66, 1. , 1.1 , 0.9 ],
[ 8. , 1. , 1. , 0.9 , 0. , 0. , 1. , 1. , 0. ,
12.66, 1. , 1.1 , 0.9 ],
[ 9. , 1. , 0.6 , -0.5 , 0. , 0. , 1. , 1. , 0. ,
12.66, 1. , 1.1 , 0.9 ],
[10. , 1. , 4.5 , -1.7 , 0. , 0. , 1. , 1. , 0. ,
12.66, 1. , 1.1 , 0.9 ],
[11. , 1. , 1. , 0.9 , 0. , 0. , 1. , 1. , 0. ,
12.66, 1. , 1.1 , 0.9 ],
[12. , 1. , 1. , -1.1 , 0. , 0. , 1. , 1. , 0. ,
12.66, 1. , 1.1 , 0.9 ],
[13. , 1. , 1. , 0.9 , 0. , 0. , 1. , 1. , 0. ,
12.66, 1. , 1.1 , 0.9 ],
[14. , 1. , 2.1 , -0.8 , 0. , 0. , 1. , 1. , 0. ,
12.66, 1. , 1.1 , 0.9 ]]),
'gen': array([[ 0., 0., 0., 10., -10., 1., 100., 1., 10., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]]),
'branch': array([[ 0.00000000e+00, 1.00000000e+00, 0.00000000e+00,
6.23925289e-10, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.00000000e+00, -3.60000000e+02,
3.60000000e+02],
[ 1.00000000e+00, 2.00000000e+00, 7.49958197e-03,
1.00015224e-02, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.00000000e+00, -3.60000000e+02,
3.60000000e+02],
[ 2.00000000e+00, 3.00000000e+00, 7.99872220e-03,
1.09998028e-02, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.00000000e+00, -3.60000000e+02,
3.60000000e+02],
[ 2.00000000e+00, 4.00000000e+00, 8.99700266e-03,
1.80002446e-02, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.00000000e+00, -3.60000000e+02,
3.60000000e+02],
[ 4.00000000e+00, 5.00000000e+00, 3.99936110e-03,
3.99936110e-03, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.00000000e+00, -3.60000000e+02,
3.60000000e+02],
[ 1.00000000e+00, 6.00000000e+00, 1.09998028e-02,
1.09998028e-02, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.00000000e+00, -3.60000000e+02,
3.60000000e+02],
[ 6.00000000e+00, 7.00000000e+00, 7.99872220e-03,
1.09998028e-02, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.00000000e+00, -3.60000000e+02,
3.60000000e+02],
[ 6.00000000e+00, 8.00000000e+00, 1.09998028e-02,
1.09998028e-02, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.00000000e+00, -3.60000000e+02,
3.60000000e+02],
[ 7.00000000e+00, 9.00000000e+00, 1.09998028e-02,
1.09998028e-02, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.00000000e+00, -3.60000000e+02,
3.60000000e+02],
[ 7.00000000e+00, 1.00000000e+01, 7.99872220e-03,
1.09998028e-02, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.00000000e+00, -3.60000000e+02,
3.60000000e+02],
[ 1.00000000e+00, 1.10000000e+01, 1.09998028e-02,
1.09998028e-02, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.00000000e+00, -3.60000000e+02,
3.60000000e+02],
[ 1.10000000e+01, 1.20000000e+01, 8.99700266e-03,
1.19980833e-02, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.00000000e+00, -3.60000000e+02,
3.60000000e+02],
[ 1.10000000e+01, 1.30000000e+01, 7.99872220e-03,
1.09998028e-02, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.00000000e+00, -3.60000000e+02,
3.60000000e+02],
[ 1.30000000e+01, 1.40000000e+01, 3.99936110e-03,
3.99936110e-03, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.00000000e+00, -3.60000000e+02,
3.60000000e+02]]),
'gencost': array([[ 2., 0., 0., 3., 0., 20., 0.]])}
import pandapower as pp
net = pp.converter.from_mpc('/Your/path/to/Matpower/Cases/case69.m', f_hz=50, validate_conversion=False)
pp.runpp(net)
FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
net[table] = pd.concat([net[table], dd[dd.columns[~dd.isnull().all()]]], sort=False)
/opt/anaconda3/envs/Envs/lib/python3.9/site-packages/pandapower/converter/pypower/from_ppc.py:277: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '[]' has dtype incompatible with int64, please explicitly cast to a compatible dtype first.
branch_lookup.loc[~is_line, "element"] = idx_trafo
---------------------------------------------------------------------------
LoadflowNotConverged Traceback (most recent call last)
Cell In[2], line 3
1 network = pp.converter.from_mpc('case69.m', f_hz=50, validate_conversion=False)
2 network.line["max_i_ka"]=0.1
----> 3 pp.runpp(network)
File /opt/anaconda3/envs/Envs/lib/python3.9/site-packages/pandapower/run.py:250, in runpp(net, algorithm, calculate_voltage_angles, init, max_iteration, tolerance_mva, trafo_model, trafo_loading, enforce_q_lims, check_connectivity, voltage_depend_loads, consider_line_temperature, run_control, distributed_slack, tdpf, tdpf_delay_s, **kwargs)
248 _check_bus_index_and_print_warning_if_high(net)
249 _check_gen_index_and_print_warning_if_high(net)
--> 250 _powerflow(net, **kwargs)
File /opt/anaconda3/envs/Envs/lib/python3.9/site-packages/pandapower/powerflow.py:86, in _powerflow(net, **kwargs)
84 result = _run_pf_algorithm(ppci, net["_options"], **kwargs)
85 # read the results (=ppci with results) to net
---> 86 _ppci_to_net(result, net)
File /opt/anaconda3/envs/Envs/lib/python3.9/site-packages/pandapower/powerflow.py:190, in _ppci_to_net(result, net)
188 algorithm = net["_options"]["algorithm"]
189 max_iteration = net["_options"]["max_iteration"]
--> 190 raise LoadflowNotConverged("Power Flow {0} did not converge after "
191 "{1} iterations!".format(algorithm, max_iteration))
192 else:
193 net["_ppc"] = result
LoadflowNotConverged: Power Flow nr did not converge after 10 iterations!
matpowercaseframes==1.0.8 matpower==8.0.0.2.1.8 pandapower==2.14.11
Hmm, something must be wrong here.
from matpower import start_instance
from matpowercaseframes import CaseFrames
m = start_instance()
CASE_NAME = "case16am.m"
cf = CaseFrames(CASE_NAME, load_case_engine=m)
import numpy as np
import pandas as pd
def _adjust_ppc_indices(ppc):
# adjust indices of ppc, since ppc must start at 0 rather than 1 (matlab)
ppc["bus"][:, 0] -= 1
ppc["branch"][:, 0] -= 1
ppc["branch"][:, 1] -= 1
# if in ppc is only one gen -> numpy initially uses one dim array -> change to two dim array
if len(ppc["gen"].shape) == 1:
ppc["gen"] = np.array(ppc["gen"], ndmin=2)
ppc["gen"][:, 0] -= 1
mpc_frames = cf
ppc = {key: mpc_frames.__getattribute__(key) if not isinstance(
mpc_frames.__getattribute__(key), pd.DataFrame) else mpc_frames.__getattribute__(
key).values for key in mpc_frames._attributes}
_adjust_ppc_indices(ppc)
import pandapower as pp
import pandapower.converter as pc
net = pc.from_ppc(ppc)
pp.runpp(net)
LoadflowNotConverged: Power Flow nr did not converge after 10 iterations!
Yes. This is supposed to be that the following codes in the ending of .m file is not corrected processed by "from_mpc" function:
For your "case16am.m" case, the para should be:
%% convert branch impedances from Ohms to p.u.
[PQ, PV, REF, NONE, BUS_I, BUS_TYPE, PD, QD, GS, BS, BUS_AREA, VM, ...
VA, BASE_KV, ZONE, VMAX, VMIN, LAM_P, LAM_Q, MU_VMAX, MU_VMIN] = idx_bus;
[F_BUS, T_BUS, BR_R, BR_X, BR_B, RATE_A, RATE_B, RATE_C, ...
TAP, SHIFT, BR_STATUS, PF, QF, PT, QT, MU_SF, MU_ST, ...
ANGMIN, ANGMAX, MU_ANGMIN, MU_ANGMAX] = idx_brch;
Vbase = mpc.bus(1, BASE_KV) * 1e3; %% in Volts
Sbase = mpc.baseMVA * 1e6; %% in VA
mpc.branch(:, [BR_R BR_X]) = mpc.branch(:, [BR_R BR_X]) / (Vbase^2 / Sbase);
%% convert loads from kW to MW
mpc.bus(:, [PD, QD]) = mpc.bus(:, [PD, QD]) / 1e3;
Which code? Which error?
Now you find and reproduce the code and error :)
Now you find and reproduce the code and error :)
@Len02805 No, it seems you don't understand it yet. load_case_engine
already execute the MATLAB code of:
%% convert branch impedances from Ohms to p.u.
[PQ, PV, REF, NONE, BUS_I, BUS_TYPE, PD, QD, GS, BS, BUS_AREA, VM, ...
VA, BASE_KV, ZONE, VMAX, VMIN, LAM_P, LAM_Q, MU_VMAX, MU_VMIN] = idx_bus;
[F_BUS, T_BUS, BR_R, BR_X, BR_B, RATE_A, RATE_B, RATE_C, ...
TAP, SHIFT, BR_STATUS, PF, QF, PT, QT, MU_SF, MU_ST, ...
ANGMIN, ANGMAX, MU_ANGMIN, MU_ANGMAX] = idx_brch;
Vbase = mpc.bus(1, BASE_KV) * 1e3; %% in Volts
Sbase = mpc.baseMVA * 1e6; %% in VA
mpc.branch(:, [BR_R BR_X]) = mpc.branch(:, [BR_R BR_X]) / (Vbase^2 / Sbase);
%% convert loads from kW to MW
mpc.bus(:, [PD, QD]) = mpc.bus(:, [PD, QD]) / 1e3;
After checking, even MATPOWER itself did not converge. Using other file, that is case69.m
, my example is working.
from matpower import start_instance
from matpowercaseframes import CaseFrames
m = start_instance()
CASE_NAME = 'case69.m`
cf = CaseFrames(CASE_NAME, load_case_engine=m)
mpc_frames = cf
if CASE_NAME == 'case16am.m':
baseMVA69 = 10
Sbase69 = baseMVA69 * 1e6
Vbase69 = 12.66 * 1e3
mpc_frames.branch[["BR_R", "BR_X"]] = mpc_frames.branch[["BR_R", "BR_X"]] / (Vbase69**2 / Sbase69)
mpc_frames.bus[["PD", "QD"]] = mpc_frames.bus[["PD", "QD"]]/1000
import numpy as np
import pandas as pd
def _adjust_ppc_indices(ppc):
# adjust indices of ppc, since ppc must start at 0 rather than 1 (matlab)
ppc["bus"][:, 0] -= 1
ppc["branch"][:, 0] -= 1
ppc["branch"][:, 1] -= 1
# if in ppc is only one gen -> numpy initially uses one dim array -> change to two dim array
if len(ppc["gen"].shape) == 1:
ppc["gen"] = np.array(ppc["gen"], ndmin=2)
ppc["gen"][:, 0] -= 1
ppc = {key: mpc_frames.__getattribute__(key) if not isinstance(
mpc_frames.__getattribute__(key), pd.DataFrame) else mpc_frames.__getattribute__(
key).values for key in mpc_frames._attributes}
_adjust_ppc_indices(ppc)
import pandapower as pp
import pandapower.converter as pc
net = pc.from_ppc(ppc, f_hz=60, validate_conversion=False)
if "mpc_additional_data" in ppc:
if "_options" not in net:
net["_options"] = dict()
net._options.update(ppc["mpc_additional_data"])
try:
pp.runpp(net)
except Exception as e:
print(e)
If cf = CaseFrames(CASE_NAME, load_case_engine=m)
changed to cf = CaseFrames(CASE_NAME)
(currently used by pandapower
), the NR will not converge.
So, the correct use of matpowercaseframes
in this case is to use load_case_engine
.
As mentioned in https://github.com/MATPOWER/matpower/issues/208, case16am.m
is non convergent case. My bad changing your initial case69.m
to case16am.m
.
Yes, it seems you understand this bug in Pandapower on using matpowercaseframes (load_case_engine).
Bug report checklis
[X] Searched the issues page for similar reports
[X] Read the relevant sections of the documentation
[X] Browse the tutorials and tests for usefull code snippets and examples of use
[X] Reproduced the issue after updating with
pip install --upgrade pandapower
(orgit pull
)[X] Tried basic troubleshooting (if a bug/error) like restarting the interpreter and checking the pythonpath
Reproducible Example
Issue Description and Traceback
The original file case69.m from https://github.com/MATPOWER/matpower/blob/master/data/case69.m cannot be converted to the right pp network. Actually, a lof of .m file under this dir (github.com/MATPOWER/matpower/blob/master/data) cannot be successfully processed.
This is becase the following code para in the ending of .m file is not corrected processed by "from_mpc" function:
So for this .m file, if you add the follwoing code to /opt/anaconda3/envs/env_name/lib/python3.9/site-packages/pandapower/converter/matpower/from_mpc.py after the code line 100
Then the reproducible Example can be successfully run by pandapower as smoothly and accurately as the matlab :)
However, this is the customized method for this spcific .m file. It would be helpful if you guys could rewrite a more general method.
Expected Behavior
Fix the bug as I mentioned :)
Installed Versions
INSTALLED VERSIONS
commit : d9cdd2ee5a58015ef6f4d15c7226110c9aab8140 python : 3.9.15.final.0 python-bits : 64 OS : Darwin OS-release : 23.3.0 Version : Darwin Kernel Version 23.3.0: Thu Dec 21 02:29:41 PST 2023; root:xnu-10002.81.5~11/RELEASE_ARM64_T8122 machine : arm64 processor : arm byteorder : little LC_ALL : None LANG : None LOCALE : None.UTF-8
pandas : 2.2.2 numpy : 1.23.5 pytz : 2024.1 dateutil : 2.9.0.post0 setuptools : 73.0.0 pip : 24.0 Cython : None pytest : 8.3.2 hypothesis : None sphinx : 7.4.7 blosc : None feather : None xlsxwriter : 3.2.0 lxml.etree : None html5lib : None pymysql : None psycopg2 : 2.9.9 jinja2 : 3.1.4 IPython : 8.18.1 pandas_datareader : None adbc-driver-postgresql: None adbc-driver-sqlite : None bs4 : 4.12.3 bottleneck : 1.3.7 dataframe-api-compat : None fastparquet : 2024.5.0 fsspec : 2024.6.1 gcsfs : None matplotlib : 3.8.4 numba : 0.60.0 numexpr : 2.8.7 odfpy : None openpyxl : 3.1.5 pandas_gbq : None pyarrow : 17.0.0 pyreadstat : None python-calamine : None pyxlsb : None s3fs : None scipy : 1.13.1 sqlalchemy : None tables : None tabulate : 0.9.0 xarray : 2024.7.0 xlrd : 2.0.1 zstandard : None tzdata : 2023.3 qtpy : 2.4.1 pyqt5 : None
Label