Closed WinfriedL closed 6 years ago
I think the two reasons we didn't allow it was possible inconsistencies with the voltage setpoints as well as the ambiguity about the reactive power set points. As I remember it, we thought it would be cleaner not to allow those cases. Not sure I agree with this now, because we also make usage much harder to model such a case. If pypower already has a rule for reactive power distribution with the Q-range and we include a check for the voltage setpoint, I don't see why this shouldn't be allowed.
If pypower already has a rule for reactive power distribution with the Q-range
Yes, it's in https://github.com/e2nIEE/pandapower/blob/cedbf0f07fcf2d5684706523e723260a35e00632/pandapower/pf/pfsoln.py#L50-L83
The way it is implemented every generator reaches its q limit at the same time, which is important for
enforce_q_lims=True
But at the moment QMIN and QMAX are only set in the ppc if enforce_q_lims=True
, so pypower/pandapower distributes the reactive power differently depending on this parameter which is awkward for testing reactive power limits.
I will see if I can implement an easy check for the voltage setpoints in the code.
But at the moment QMIN and QMAX are only set in the ppc if enforce_q_lims=True, so pypower/pandapower distributes the reactive power differently depending on this parameter which is awkward for testing reactive power limits.
Yes I think the only reason that QMIN/QMAX are not set is because we assumed they are not needed. If they have an impact in the power flow also, they can also be set in the power flow.
I will see if I can implement an easy check for the voltage setpoints in the code.
What happens with generators that are merged by switches right now? Do both generators "survive"? In that case it should be relatively straightforward to include a check on the "gen" table after the switch merge to see if there are generators with the same bus but different voltage set points, right?
However if it is not easily possible to do this in a performant way we could also include it in the diagnostic function instead of directly in the power flow...
What happens with generators that are merged by switches right now? Do both generators "survive"? In that case it should be relatively straightforward to include a check on the "gen" table after the switch merge to see if there are generators with the same bus but different voltage set points, right?
Both survive and they have the same bus, so that would be a possible check.
My first try was to implement the check in https://github.com/e2nIEE/pandapower/blob/10af6826534c44f0ebe0f821230efbcddffc95d2/pandapower/build_bus.py#L38-L40 which works, but I don't know the performance impact of the additional check.
The parts where a check could/should be which I am aware of at the moment are the following:
The create function of generators and external grids
The bus fusing functions ds_union()
and create_bus_lookup()
After the bus fusing, for example in _build_pp_gen()
Maybe in _update_gen_ppc()
to prevent updating one of the generators with a different voltage setpoint
It might be best to do it at the latest possible point where most of the algorithms must go through. So maybe the best option would be to add it at the end of _pd2ppc(net)
and _update_ppc(net)
?
It could also be added in _get_pf_variables_from_ppci()
. I thinkt this is the part where different voltage setpoints would lead to problems, because the bus voltages are overwritten by different generators.
I think in _pd2ppc(net)
and _update_ppc(net)
after the ppc['gen'] table is ready might be the best idea? In that case the more performance critical parts have fewer checks, and I think every algorithm calls them?
It might be best to do it at the latest possible point where most of the algorithms must go through. So maybe the best option would be to add it at the end of _pd2ppc(net) and _update_ppc(net) ?
Yes this sounds like the best approach to me, because it is a nice catch-all solution that needs no differentiation between generators which at the same bus and generators that are connected through switches. The question is if this is even a general restriction that should be checked in pd2ppc or maybe a power flow restriction that might not apply in OPF or short-circuit? In that case the check could be pushed even further down the line into the power flow code?
The question is if this is even a general restriction that should be checked in pd2ppc or maybe a power flow restriction that might not apply in OPF or short-circuit?
If have little experience with OPF and even less experience with short-circuit, but I cannot imagine why the voltage setpoint of generators would not matter. And without the check the last generator at the bus would define the voltage setpoint for that bus and overrides all other generators at that bus. So the user does not know for sure which setpoint is used. btw Integral uses the mean value of the voltage setpoints in these cases. This of course possible and better defined (if documented...) but I assume this kind of implicit behaviour is not something you would prefer...
if there are generators with the same bus but different voltage set points, right?
I implemented such a test, but I am not sure if I found the best way for that check. My current approach is to first create an array of voltage setpoints where all generators at the same bus use the setpoint of the first generator at that bus and then compare it with the real voltage setpoint array to see if there are any difference:
# generator buses:
gen_bus=ppc['gen'][:,0].astype(int)
# generator setpoints:
gen_vm=ppc['gen'][:,5]
# buses with one or more generators and their index
unique_bus, index_bus =np.unique(gen_bus, return_index=True)
# voltage setpoint lookup with the voltage of the first occurence of that bus
first_gen_vm=-np.ones(gen_bus.max()+1)
first_gen_vm[unique_bus]=gen_vm[index_bus]
# generate voltage setpoints where all generators at the same bus have the voltage of the first generator at that bus
gen_vm_equal = first_gen_vm[gen_bus]
if not np.array_equal(gen_vm, gen_vm_equal):
raise UserWarning("Generators with different voltage setpoints connected to the same bus")
Am I missing some other more or less obvious way to implement that check?
However if it is not easily possible to do this in a performant way we could also include it in the diagnostic function instead of directly in the power flow...
I just looked at the performance of my check, it takes about 1 ms for my network with 8000 generators... I think that would be acceptable...
btw Integral uses the mean value of the voltage setpoints in these cases. This of course possible and better defined (if documented...) but I assume this kind of implicit behaviour is not something you would prefer...
Maybe a parameter that allows this as an explicit option could be implemented in the future, but I suggest we stick with the error for now.
If have little experience with OPF and even less experience with short-circuit, but I cannot imagine why the voltage setpoint of generators would not matter.
In the OPF the voltage set point is flexible, so that it is a result of the OPF and not an input parameter. If I am not mistaken, the initial set point is not used in the OPF except maxbe for initialization? In short-circuit, I don't think the set point is used as it is considered an operational state, and the IEC short-circuit analysis is basde on worst-case assumption rather than the operational set points.
Am I missing some other more or less obvious way to implement that check?
I can think of a couple of solutions that are easier to read, but I can't come up with an easier numpy-based solution, so this looks good.
I just looked at the performance of my check, it takes about 1 ms for my network with 8000 generators... I think that would be acceptable...
Yes seems to be very fast, using this code:
import numpy as np
def test_gen_setpoints(gen):
# generator buses:
gen_bus=gen[:,0].astype(int)
# generator setpoints:
gen_vm=gen[:,5]
# buses with one or more generators and their index
unique_bus, index_bus =np.unique(gen_bus, return_index=True)
# voltage setpoint lookup with the voltage of the first occurence of that bus
first_gen_vm=-np.ones(gen_bus.max()+1)
first_gen_vm[unique_bus]=gen_vm[index_bus]
# generate voltage setpoints where all generators at the same bus have the voltage of the first generator at that bus
gen_vm_equal = first_gen_vm[gen_bus]
return np.array_equal(gen_vm, gen_vm_equal)
n = 8*1000
gen = np.zeros(shape=(n, 21), dtype=float)
gen[:, 0] = np.array(list(range(n)))
gen[:, 5] = np.ones(n)
even gives me only 0.31ms:
%timeit test_gen(gen)
1000 loops, best of 3: 314 µs per loop
That should not be relevant in a grid with 8000 generators, so I think its okay to keep this in the power flow.
I have implemented that check. Changes I did:
I am unsure what to do with ext_grids and generators, though. From a power flow perspective there is no reason to not allow it. But there are some things to consider:
I have implemented that check. Changes I did:
Added a function to check the voltage set points Call that function in _pd2ppc and _update_ppc Removed the pv checks when fusing buses Alway set the qlimits in ppc Fixed a bug in _update_gen_ppc when setting the qlimits Marked the test for fusing two pv buses as xfail Added a similar test for generators with different vm_pu
First of all, thanks a lot, this has been mergend some time ago and seems to work very well.
I am unsure what to do with ext_grids and generators, though.
I have now looked into this and pushed a possible fix in #160 .
The distribution of reactive power to the generators and ext_grid is not ideal. If the generator has qlimits, most of the reactive power would end up at the ext_grid, since it's qlimits are huge. What from my perspective should happen is that up to its limit all reactive power should go to the generator and only the rest (if any) should go to the ext_grid.
I agree that is what should happen, because the ext_grid is a balancing element that provides all power that can not be provided by other elements in the grid. So reactive power should first come from gens if possible, and then from ext_grids.
Instead of adapting pfsoln, I have changed the convention that ext_grid generators gets huge reactive power limits (just in the power flow, OPF is the same as before). I have simply changed the reactive power limits to 0/0 for ext_grids, for gens its still -inf/inf. So if there is a gen and an ext_grid at the same bus, the reactive power is distributed completely to the gen. If there is no gen at the ext_grid bus, the ext_grid still gets all the Q because there is no other option. This is maybe a little implicit, but it works well without changing much code.
If enforce_q_lims=True, is there the possibility that the slack bus gets converted to pq? Have not looked yet, but should be easy to see in the code.
Yes this could happen until now. The basic problem is that PYPOWER always uses a bus-branch perspective, where there are only REF or PV buses. A reference generator is just defined as a generator at a REF bus, so there is no inherent way to differentiate between PV generators and REF generators if they are both connected to a REF type bus. I have used the fact that ext_grid generators now always have Q_MIN / Q_MAX set to zero as a way to identify them (which I admit is not very clean, but works for now). These generators are now removed from consideration for enforcing q_lims, and reference buses are never converted to PQ. This allows to enforce Q-lims for generators that are at the same bus as an ext_grid.
So if an ext_grid and a PV bus are at the same bus and q-limits are enforced, what happens is:
Iteration 1:
Iteration 2:
This results in the gen operating at the Q-limit and the ext_grid taking over the rest of the Q.
I have added a test to validate this, including comments so that it is clear what is tested here. @WinfriedL can you please have a look at this? If you agree with this implementation I will merge #160.
I have added another commit to allow multiple slacks at one bus. This almost already worked with the previous version, I only adapted pfsoln so that the slack active power is distributed evenly to all ext_grids at the bus. Before the entire power was assigned to the first generator. Now at least in my opinion all possible combinations are properly supported. A test for multiple ext_grids at one bus is also included in the new test function (see above).
The only caveat is that a gen with max_q_kvar = 0 and min_q_kvar = 0 will behave like an ext_grid with regard to reactive power. We should find a better solution to differentiate gens/ext_grids in the ppc gen table, but for now I think its an alright solution.
edit: if we allow multiple slacks at one bus, we should probably also check if they have the same voltage angle value (same magnitude is already checked)
I have added this check and also a comprehensive test that checks the different possibilities of ext_grids at the same bus here.
Looks great, thanks! I will test it sometime this week.
The only caveat is that a gen with max_q_kvar = 0 and min_q_kvar = 0 will behave like an ext_grid with regard to reactive power.
One could argue that a gen with these limits should have been a static generator in the first place...
One purely aesthetic comment: If the general way in which the reactive power is distributed between generators has not changed it could happen that two generators at a bus with zero reactive power demand both have q_kvar != zero but with different sign. E.g one generator with 100 Mvar reactive power and one with -100 Mvar, which looks odd.
In principle this could be changed so that both generators reach q_kvar=0 when the demand is zero (Integral does that...), but when I looked at that I decided it's not worth the effort because it has no influence whatsoever on the power flow. For the power flow the only important part is that both generators reach their limits at the same time. I just want to mention this in cases someone comes across this behaviour and assumes it to be an error.
One could argue that a gen with these limits should have been a static generator in the first place...
That is true, but it still didn't feel like a very nice solution, and while I am at it, I just as well might do it right. So I now stored the ref_gens in ppci['internal'] when building the ppc so that they can be accessed in _get_pf_variables_from_ppci inside the power flow, so that pv and ext_grid gens can be explicitely distinguished.
One purely aesthetic comment: If the general way in which the reactive power is distributed between generators has not changed it could happen that two generators at a bus with zero reactive power demand both have q_kvar != zero but with different sign. E.g one generator with 100 Mvar reactive power and one with -100 Mvar, which looks odd. In principle this could be changed so that both generators reach q_kvar=0 when the demand is zero (Integral does that...), but when I looked at that I decided it's not worth the effort because it has no influence whatsoever on the power flow. For the power flow the only important part is that both generators reach their limits at the same time. I just want to mention this in cases someone comes across this behaviour and assumes it to be an error.
Interesting. Can this only happen if the reactive bus power is zero? Or could it also happen that the bus power is 10MW, and one generator has -30MW and the other +40MW? I must admit I gave up trying to understand the part of pfsoln where reactive power is distributed, it is very convoluted...
Or could it also happen that the bus power is 10MW, and one generator has -30MW and the other +40MW?
Yes, this can happen as well. It depends on the q limits of the generators and is a result of the linear interpolation of those. So it is not random.
I must admit I gave up trying to understand the part of pfsoln where reactive power is distributed, it is very convoluted...
I know that feeling. I think I understood the code at some point, but in the end I just tried it out:
What pypower does at the moment is the linear interpolation between the q limits in var1. This ensures that both generators reach their limits at the same time. One could also interpolate between their limits and zero, which is var2 in the diagram.
But since there is nothing inherently wrong with var1 (as far as I know) and it's implementation is easier, I did not see the need to change that.
That is interesting, thanks for the exaplanation. I wrote a little test to reproduce this behaviour:
def test_reactive_power_distribution_with_two_gens_at_one_bus()
net = pp.create_empty_network()
b1 = pp.create_bus(net, vn_kv=110, index=3)
b2 = pp.create_bus(net, vn_kv=110, index=5)
pp.create_ext_grid(net, b1, vm_pu=1.01, index=2)
pp.create_line(net, b1, b2, 5., std_type="305-AL1/39-ST1A 110.0")
pp.create_gen(net, b2, vm_pu=1.0, p_kw=-1e3, min_q_kvar=-50e3, max_q_kvar=150e3)
pp.create_gen(net, b2, vm_pu=1.0, p_kw=-1e3, min_q_kvar=-100e3, max_q_kvar=100e3)
pp.runpp(net)
pp.create_load(net, b2, p_kw=0, q_kvar=net.res_gen.q_kvar.sum())
pp.runpp(net)
assert np.isclose(net.res_gen.q_kvar.sum(), 0)
assert np.allclose(net.res_gen.q_kvar.values, [0., 0.])
This test fails right now because the generator q_kvar values come out as [2500, -2500] instead of [0, 0]. I thought about adding this as an xfail to the test suite, but this implies we intend to fix that behaviour, which I don't think is the case right now. I will just leave this here in case we ever change our opinion.
This allows to enforce Q-lims for generators that are at the same bus as an ext_grid.
I am not sure if this can work as intended at the moment. I created a small test to try it out:
def test_reactive_power_distribution_with_gen_and_ext_grid_at_one_bus():
net = pp.create_empty_network()
b1 = pp.create_bus(net, vn_kv=110, index=3)
b2 = pp.create_bus(net, vn_kv=110, index=5)
pp.create_line(net, b1, b2, 5., std_type="305-AL1/39-ST1A 110.0")
net.line.c_nf_per_km=0
pp.create_gen(net, b1, vm_pu=1.01, p_kw=0, min_q_kvar=50, max_q_kvar=150)
pp.create_ext_grid(net, b1, vm_pu=1.01, index=2)
pp.runpp(net)
Since in the ppc creation the REF bus is first detected with the ext_grids, and afterwards the PV buses are detected with the gens, there is no reference bus in the system anymore. If one would allow gens and ext_grids at the same bus, the ext_grid should have preference over the gen for the bus type...?
Since in the ppc creation the REF bus is first detected with the ext_grids, and afterwards the PV buses are detected with the gens, there is no reference bus in the system anymore.
Are you sure you checked out the right branch? Because I fixed this by changing the order in which gens/ext_grids are added to avoid this: https://github.com/lthurner/pandapower/blob/006bcb0aa972503316a957070e151120ad83b7ff/pandapower/build_gen.py#L48-L51
The test that you wrote works for me without problems.
I did not merge the changes on develop yet, you will have to check out PR #160
Oops, sorry, no, I tried the develop branch. Changing the order would have been my proposed fix as well.
That case works, you are right, but there is an index error in _update_p when there is more than one bus with ext_grid and gen. The following case does not work for me:
def test_reactive_power_distribution_with_gen_and_ext_grid_at_two_buses():
net = pp.create_empty_network()
b1 = pp.create_bus(net, vn_kv=110)
b2 = pp.create_bus(net, vn_kv=110)
pp.create_gen(net, b1, vm_pu=1.01, p_kw=0, min_q_kvar=50, max_q_kvar=150)
pp.create_ext_grid(net, b1, vm_pu=1.01)
pp.create_gen(net, b2, vm_pu=1.01, p_kw=0, min_q_kvar=50, max_q_kvar=150)
pp.create_ext_grid(net, b2, vm_pu=1.01)
pp.runpp(net)
The array gens_at_bus has two entries, but pv_gens is 3 for the second bus since it is the index from the gen array. I think.
So gens_at_bus[pv_gens]
throws an error.
Yes you are right, I fixed the calculation of p_ext_grids, should work now
Looks good so far.
Now the check for dc lines can be removed as well: https://github.com/lthurner/pandapower/blob/b6958b5089f90e22465f23ba3d92346bb69cf6aa/pandapower/create.py#L2277-L2283
The following code works now, and I might be able to use the pandapower implementation of dc lines in the future:
def test_dc_with_ext_grid_at_one_bus():
net = pp.create_empty_network()
b1 = pp.create_bus(net, vn_kv=110)
b2 = pp.create_bus(net, vn_kv=110)
pp.create_ext_grid(net, b1, vm_pu=1.01)
pp.create_ext_grid(net, b2, vm_pu=1.01)
pp.create_dcline(net, from_bus=b1, to_bus=b2, p_kw=10,loss_percent=0,loss_kw=0, vm_from_pu=1.01, vm_to_pu=1.01)
pp.create_sgen(net,b1,p_kw=-10)
pp.create_load(net,b2,p_kw=10)
pp.runpp(net)
Thanks, I removed that check and included the test.
I might be able to use the pandapower implementation of dc lines in the future
That would be great to avoid duplicate implementations
I tried several different combinations so far, with different numbers of ext_grids, generators, different q limits, dclines... So far I have not found any further problems.
Only two minor things:
if q_min >q_max, the q distribution behaves a little bit strange. Since it is a user error and it affects only how the (correctly calculated) reactive power is distributed there is imho nothing to worry about. One could consider adding a check for it...
If there are no branches the q limits are not enforced. I am not sure if this is at any point relevant appart from special testcases. For example the dc test above would not work as expected with q limits enforced if there would be a q load at b2 which is larger than the limits of the dc line. I don't think that really matters.
Great, then I am merging the change into develop. The two points you mentioned are worth to keep in mind, but I agree they are not necessarily relevant.
I think there may be an index error somewhere when extracting the active power results of the generators. I was not yet able to find the cause, but I get mixed up active power results between external grids and generators. The internal ppc['gen'] results seems to be correct. I don't know it is related to this issue at all, but maybe the change in https://github.com/lthurner/pandapower/blob/006bcb0aa972503316a957070e151120ad83b7ff/pandapower/build_gen.py#L48-L51 has something to do with it?
I am still digging into this issue, but maybe you have an idea where to look for the problem...
If it is correct in the ppc but wrong in the net, that sounds like there is something wrong with the generator lookups (net._pd2ppc_lookups["ext_grid"], net._pd2ppc_lookups["gen"]). The lookups for gens that are later used to read the results are added here:
While gens are now added before ext_grids in the code, ext_grids are still added to the top of the gen table, so the order of generators in the gen table should be the same as before. So based on that I think the lookup should be unaffected, but of course there could be something I am missing...
So far I have confirmed that in my net at least the net._pd2ppc_lookups["ext_grid"] is wrong, but I have not found why it is wrong. I tried to reproduce the error in a smaller net but for now without success.
Still not sure, but I think there is a problem with the sorting in https://github.com/e2nIEE/pandapower/blob/602182d9c4629d220221f7324d5bd180ee5450f2/pandapower/pd2ppc.py#L198-L202 which is not well defined with multiple generators at one bus.
In a (relatively) smaller net I could solve the problem by passing kind='mergesort'
to argsort()
.
At first look this solved not all problems in the original net, however.
Update: I don't think the sorting is the real problem (it is somehow connected, though). While it is not well defined it is consistent during runtime.
And I was wrong that the ppc is correct and the lookup is wrong, I found a problem later on in the (dc) loadflow: In https://github.com/e2nIEE/pandapower/blob/602182d9c4629d220221f7324d5bd180ee5450f2/pandapower/pf/run_dc_pf.py#L47-L51
the reference generator (ext_grid) is determined by the generator connected to the ref bus. If there is a generator and an external grid connected to the same bus, it depends on the ordering of those generators. Since always the first one get's chosen it is related to the sorting.
But the fix is easier, since _get_pf_variables_from_ppci()
returns the refgens anyway, there is no need to look for them in this loop.
I will do some more testing and open a pull request if everything checks out. I don't know yet if there is a similar problem in the ac powerflow.
The pull request should fix the problem in the dc powerflow. The problem does not occur in the ac powerflow, since it should be correct in _update_p()
However, imho it is not completely satisfactory that the same functionality (assigning the active power to ext_grid generators) is solved completely different for ac and dc powerflow. While I have a somewhat working code for ac which is at least similar to my dc version (there is still a bug when enforcing q limits), I am unsure if and how I should continue that work, since it is a) important core functionality, b) very easy to break something and c) at the moment both implementations seem to work...
The pull request should fix the problem in the dc powerflow. The problem does not occur in the ac powerflow, since it should be correct in _update_p()
:+1:
However, imho it is not completely satisfactory that the same functionality (assigning the active power to ext_grid generators) is solved completely different for ac and dc powerflow.
Definitely not.
While I have a somewhat working code for ac which is at least similar to my dc version (there is still a bug when enforcing q limits), I am unsure if and how I should continue that work, since it is a) important core functionality, b) very easy to break something and c) at the moment both implementations seem to work...
This just means we need tests that guaruantee that no functionality is lost when refactoring. In that spirit, can you add a test for the bug fixed in #175, so a test that fails without the fix but passes now?
In that spirit, can you add a test for the bug fixed
I have added a test. It was the simplest net I came up with which shows this behaviour. It is not easy to trigger in small networks, since it depends on the sorting algorithm in pd2ppc, and in most cases the ext_grid ends up do be the first gen at each bus.
Apart from the failing tests (which I have not yet looked at yet), I am not sure where the test network has to be added and where the test should go. I think at the moment it only tests the ac powerflow (which worked already) and not the dc powerflow. Since there are different code paths for ac and dc, the test should be called twice. So if you can point me to the place(s) where the test should be, I will update the pull request.
I have added a test.
Great, thanks :+1:
Apart from the failing tests (which I have not yet looked at yet), I am not sure where the test network has to be added and where the test should go.
The classification of the loaflow tests is:
Imho this issue falls under the last category, so I would suggest to add it in test_scenarios.
Is there a reason why pandapower does not allow multiple generators at the same bus? It is supported by pypower and in fact pandapower seems to work just fine with them.
For my tests I just removed the checks in https://github.com/e2nIEE/pandapower/blob/10af6826534c44f0ebe0f821230efbcddffc95d2/pandapower/build_bus.py#L170-L174
and
https://github.com/e2nIEE/pandapower/blob/10af6826534c44f0ebe0f821230efbcddffc95d2/pandapower/build_bus.py#L39-L40
All results seem to be fine. In addition I changed https://github.com/e2nIEE/pandapower/blob/10af6826534c44f0ebe0f821230efbcddffc95d2/pandapower/build_gen.py#L301-L303 to always set the q limits, because pypower uses the q range to distribute the reactive power to the individual generators. Is there a reason to set QMAX and QMIN in the ppc only for specific cases? All tests run just fine if the limits are always set.
Of course if multiple generators would be allowed there should be a check if the voltage setpoints are identical.