New feature: Make Qmin/Qmax methods

[miek770]

Hi Santiago,

I believe Qmin and Qmax for the ControlledGenerator class should be changed from attributes to methods. By default the methods could return the related attributes:

def q_min(self):
    return self.Qmin

def q_max(self):
    return self.Qmax

...but doing it this way (through methods) allows users to subclass ControlledGenerator and change this behavior without having to rewrite the whole __init__() method.

The reason why this is interesting is that the VAr limits are actually variable, and that the Qmin / Qmax we currently use are points on curves (Qmax is point B in this reference). For synchronous generators Qmin/max mostly depend on the active power, but for other types of generators it might be different. For example, the last wind turbine I worked with had its VAr limits vary as a function of both voltage and active power.

Would you be ok with that?

Thank you. - Michel

[SanPen]

Hi Michel,

As always, the issue is saving and long term "survival" of the code. Of course one might subclass ControlledGenerator, but then when saving none of that would persist.

Therefore, I'd go for the creation of new devices with more complex models that are fully parametrized by numbers. Then in the powerflow of wherever, those values that define the model would be replicated. The idea is to never use object orientation in the numerical methods because it is terribly slow, replacing those by DoD (data oriented development) i.e. a polynomial representing a curve of whatever other model that is defined by parameters. Not sure if I explained myself.

This is the only way to maintain long term coherence in the code, by separating the data management from the process.

[miek770]

Indeed saving becomes an issue in this case... that's a bummer. One way to do it would be to move from Excel to a more flexible save format (ex.: SQL database), and include the method description in the controlled generator data, but that feels wrong and insecure...

A synchronous generator device could easily be created with an all-numerical definition, and a standard method to return Qlimits based on the active power.

For the wind turbine, it's a bit more complicated but it could also be done. In my case I had data I needed to interpolate with a 4th-order polynomial interpolation, for each significant active power level (4 power level in my case, ex.: 0%, 90%, 95% and 100%). I'll think about how it could be done, but it doesn't look simple or straightforward if we want to cover the whole 0-100% range. At the moment, subclassing and forgoing the builtin save method seem simpler in these special cases.

I'm not sure I understand the issue about the numerical model, but I mostly explored the power flow analysis in GridCal. Qmin and Qmax should depend on existing conditions (current active power and voltage), but wouldn't the Qlimits only be queried in the already somewhat inefficient outer loop? Only the q_control methods would need to get the current Qlimits. Is it different for other analyses?

Thanks.

[miek770]

Now that I think about it, the fact that a generator is either static or controlled should be an attribute. We shouldn't have a different object and save entry for either, but a simple "control" attribute that would either be "voltage" or "VAr" (and maybe others).

[SanPen]

Haha!

This is the same thing as with the Branch element, fortunately I was smart enough to separate them.

Eventually, you'll want to do stuff with the controlled or the static generator that would make you wish you had them as separated entities. For instance, dynamic simulations, which I started but I've had very little time to complete.

In general, the devices should be as specialized as possible, using inheritance where it makes sense.

[miek770]

But it's actually the very same device in real life. you can change it from "voltage" to "VAr" (or "power factor") with a setting. It's quite different from the transformer vs cable (where Transformer probably should be a Branch subclass).

In one simulation you might have to run your generator with a fixed Q, and in the next with a voltage setpoint. Right now you'd have to have 2 devices (a static and a controlled generator) and manually alternate between either by setting the other to 0 VA. That's very unpractical.

[SanPen]

I truly recommend you reading, clean architecture and you'll see all the points that I make that seem to make no sense.

Software has two sides, it computes something faster than a human beign (this is the obvious, easy to achieve) and it can be scalable (this is the hard one, because is not obvious at all). Observing long lasting architectures in power systems (i.e. the CIM format, Power Factory, among others) you'll se device specialization everywhere as opposed to device aggregation. The reason for this is to make things easier in the future, regardless of the current "looks the same" situation.

A particular example, as I said is the dynamics. For a synchronous generator I would have a very large set of variables defining the dynamic behaviour of a n-degree machine, and on the other hand I'd have another set of variables defining an inverter (the "static machine"). Merging those two sets of variables in the same generator would be terrible from the maintenance side, and very confusing for the future user. Hence separating the generators is a good decision, even if it doesn't seem practical today.

On the non-excel comment: Excel and SQL are exactly the same: cells that store data. Plus having an SQL backend is a massive dependency that I don't want unless it is absolutely mandatory (like having a system with fail-over capabilities) If we would have more complex behaviours, we should think about taking the reactive power controller as a separated device, in a similar way as the transformer types.

[miek770]

Thanks for the reference, I've added it to my Christmas wishlist!

I think I see the confusion: I see the static_generator as a synchronous generator, operating with a constant power factor setpoint; and the controlled_generator as the same generator, operating under a constant voltage setpoint. I didn't consider either as an inverter. But then again, Battery inherits from the controlled_generator, not the static one, which confuses me. An inverters can also be voltage controlled, PF controlled, etc., so the same issue applies.

But the same synchronous generator can operate under different control methods (voltage regulation, power factor regulation, etc.) and still remain a synchronous generator with the dynamic variables. Unless I'm mistaken, I think you currently have to model it twice (once as a static generator, once as a controlled generator) to make this work in GridCal.

I agree about taking the generator controller out of ControlledGenerator and making it a separate device, it's what I was thinking. But it should be able to handle multiple control methods (ex.: voltage regulation and power factor regulation). It's still the same real-life device that handles mutliple modes.

For SQL, I'd use SQLite3 which is already builtin, and dataset which is simply awesome, but you're right changing format doesn't change the problem, and I shouldn't have brought it up (doesn't add anything to the discussion).

I unfortunately don't have access to the CIM documentation, but I like how PyPSA defined their devices (see Table II. Would the following make sense?

1. Rename StaticGenerator to Inverter;
2. Rename ControlledGenerator to Generator;
3. Create the GeneratorController class, with the control method as an attribute;
4. Make Qmin and Qmax methods which return static values, instead of attributes;
5. Create the SynchronousGenerator(Generator), with Qmin and Qmax methods that depend on the active power and other predefined, numerical attributes (that could be saved in a new Excel sheet);
6. Think about doing the same for a WindTurbineGenerator(Generator), with a similar implementation for Qmin and Qmax, but also condering the voltage.

[miek770]

Hi again,

I really hope I'm not being too annoying with this, I'm really only trying to reach 2 goals:

1. Generally help improve GridCal;
2. Help make GridCal able to do the simulations I need.

I'll try to take more time to better explain my issues to make it all easier, and I know you have a MUCH better vision of the whole software than I. Identifying that my initial suggestion would break the save process is a good example (I don't use the GUI and the save files, so I tend to not think about those). I also replied a bit too quickly because of other stuff today.

About the issue, I'd like to take a step back. My basic need can be summarized so:

a) I need to be able to model synchronous generators that can alternate between control mode (constant voltage, or constant Q). I would rather not have to model the same physical component twice for software reasons; b) In either mode, I need to be able to define Q limits that depend on the current simulation results (voltage and active power).

In order to reach a), I think the new Controller class you suggested makes sense, although it's not strickly required. A simple attribute could do the trick (ControlledGenerator.control = "voltage", for example). It would remain the same synchronous generator in both modes, with the same dynamic attributes, but it would not be simulated the same way.

In order to reach b), I think the best way to do it without breaking anything would be to make Qmin and Qmax methods that evaluate a custom string. I did a simple example below. We can limit the scope of the eval() statement to the desired variables (context and self in this example). It doesn't have access to any imported library, but we could allow certain minimal mathematical functions (ex.: max(), sum(), etc.).

That command string could be saved like any other attribute. No custom class, no custom method, just a custom string to be evaluated. It also wouldn't be a safety issue with strict limitations (as below).

class Device:
    def __init__(self, name, attr, cmd="self.attr_"):
        self.name = name
        self.attr_ = attr
        self.cmd = cmd

    def attr(self, context=None):
        return eval(self.cmd, {}, {"x": context, "self": self})

d1 = Device(name="G1", attr=2)

cmd = "x['Vm'] * self.attr_"
d2 = Device(name="G2", attr=3, cmd=cmd)

x = {"Vm": 1.05}
for d in (d1, d2):
    print(f"{d.name}: attr = {d.attr(x)}")

About the speed issue, wouldn't only the outer control loop have to get the Q limits when running in "constant voltage", at least for the power flow?

We'd probably need another control loop to adapt the generators in "constant Q" that get beyond their new Q limit if it depends on the voltage, but that'd still be in the outer (inefficient) loop.

Thanks.

[SanPen]

I think, that having the generator as you propose is a good thing. But since it is going to produce a lot of changes, I'll create a new branch for it.

I think this is also an opportunity for you to dive deeper on the design.

As a general remark, I made GridCal in order to have the faster power flow tool available, to enable stochastic calculations. The first is a technological enabler for the second.

Another general remark is to never use strings for configuration. I had them for plotting results and they turn out to be not a good idea from the maintainability point of view. Instead use Enums.

Also, as I am not that familiar with all the control modes, let's gather requirements to analyze the best possible design.

[miek770]

I think, that having the generator as you propose is a good thing. But since it is going to produce a lot of changes, I'll create a new branch for it.

Great, thank you.

I think this is also an opportunity for you to dive deeper on the design.

Indeed!

As a general remark, I made GridCal in order to have the faster power flow tool available, to enable stochastic calculations. The first is a technological enabler for the second.

This is definitely a big plus. As I mentioned somewhere, I've started playing with genetic algorithms and GridCal a few weeks ago and speed definitely helps.

For new features, my general philosophy would be: if it's useful but it has a significant speed impact, let's make it optional until we find a more efficient implementation (or forever).

In the long run, I would like to help improve the accuracy of components (transformers, generators, etc.) in open source power systems software. I see a lot of talented researchers working on new exciting algorithms and optimizations, but I don't get the impression that many of us actually design and build power plants and substations on a regular basis. Some vital aspects (such as the current issue) are naturally overlooked.

So if it's ok with you, I have a few other things I'd like to work on in GridCal once this and the other issues are resolved (new features for temperature correction and transformer impedance precision). For example, changing a transformer's tap has a effect on impedance. We should be able to model cooling stages. Etc etc.

I also want to improve test coverage (with pytest) in the long run.

Another general remark is to never use strings for configuration. I had them for plotting results and they turn out to be not a good idea from the maintainability point of view. Instead use Enums.

I'm not sure what you mean by configuration, do you mean the cmd attribute in my example? I was thinking of having the possibility of using default options from an Enum but leaving the possibility of using custom strings as well. The default option (from the Enum) would be exactly as it currently is (i.e.: static Qmin and Qmax). Another option in the Enum could use a standard formula from a recognized publication with calculated values based on new optional attributes of the Generator class (ex.: A practical method for calculation of over-excited region in the synchronous generator capability curves or one of the standard methods they refer to).

Also, as I am not that familiar with all the control modes, let's gather requirements to analyze the best possible design.

I am, so I'll try to provide a bit more background whenever I propose a change that's related to generator controls.

[SanPen]

Great.

I fully agree on making the models better, but I am concerned with how to do it and the effects. For instance, if you've ever used PSS/e, it allows you to use any control you want if you can provide it as Fortran library. I've thought about how to do it in GridCal without speed loss and it is very hard. The reason is that compiled languages (C++/Fortran mainly) do not have a significant overhead by using object oriented programming over arrays, but python does have a lot of overhead.

An example is the handling of the dynamic response, from the top of my head it is something like:

• 1 Run a power flow
• 2 Modify Ybus.
• 3 Compute the Ibus vector by "asking" every machine model their current response (solve a differential equation)
• 4 get the voltage
• go to 3

The step 3 is very slow, since within the numerical algorithm, we have to cycle through the generator objects, asking to the different controls how do they response. The alternative is a very complicated standardisation of the models into arrays, which I tried and it is super fast, but not flexible to model changes (hard to maintain).

This is exactly my point with the custom Q limits as functions; It forces us to have objects in a numerical process, becoming a bottle neck. I am afraid that there is a trade-of between model accuracy and speed in python. Finding a design or technology that solves or alleviates this is key.

About the documentation, do you have a preferred way of proceeding? I started a "book" like the EMTP theory book for the models in GridCal (here)

[SanPen]

I close this

[miek770]

I understand, I'll reopen it when I manage to work on it, but I haven't started yet.