Cost function definition

[lthurner]

I am trying to generate a valid powermodels data structure directly without going through a .m or .raw file for interfacing with pandapower (see e2nIEE/pandapower#207), but I am unsure about how to define the cost variable. The documentation says:

When present, the gencost data is incorporated into the gen data, the column names remain the same.

But not each column in the gencost matrix has a name, as the cost function can have different lengths. I figured out that for polynomial costs the cost variable is defined as a list of the coefficients, e.g.

cost = [c2, c1, c0]

Does that mean a piecewise linear cost function is defined as:

cost = [p0, f0, p1, f1]

?

Also: matpower supports reactive power costs, but from what I can see in the code, that is not supported yet by PowerModels, is that correct?

[ccoffrin]

You can see here how PowerModels interprets the Matpower cost model here,

https://github.com/lanl-ansi/PowerModels.jl/blob/master/src/io/matpower.jl#L317

I think the only truly required parameters are "model" and "cost"; but "ncost" might also be needed. At the moment, I am not aware of any code that is using "startup" or "shutdown".

Also, as a general tool to review the dictionary that PowerModels builds, you can use the following commands,

network_data = PowerModels.parse_file("case.m")
display(network_data) # raw dictionary
PowerModels.print_summary(network_data) # a table-like summary

At the moment, PowerModels does not considered the reactive cost model in Matpower; if you will use that feature, I will add it. Lets just open a specific Issue for that feature request.'

[lthurner]

Thanks! I was asking because I was getting an error when trying to define a linear function with

cost = [c1, c0]

I now noticed that the matpower parser converts the costs always to a quadratic function, so that a linear function would be modeled as:

cost = [0, c1, c0]

So I assume PowerModels always considers the objective function to be quadratic?

At the moment, PowerModels does not considered the reactive cost model in Matpower; if you will use that feature, I will add it. Lets just open a specific Issue for that feature request.'

Thanks a lot, but I don't currently need it myself, I am just trying to figure out what PowerModels can do so that I can properly set up the interface. So this maybe something to keep in mind, but not a priority right now.

[ccoffrin]

Very good point. In the case of the polynomial cost model (i.e. model 2), internally PowerModels assumes it is at least a quadratic function; so your implementation cost = [0.0, c1, c0] is what PowerModels expects for a linear function.

If you run into any disadvantages of this more general function let me know and I can look into adding an objective that is more suitable for linear cost functions.

[lthurner]

If you run into any disadvantages of this more general function let me know and I can look into adding an objective that is more suitable for linear cost functions.

Thanks I will, but everything seems to work fine for now!

I think the only truly required parameters are "model" and "cost"; but "ncost" might also be needed.

I get that ncost ist not necessary in PowerModels, as the length of the cost function is defined by the length of the cost parameter list. But shouldn't the ncost parameter be considered when parsing MATPOWER data? My understanding of the mpc data structure is that this

MODEL ... NCOST   COST
1              2            0          0        1        1        0       0
1              3           -1          1        0        0        1       1

Should be interpreted as:

cost = [0, 0, 1, 1]
cost = [-1, 1, 0, 0, 1,1]

But currently, it would get interpreted as:

cost = [0, 0, 1, 1, 0 ,0]
cost = [-1, 1, 0, 0, 1,1]

Because the iteration goes to the end of the matrix for all rows, regardless of the ncost parameter: https://github.com/lanl-ansi/PowerModels.jl/blob/c762511d654b758d04a967bddf3a716f5bbb910a/src/io/matpower.jl#L321

From my understanding, the ncost parameter gives the number of points in the PWL case, so it should be something like this if model==1:

 "cost" => [InfrastructureModels.check_type(Float64, x) for x in cost_row[5:5+2*cost_row[4]]] 

and it gives the number of coefficients for polynomial, so should be something like this for model==2:

"cost" => [InfrastructureModels.check_type(Float64, x) for x in cost_row[5:5+cost_row[4]]] 

(this is just of the top of my head untested)

Or is this distinction made somewhere later in the code?

[ccoffrin]

The example you mention is not one I had though of before. I had assumed becouse Matpower only support data a matrix, all of the ncost values would need to be the same in a given dataset. if you can confirm that Matpower does support the interoperation you discuss here, then we can add support for that in PowerModels' parser as well.

[lthurner]

I had assumed becouse Matpower only support data a matrix, all of the ncost values would need to be the same in a given dataset.

If the number of coefficients was always equal to the length of the row, then the NCOST parameter would be redundant, so I that is how I understand the documentation of NCOST as:

number of cost coefficients for polynomial cost function, or number of data points for piecewise linear

It can also be seen that NCOST is used to define the length of the polynomial in polycost.m.

If you agree, I could prepare a Pull Request for this. But I do have one basic question: how can run the tests locally for the develop version? I can run the tests through Pkg, but when I clone the repo and work on the develop version, PowerModels isn't registered with Pkg. I tried to run the tests by executing the test/runtests.jl file directly, but that just failed with a lot of path errors.

[ccoffrin]

A PR would be welcome! Just be warned I can be quite pick in the code review. :-)

The fix would require a small change in this function, https://github.com/lanl-ansi/PowerModels.jl/blob/master/src/io/matpower.jl#L315

Based on the semantics that you refrence, it's important to check wide side of the cost coefficient list is being truncated. For PWL it seems it could be arbitrary but for polynomial it should be the higher degree ones, correct?

[ccoffrin]

Concerning your question on testing. I presume you are using Julia 1? In that case, it is important make sure to checkout the package in "develop" mode. This checkouts out the full git repo for development in .julia/dev/PowerModels/

[lthurner]

A PR would be welcome! Just be warned I can be quite pick in the code review. :-)

Sounds like a chance for a Julia beginner like me to get some free advice on how to properly code in Julia ;)

Based on the semantics that you refrence, it's important to check wide side of the cost coefficient list is being truncated. For PWL it seems it could be arbitrary but for polynomial it should be the higher degree ones, correct?

I don't really understand what you mean by this, can you give an example? I would think it would be as straightforward as just going from to COST+NCOST for poly and for COST+NCOST*2 for PWL instead of to the end of the row to extract the cost function.

Concerning your question on testing. I presume you are using Julia 1? In that case, it is important make sure to checkout the package in "develop" mode. This checkouts out the full git repo for development in .julia/dev/PowerModels/

I am using 0.64. I still have to get acustomed to the Julia way of doing things, but I think i got it now, thanks.

[ccoffrin]

I don't really understand what you mean by this, can you give an example? I would think it would be as straightforward as just going from to COST+NCOST for poly and for COST+NCOST*2 for PWL instead of to the end of the row to extract the cost function.

Let me elaborate with an example, lets say you have the following gencost matrix rows,

1             2             1           2         3         4         5        6
2             4            11          12        13        14        15       16

Does this imply,

cost = [1,2,3,4]
cost = [11,12,13,14]

or

cost = [3,4,5,6]
cost = [13,14,15,16]

or

cost = [1,2,3,4]
cost = [13,14,15,16]

We just need to check the semantic that Matpower is using and match that.

[lthurner]

The matpower documentation says in the definition of the COST column that "the parameters begin in this column":

So according to this it should clearly be interpreted as

cost = [1,2,3,4] cost = [11,12,13,14]

That is also how I implemented it in #419.

[ccoffrin]

Based on the subscript numbering provided in this documentation, I would presume the Matpower implementation is actually this one,

cost = [1,2,3,4]
cost = [13,14,15,16]

Which I also think makes the most sense, mathematically.

That said, what makes sense to me is not really that important, matching whatever Matpower implements is the requirement for PowerModels.

To settle the issue, can you test the Matpower cases that you have updated inside of Matpower and make sure the AC-OPF solutions match those found by PowerModels? If you don't have easy access to Matpower I can run the test in a couple of days.

[lthurner]

Based on the subscript numbering provided in this documentation, I would presume the Matpower implementation is actually this one,

I don't really follow your logic here, I believe the subscribt indices just give the order in which the parameters are given, the statement that the parameters start at the COST column still stands regardless of ordering. But you are right that the best way to settle it is to test it with matpower.

To settle the issue, can you test the Matpower cases that you have updated inside of Matpower and make sure the AC-OPF solutions match those found by PowerModels? If you don't have easy access to Matpower I can run the test in a couple of days.

I would have to install MATLAB first since I don't use it these days, and I won't have the time in the next days, so I would appreciate it if you could look into it.

[ccoffrin]

Sure thing, glad to lend a hand. On travel at the moment, but should be able to check in about 48 hours.

[ccoffrin]

With #419 merged, shall we close this issue?

[lthurner]

Thanks a lot for veryfing!