search

ames-market / psst

PSST - Power System Simulation Toolbox
Other
39 stars 27 forks source link

Matpower cases do not conform to pyomo constraints #6

Open djinnome opened 4 years ago

djinnome commented 4 years ago

Hello folks,

I am interested in running security-constrained unit commitment for the following matpower cases:

I can load the cases just fine using read_matpower() (except for case3012wp and case3375wp which have Inf and -Inf for some entries of Qmax and Qmin), and I can create_network() for each case, but when I try to build_model() some of the generators violate the model.MinimumProductionCost within=NonNegativeReals constraint. For example, case1888rte violates this constraint because GenCo32 has a PMAX=0 (which causes a warning) and PMIN=-8 which causes a constraint violation.
I tried manually setting PMIN=0 and also tried removing the within=NonNegativeReals constraint, but in both cases, I ended up getting this error instead:

/Users/zuck016/Projects/Proposals/DataModelConvergence_2019/Matpower_instances/psst/psst/model/__init__.py:81: UserWarning: Generators with zero PMAX found: ['GenCo32']
  warnings.warn("Generators with zero PMAX found: {}".format(zero_generation))
ERROR:pyomo.core:Constructing component 'ComputeProductionCosts' from data=None failed:
ValueError: 'ComputeProductionCosts[GenCo32,0]' does not have a list of domain points that is non-decreasing
ERROR: Constructing component 'ComputeProductionCosts' from data=None failed:
    ValueError: 'ComputeProductionCosts[GenCo32,0]' does not have a list of
    domain points that is non-decreasing

Please advise.

Sincerely,

Jeremy

kdheepak commented 4 years ago

The data for the costs for the generator is here:

https://github.com/MATPOWER/matpower/blob/ee27a86a9127538c3013366e3044674c1fe58645/data/case1888rte.m#L4780

Since it is of cost model 2, this is what is used to create the cost curve:

https://github.com/power-system-simulation-toolbox/psst/blob/00998b418dc1bcac051c2bbd91120ab9a7691433/psst/model/__init__.py#L212-L213

This requires PMIN and PMAX to be different, and PMIN to be strictly less than PMAX for every generator. Also, if I remember correctly, PMIN cannot be negative in this model.

djinnome commented 4 years ago

Hi @kdheepak

Thanks for responding so quickly. I agree with everything you said, and yet I observe that case1888rte is published with a negative PMIN as evidenced on line 1971 of the file you linked.

All the other cases I reference above also have at least one generator with a negative PMIN. for example, case1951rte has GenCo35 with PMIN=-8

case2848rte has GenCo191 GenCo192 and GenCo314 with a negative PMIN.

Is this a problem with the MATPOWER instances or is it a problem with PSST?

This is the minimum code necessary to reproduce the problem:

!git clone https://github.com/MATPOWER/matpower.git
import psst
from psst.case import read_matpower
from psst.model import build_model
case = read_matpower('matpower/data/case1888rte.m')
print(case.gen.loc['GenCo32', 'PMIN'])
model = build_model(case)

This results in the following output:

-8.0
/Users/zuck016/Projects/Proposals/DataModelConvergence_2019/Matpower_instances/psst/psst/model/__init__.py:81: UserWarning: Generators with zero PMAX found: ['GenCo32']
  warnings.warn("Generators with zero PMAX found: {}".format(zero_generation))
ERROR:pyomo.core:evaluating object as numeric value: MinimumProductionCost[GenCo32,0]
    (object: <class 'pyomo.core.base.param._ParamData'>)
Error evaluating Param value (MinimumProductionCost[GenCo32,0]):
    The Param value is currently set to an invalid value.  This is
    typically from a scalar Param or mutable Indexed Param without
    an initial or default value.
ERROR: evaluating object as numeric value: MinimumProductionCost[GenCo32,0]
        (object: <class 'pyomo.core.base.param._ParamData'>)
    Error evaluating Param value (MinimumProductionCost[GenCo32,0]):
        The Param value is currently set to an invalid value.  This is typically
        from a scalar Param or mutable Indexed Param without an initial or
        default value.
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-6081adf609b4> in <module>
      6 case = read_matpower('matpower/data/case1888rte.m')
      7 print(case.gen.loc['GenCo32', 'PMIN'])
----> 8 model = build_model(case)

~/Projects/Proposals/DataModelConvergence_2019/Matpower_instances/psst/psst/model/__init__.py in build_model(case, generator_df, load_df, branch_df, bus_df, previous_unit_commitment_df, timeseries_pmax, timeseries_pmin, base_MVA, base_KV, config)
    249 
    250     minimum_production_cost(model)
--> 251     production_cost(model)
    252 
    253     # setup start up and shut down costs for generators

~/Projects/Proposals/DataModelConvergence_2019/Matpower_instances/psst/psst/model/generators.py in production_cost(model)
    214     for g in model.Generators:
    215         for t in model.TimePeriods:
--> 216             power_generation_piecewise_points_rule(model, g, t)
    217 
    218 

~/Projects/Proposals/DataModelConvergence_2019/Matpower_instances/psst/psst/model/generators.py in power_generation_piecewise_points_rule(m, g, t)
    220     #print("Minimum production cost for Model {} Generator {} Time {}".format( m, g, t))
    221     #print(value(m.MinimumProductionCost[g, t]))
--> 222     minimum_production_cost = value(m.MinimumProductionCost[g, t])
    223     if len(m.CostPiecewisePoints[g]) > 0:
    224         m.PowerGenerationPiecewisePoints[g,t] = list(m.CostPiecewisePoints[g])

~/.pyenv/versions/miniconda3-latest/envs/idp_new/lib/python3.6/site-packages/pyomo/core/expr/numvalue.py in value(obj, exception)
    224 
    225         try:
--> 226             tmp = obj(exception=True)
    227             if tmp is None:
    228                 raise ValueError(

~/.pyenv/versions/miniconda3-latest/envs/idp_new/lib/python3.6/site-packages/pyomo/core/base/param.py in __call__(self, exception)
    116                     "from a scalar Param or mutable Indexed Param without\n"
    117                     "\tan initial or default value."
--> 118                     % ( self.name, ))
    119             else:
    120                 return None

ValueError: Error evaluating Param value (MinimumProductionCost[GenCo32,0]):
    The Param value is currently set to an invalid value.  This is
    typically from a scalar Param or mutable Indexed Param without
    an initial or default value.
kdheepak commented 4 years ago

Thanks for the detailed report.

Can you try changing this line to remove the NonNegativeReals within initialization?

https://github.com/power-system-simulation-toolbox/psst/blob/00998b418dc1bcac051c2bbd91120ab9a7691433/psst/model/generators.py#L51

I problem is that I didn't expect PMIN to be negative, but I think changing this line would work. I would first try it on a smaller test system, for example the 5 bus system, and make sure that psst is performing as you'd expect it.

I'll be able to try it as well later this week.

djinnome commented 4 years ago

Hi @kdheepak

OK, I have reproduced the error on a small matpower instance, and even after implementing all the changes you suggested and that I implemented in https://github.com/power-system-simulation-toolbox/psst/pull/7 I am still getting an error message that I am not sure how to fix.

%matplotlib inline
import pandas as pd
import psst,os
from psst.case import read_matpower
from psst.model import build_model
from psst.network import create_network
casedir = 'matpower/data'
case5 = read_matpower(os.path.join(casedir,'case5.m'))
generator_df = pd.merge(case5.gen, case5.gencost, left_index=True, right_index=True)
network5 = create_network( case5, prog='neato')
network5.draw()

case5

Display pmin and pmax for case 5

generator_df[['PMIN','PMAX']]
PMIN PMAX
GenCo0 0 40
GenCo1 0 170
GenCo2 0 520
GenCo3 0 200
GenCo4 0 600

Building a model works just fine

model5 = build_model(case5, generator_df)

Changed model.MinimumPowerOutput to use within=Reals

Set pmin=-10 for GenCo0

generator_df.loc['GenCo0','PMIN'] = -10
generator_df
PMIN PMAX
GenCo0 -10 40
GenCo1 0 170
GenCo2 0 520
GenCo3 0 200
GenCo4 0 600

Building a model results in error

model5pmin = build_model(case5, generator_df)
ERROR:pyomo.core:Constructing component 'ComputeProductionCosts' from data=None failed:
ValueError: 'ComputeProductionCosts[GenCo0,0]' does not have a list of domain points that is non-decreasing
Changed model.MinimumPowerOutput to use within=Reals
ERROR: Constructing component 'ComputeProductionCosts' from data=None failed:
    ValueError: 'ComputeProductionCosts[GenCo0,0]' does not have a list of
    domain points that is non-decreasing
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-9-2f6098ed98df> in <module>
----> 1 model5pmin = build_model(case5, generator_df)

~/Projects/Proposals/DataModelConvergence_2019/PSST/psst/psst/model/__init__.py in build_model(case, generator_df, load_df, branch_df, bus_df, previous_unit_commitment_df, timeseries_pmax, timeseries_pmin, base_MVA, base_KV, config)
    296     constraint_generator_power(model)
    297     constraint_up_down_time(model)
--> 298     constraint_for_cost(model)
    299 
    300     # Add objective function

~/Projects/Proposals/DataModelConvergence_2019/PSST/psst/psst/model/constraints.py in constraint_for_cost(model)
    384 def constraint_for_cost(model):
    385 
--> 386     model.ComputeProductionCosts = Piecewise(model.Generators * model.TimePeriods, model.ProductionCost, model.PowerGenerated, pw_pts=model.PowerGenerationPiecewisePoints, f_rule=production_cost_function, pw_constr_type='LB', warning_tol=1e-20)
    387 
    388     model.ComputeHotStart = Constraint(model.Generators, model.TimePeriods, rule=compute_hot_start_rule)

~/.pyenv/versions/miniconda3-latest/envs/idp_new/lib/python3.6/site-packages/pyomo/core/base/block.py in __setattr__(self, name, val)
    566                 # Pyomo components are added with the add_component method.
    567                 #
--> 568                 self.add_component(name, val)
    569             else:
    570                 #

~/.pyenv/versions/miniconda3-latest/envs/idp_new/lib/python3.6/site-packages/pyomo/core/base/block.py in add_component(self, name, val)
   1006                              _blockName, str(data))
   1007             try:
-> 1008                 val.construct(data)
   1009             except:
   1010                 err = sys.exc_info()[1]

~/.pyenv/versions/miniconda3-latest/envs/idp_new/lib/python3.6/site-packages/pyomo/core/base/piecewise.py in construct(self, *args, **kwds)
   1193                 if generate_debug_messages:
   1194                     logger.debug("  Constructing Piecewise index "+str(index))
-> 1195                 self.add(index, _is_indexed=is_indexed)
   1196         timer.report()
   1197 

~/.pyenv/versions/miniconda3-latest/envs/idp_new/lib/python3.6/site-packages/pyomo/core/base/piecewise.py in add(self, index, _is_indexed)
   1366         self._data[index] = comp
   1367         comp.updateBoundType(self._bound_type)
-> 1368         comp.updatePoints(_self_domain_pts_index,range_pts)
   1369         comp.build_constraints(func,_self_xvar,_self_yvar)
   1370 

~/.pyenv/versions/miniconda3-latest/envs/idp_new/lib/python3.6/site-packages/pyomo/core/base/piecewise.py in updatePoints(self, domain_pts, range_pts)
    220             msg = "'%s' does not have a list of domain points "\
    221                   "that is non-decreasing"
--> 222             raise ValueError(msg % (self.name,))
    223         self._domain_pts = domain_pts
    224         self._range_pts = range_pts

ValueError: ComputeProductionCosts[GenCo0,0] does not have a list of domain points that is non-decreasing
kdheepak commented 4 years ago

Thanks for the detailed report. I'll take a closer look at this over the weekend.

djinnome commented 4 years ago

Hey @kdheepak

Any progress on any solutions or workarounds for the problem of negative PMIN?

Sincerely,

Jeremy

kdheepak commented 4 years ago

Sorry I've been swamped with a few things, I'll look at it today.

djinnome commented 4 years ago

Thanks so much!

Sincerley,

Jeremy

From: Dheepak Krishnamurthy notifications@github.com Reply-To: power-system-simulation-toolbox/psst reply@reply.github.com Date: Friday, November 8, 2019 at 10:59 AM To: power-system-simulation-toolbox/psst psst@noreply.github.com Cc: "Zucker, Jeremy D" jeremy.zucker@pnnl.gov, Author author@noreply.github.com Subject: Re: [power-system-simulation-toolbox/psst] Matpower cases do not conform to pyomo constraints (#6)

Sorry I've been swamped with a few things, I'll look at it today.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://protect2.fireeye.com/v1/url?k=a7b0edfd-fb05d244-a7b0c7e8-0cc47adc5fce-1035f9ad85bbaded&q=1&e=8ac70b62-d18a-4e27-a0aa-2a807c12e791&u=https%3A%2F%2Fgithub.com%2Fpower-system-simulation-toolbox%2Fpsst%2Fissues%2F6%3Femail_source%3Dnotifications%26email_token%3DAABDGUUZL7HLTOSZCHWHI6DQSWZINA5CNFSM4I4P3J6KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDTBKLY%23issuecomment-551949615, or unsubscribehttps://protect2.fireeye.com/v1/url?k=1db361af-41065e16-1db34bba-0cc47adc5fce-1da3d1e818d045aa&q=1&e=8ac70b62-d18a-4e27-a0aa-2a807c12e791&u=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAABDGUTADCZNJY67LLEHKCDQSWZINANCNFSM4I4P3J6A.

djinnome commented 4 years ago

Hi @kdheepak

Any luck with relaxing the pyomo constraints to allow a negative PMIN? I was hoping I could work around the problem by just setting PMIN=max(0,PMIN), but then there are some problems where PMAX=0. I then tried finessing the problem by subtracting negative PMIN from PMAX and the load, but then that would imply units that are active when P == PMIN would be inactive when P == 0

I am also looking into pyomo to see if I can understand the ValueError exception, but I wanted to check in with you first to see where you are at.

kdheepak commented 4 years ago

Sorry, I keep promising to look at it but something keeps coming up. I'll definitely be able to look at it this week. I'll post here with my comments.

djinnome commented 4 years ago

Thanks @kdheepak I appreciate it.

djinnome commented 4 years ago

Hi @kdheepak I'm going to take a look at the pyomo error that keeps coming up when PMIN<0 and see if I can debug it this week. If you are around let me know, and perhaps we can try to tackle it together.

Sincerely,

Jeremy

djinnome commented 4 years ago

Hi Dheepak,

Thanks for meeting with me this morning about the negative PMIN problem. While you are working on a fix, I started to implement the workaround you suggested for removing generators with PMIN < 0, and subtracting PG from the load, but I noticed inconsistencies in some Matpower cases that I wanted to check in with you about.

There are some generators with PG < 0 but have a PMIN > 0.
For example, with case1888rte

GEN_BUS PG QG QMAX QMIN VG MBASE GEN_STATUS PMAX PMIN PC1 PC2 QC1MIN QC1MAX QC2MIN QC2MAX RAMP_AGC RAMP_10 RAMP_30 RAMP_Q APF STARTUP_RAMP SHUTDOWN_RAMP MINIMUM_UP_TIME MINIMUM_DOWN_TIME MODEL STARTUP SHUTDOWN NCOST COST_2 COST_1 COST_0
GenCo26 Bus876 -1.22 0.0 0.0 0.0 1.04 100 1 54.0 10.8 0 0 0 0 0 0 0 54.0 0 0 0 54.0 54.0 0 0 2 0 0 2 0 1 0

case1951rte

GEN_BUS PG QG QMAX QMIN VG MBASE GEN_STATUS PMAX PMIN PC1 PC2 QC1MIN QC1MAX QC2MIN QC2MAX RAMP_AGC RAMP_10 RAMP_30 RAMP_Q APF STARTUP_RAMP SHUTDOWN_RAMP MINIMUM_UP_TIME MINIMUM_DOWN_TIME MODEL STARTUP SHUTDOWN NCOST COST_2 COST_1 COST_0
GenCo26 Bus885 -0.50 0.0 0.0 0.0 1.079 100 1 8.0 1.6 0 0 0 0 0 0 0 8.0 0 0 0 8.0 8.0 0 0 2 0 0 2 0 1 0
GenCo27 Bus885 -0.50 0.0 0.0 0.0 1.079 100 1 8.0 1.6 0 0 0 0 0 0 0 8.0 0 0 0 8.0 8.0 0 0 2 0 0 2 0 1 0
GenCo28 Bus876 -0.45 0.0 0.0 0.0 1.046 100 1 54.0 10.8 0 0 0 0 0 0 0 54.0 0 0 0 54.0 54.0 0 0 2 0 0 2 0 1 0
GenCo37 Bus1118 -5.10 0.0 0.0 0.0 1.041 100 1 55.0 11.0 0 0 0 0 0 0 0 55.0 0 0 0 55.0 55.0 0 0 2 0 0 2 0 1 0

case2848rte

GEN_BUS PG QG QMAX QMIN VG MBASE GEN_STATUS PMAX PMIN PC1 PC2 QC1MIN QC1MAX QC2MIN QC2MAX RAMP_AGC RAMP_10 RAMP_30 RAMP_Q APF STARTUP_RAMP SHUTDOWN_RAMP MINIMUM_UP_TIME MINIMUM_DOWN_TIME MODEL STARTUP SHUTDOWN NCOST COST_2 COST_1 COST_0
GenCo8 Bus1127 -0.66 0.00 0.00 0.00 1.033 100 1 16.00 3.20 0 0 0 0 0 0 0 16.00 0 0 0 16.00 16.00 0 0 2 0 0 2 0 1 0
GenCo19 Bus642 -0.62 0.22 3.00 -3.00 1.013 100 1 10.33 2.07 0 0 0 0 0 0 0 10.33 0 0 0 10.33 10.33 0 0 2 0 0 2 0 1 0
GenCo20 Bus642 -0.22 0.22 3.00 -3.00 1.013 100 1 10.33 2.07 0 0 0 0 0 0 0 10.33 0 0 0 10.33 10.33 0 0 2 0 0 2 0 1 0
GenCo40 Bus1157 -0.71 0.00 0.00 0.00 1.007 100 1 7.20 1.44 0 0 0 0 0 0 0 7.20 0 0 0 7.20 7.20 0 0 2 0 0 2 0 1 0
GenCo47 Bus2754 -0.64 0.00 0.00 0.00 1.029 100 1 11.50 2.30 0 0 0 0 0 0 0 11.50 0 0 0 11.50 11.50 0 0 2 0 0 2 0 1 0
GenCo70 Bus1133 -0.88 0.00 0.00 0.00 1.027 100 1 17.25 3.45 0 0 0 0 0 0 0 17.25 0 0 0 17.25 17.25 0 0 2 0 0 2 0 1 0
GenCo97 Bus564 -0.83 0.00 0.00 0.00 1.010 100 1 10.00 2.00 0 0 0 0 0 0 0 10.00 0 0 0 10.00 10.00 0 0 2 0 0 2 0 1 0
GenCo98 Bus1047 -0.34 0.00 0.00 0.00 1.038 100 1 12.00 2.40 0 0 0 0 0 0 0 12.00 0 0 0 12.00 12.00 0 0 2 0 0 2 0 1 0
GenCo179 Bus461 -0.68 -5.38 12.42 -11.59 1.060 100 1 41.40 8.28 0 0 0 0 0 0 0 41.40 0 0 0 41.40 41.40 0 0 2 0 0 2 0 1 0
GenCo193 Bus552 -1.27 0.00 0.00 0.00 1.037 100 1 21.00 4.20 0 0 0 0 0 0 0 21.00 0 0 0 21.00 21.00 0 0 2 0 0 2 0 1 0
GenCo207 Bus300 -0.41 0.00 0.00 0.00 1.025 100 1 40.80 8.16 0 0 0 0 0 0 0 40.80 0 0 0 40.80 40.80 0 0 2 0 0 2 0 1 0
GenCo282 Bus1714 -0.41 0.00 0.00 0.00 1.035 100 1 6.00 1.20 0 0 0 0 0 0 0 6.00 0 0 0 6.00 6.00 0 0 2 0 0 2 0 1 0
GenCo334 Bus1164 -2.99 0.00 0.00 0.00 1.020 100 1 8.60 1.72 0 0 0 0 0 0 0 8.60 0 0 0 8.60 8.60 0 0 2 0 0 2 0 1 0

How is this possible?

There are also some generators with PMIN < 0, but PG > 0. For example, with case3012wp

GEN_BUS PG PMAX PMIN
GenCo81 Bus240 179.0 179.0 -200.0
GenCo82 Bus240 179.0 179.0 -200.0
GenCo84 Bus241 179.0 179.0 -200.0
GenCo313 Bus2222 29.0 29.0 -5.0
GenCo314 Bus2222 28.0 28.0 -5.0

Should I still subtract those PG’s from the load? Should I just drop those generators, or should I reset those generators to have a PMIN=0?

Alternately, should I just subtract PMIN from the load instead of PG? If I do that, should I drop those generators, or should I set PMAX = PMAX - PMIN and then set PMIN=0?

kdheepak commented 4 years ago

There are some generators with PG < 0 but have a PMIN > 0.

You can set Gg to 0 and model them like any other generator.

There are also some generators with PMIN < 0, but PG > 0.

This is what I would do in this case. Delete the generators from the dataframe, and add loads of value -{PG} to the same buses, i.e. "subtract PG (not PMIN) from the load" and "drop those generators"

How is this possible?

Welcome to power systems data sets :)

djinnome commented 4 years ago

You can set Gg to 0 and model them like any other generator.

What is Gg? Did you mean PG?

kdheepak commented 4 years ago

Yes, sorry that was a typo on my part.

On Wed, Nov 27, 2019, 4:11 PM Jeremy Zucker notifications@github.com wrote:

You can set Gg to 0 and model them like any other generator.

What is Gg? Did you mean PG?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/power-system-simulation-toolbox/psst/issues/6?email_source=notifications&email_token=AAN2VAMKUCP7MBWYB7UUDRTQV35BNA5CNFSM4I4P3J6KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFK7HZI#issuecomment-559281125, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAN2VAI25VEKI2BZ5GCUG4TQV35BNANCNFSM4I4P3J6A .