OEMOF Solph does not record enough information to support stochastic programming (support wanted)

[FHell]

We have been playing around with various ways to do scenario sampling based analysis of power systems in oemof. The cleanest option seems to be to leverage the stochastic programming language built on top of pyomo, mpi-sppy.

This seems relatively straightforward. What we need to do is the following:

• For each variable and each summand of the objective function, record whether it is general (1st stage) or specific to the scenarios (2nd stage).
• Once the pyomo concrete model is built, collect all general variables/summands and pass them to mpisppy.utils.sputils.attach_root_node, see for example.

Architecturally this means the following:

• [ ] Figure out where to store the stage information for variables
• [ ] Figure out where to store the objective function summands
• [ ] Figure out where to store the stage information for them

-> Probably the blocks/components? All blocks/components?

And writing a utility function that

• [ ] Extracts all 1st stage variables from the model
• [ ] Extracts all 1st stage summands from the model

-> So the question: How do we iterate over all blocks/components and extract information from them?

If you are interested in helping with that please get in touch! We particularly need help with the architectural aspects.

In the meantime we will create a prototype where we record the information manually.

[c-moeller]

@e-zolotarevskaya I cannot assign you. No idea why.

[simnh]

Are you planning to build a stochastic programming API? I am not too much familiar with SP, however, I think it would require a whole new type of model (similar to the ExpansionModel)?

With regard to the looping over blocks we do this here already for the "normal" decision variables to create our results dataframe, maybe this helps?

[FHell]

That is extremely helpful indeed! @e-zolotarevskaya, have a look! Right now we hope that we can do this without the need for a full additional model. This is because the mpi-sppy package is built to work on top of (variations of) vanilla pyomo models. So we hope that we can work on top of vanilla oemof models. We'll see.

[jokochems]

Sorry for the late response to this one: I'm not so very familiar with stochastical programming either, but I think, it could be quite similar to the multi-period modeling I have just started to revisit, see oemof/oemof-solph#793 resp. oemof/oemof-solph#750.

I've discussed parts of it with @p-snft quite a while ago:

• Basically, you would have to index everything with a dedicated scenario index, i.e. to create another set in the models class. As @simnh suggested, you could create a dedicated model class for that. In addition, you usually would define a weighting, i.e. a propability distribution for your uncertain scenario parameters.
• An alternative could be to integrate everything into the existing (models and class) infrastructure and make the single-scenario consideration a special case which would require less duplicates, but might slightly impact performance. I think, the latter might not be much of an issue, but I haven't yet had the possibility to check that.
• Anyways, I think, this would be a very great feature for the community, but you propably have to touch quite a lot and it's probably not possible or at least not straightforward to use third-party packages for the purpose of stochastic modeling.

I agree that we definitely should discuss it in the upcoming meeting https://github.com/oemof/oemof/issues/96

[jokochems]

After discussing it at the dev meeting https://github.com/oemof/oemof/issues/96, this should be easier to add than I originally assumed. Here is a try to summarize. @simnh and @gnn: Please correct me if I'm wrong and please add to it.

• For your special use case, trying to add a (rather generalized) component that encapsulates your constraints would be the first step.
• We basically have a flow variable in the model (oemof.solph.models.Model) that it indexed by inflow, outflow and a timestep model.flow[i, o, t].
• You have to filter the flows for the ones that shall be fixed in the first stage. You could do so, by introducing an attribute, such as stochastic and then use a constraint grouping to group the respective flows.
• You have to interchange the "standard" pyomo by mpi-sppy and provide it with the list of flows that shall be fixed.
• Until our pyomo version issue isn't resolve, you'd have to use an earlier version of mpi-sppy that is compatible with pyomo version >= 5.7.0 and < 5.7.3