Variable and constraint generation #77

Closed spine-o-bot closed 3 years ago

In GitLab by @Poncelet on Mar 28, 2019, 12:45

In previous versions of SpineModel, we used to generate variables in the following fashion:

@variable(m, flow[c in commodity(), n in node(), u in unit(); (c,n,u) in a_relation_ship(c,n,u)] >= 0)

Now, we have evolved to something like:

@butcher Dict{Tuple, JuMP.VariableRef}(
        (c, n, u, d, t) => @variable(
            m, base_name="flow[$c, $n, $u, $d, $t]", lower_bound=0
        ) for (c, n, u, d, block) in commodity__node__unit__direction__temporal_block()
                for t in keys(timesliceblocks[block])
    )

If I understand it correctly, this speeds up the generation of constraints for two reasons: 1) we do not need to loop through all possible combinations of indices due to the fact that start from the relationship instead of looping over 1D sets and then applying the membership in the relationship as a condition; 2) the convenience function to look at the relationship is called only once thanks to @butcher

There is one difference however, of which the consequences are not fully clear to me at this point, and I would like to discuss. Namely, in the previous version, the JuMP variables had indices, whereas now, we create a single scalar variable for every combination of the indices. So each individual variable has a unique name "flow[$c, $n, $u, $d, $t]" and has no indices itself. This might not be an issue at all - in the end, the Dict we create allows us to use the indices to find the corresponding JuMP variable and its value. On the other hand, it is quite different as how I would do it in GAMS.

So my main question is: have the consequences of generating separate variables been thought through? Or can we think of possible negative side-affects of this?

All the above is equally valid for constraint generation. On constraint generation, I have one additional question: should we also create a dict for every constraint (allowing us to get a handle on every dual etc.)?

In GitLab by @manuelma on Mar 28, 2019, 13:59

Both 1 and 2 above are correct.

We could still write

@variable(
    m, 
    flow[(c, n, u, d, block) in commodity__node__unit__direction__temporal_block(), 
    lower_bound=0)

but then we would need to write flow[(c, n, u, d, block)] in constrain code. With the Dict approach we get to write just flow[c, n, u, d, block].

So all the Dict is doing is creating more convenient indexing. The same is true for constraints. You could still access your constraints (generated as it is now, without the Dict approach), but you'd need to figure out the right indices, some of them being tuples probably.

I'm curious what do you mean 'quite different as how I would do it in GAMS'?

In GitLab by @manuelma on Mar 28, 2019, 14:05

Looking at your example more in detail, I guess we'd rather need something like

@variable(
    m, 
    flow[
        (c, n, u, d, block) in commodity__node__unit__direction__temporal_block(),
        t in keys(timesliceblocks[block]
    ], 
    lower_bound=0)

and then access it like flow[(c, n, u, d, block), t], which is not what we'd like to do, right? So here is where the Dict becomes handy.

I don't think using the Dict has any negative side effects. I'll take a look at the @variable macro to be sure, though.

In GitLab by @Poncelet on Mar 28, 2019, 14:07

I'm curious what do you mean 'quite different as how I would do it in GAMS'

Well, in GAMS I typically use a single GAMS-variable flow which would have multiple indices. Now I think about it, I don't think I have ever had a model where a variable had different indices, but not all combinations of those indices. I actually don't know if you can declare a variable over a multi-dimensional subset in GAMS...

With the Dict approach we get to write just flow[c, n, u, d, block]

Yes, I also like the way it looks and it is easy to write constraints like this. I'm just wondering whether there might be adverse side-effects. Things I can think of are:

performance in terms of what happens inside the solver. The final equations look the same, so I dont know if this is impacted at all
ease of converting the results back into the db (I think we now have to retrieve the value of every individual variable separately instead of at once of all flow variables?)

So all the Dict is doing is creating more convenient indexing. The same is true for constraints. You could still access your constraints (generated as it is now, without the Dict approach), but you'd need to figure out the right indices, some of them being tuples probably.

So in that respect, I think if we do it for variables, it would be best to do it similar for constraints (thus using dicts).

In GitLab by @manuelma on Mar 28, 2019, 14:10

performance in terms of what happens inside the solver. The final equations look the same, so I dont know if this is impacted at all

We need to look at the @JuMP.variable macro to know about this.

ease of converting the results back into the db (I think we now have to retrieve the value of every individual variable separately instead of at once of all flow variables?)

Dict(k => JuMP.value(v) for (k, v) in flow) returns a Dict of variable values. I'm thinking we need to provide that function

value(d) = Dict(k => JuMP.value(v) for (k, v) in d)

so we can just call SpineModel.value(flow)

In GitLab by @Poncelet on Mar 28, 2019, 14:25

Dict(k => JuMP.value(v) for (k, v) in flow) returns a Dict of variable values. I'm thinking we need to provide that function

value(d) = Dict(k => JuMP.value(v) for (k, v) in d)

so we can just call SpineModel.value(flow)

I like that, I've included it in the dev_merge branch (which should become the new dev branch).

In GitLab by @manuelma on Mar 28, 2019, 15:40

(which should become the new dev branch).

let's hope so, I've been messing around with dev quite a bit, there's SpineInterface in the mix now... Let's see how we manage to merge everything.

In GitLab by @Poncelet on Mar 29, 2019, 09:13

I see, let's try to merge dev with dev_merge next week.

In GitLab by @manuelma on Mar 29, 2019, 10:56

No problem. If you want, just let me know when dev_merge is ready and I'll take care of merging.

In GitLab by @Poncelet on May 9, 2019, 13:44

Decided to keep on using the dicts. Closing issue

In GitLab by @Poncelet on May 9, 2019, 13:44

closed

spine-tools / SpineOpt.jl

Variable and constraint generation #77