Implement conditional statements in each equation to reduce model size and improve performance

[willu47]

There is a lot of potential to reduce the size of the model by implementing conditional statements in the model formulation.

This could remove the need for the short version of the OSeMOSYS code.

The following syntax is used in the index domain definition:

{i in A, (j,k) in B, l in C: i <= 5 and k <> ’Mar’}

[tniet]

Hi @willu47 - I agree, but don't believe it would remove the need for the short code. The short code has a number of other advantages that this wouldn't solve.

This is actually related to issue #13 - when does a constraint bind and when doesn't it. I started work on that one and then realized it was a bigger issue - how to specify when a constraint should not be built... My suggested solution to that issue was to have the default as -1 and then we can do an index domain and, if the parameter (max capacity) is -1, don't create the constraint.

This wouldn't be a hard change to make to the code but requires a decision if that's the direction we want to take and do we do that for all the different constraints. Maybe @abhishek0208 can chime in with his thoughts?

[willu47]

I agree, but don't believe it would remove the need for the short code. The short code has a number of other advantages that this wouldn't solve.

What are these advantages? Genuinely interested as I just know the disadvantages i.e.:

• another separate codebase to maintain, which should track any developments made to the main code
• less readable
• harder to maintain due to the above
• no access to intermediate variables, which are useful for exploring marginal costs

[tniet]

Hey @willu47 - the main advantage of the shorter code is faster solve times since there are many fewer variables and many fewer constraints. I don't have detailed info on the exact speed differences (I've only used the short code, and don't find it challenging to read, but that's me). My understanding is that it is a significant difference.

[willu47]

Yes, the performance improvements are significant. For example, generating the simplicity model using the short code results in

142133 lines were written
--- Problem Characteristics ---
Number of rows               =    19797
Number of columns            =    16254
Number of non-zeros (matrix) =   117953
Number of non-zeros (objrow) =     7479

versus

1462515 lines were written
--- Problem Characteristics ---
Number of rows               =   388736
Number of columns            =   490193
Number of non-zeros (matrix) =  1020361
Number of non-zeros (objrow) =       27

So the LP file is 10 times bigger even for this tiny model. The difference may be even larger for more complex models.

You're right in that even with maximum optimisation of the formulation size using conditional index statements, the short version of the model would be smaller.

[tniet]

Yes, that is pretty significant - thanks for doing that!

One option may be to shift our focus so the short code is the 'master' that what we update/maintain. The long code can then be more of a documentation/learning tool. I doubt any serious users are running the long code for any serious modelling so code updates and modifications are usually suggested to the short code from what I've seen. Making this shift in focus might reduce the maintenance challenges...

[abhishek0208]

Hey @willu47 and @tniet, the short code was developed to with the aim of reducing the number of lines of code to the bare minimum (the aim was one line per block of functionality). The (significant) reduction in matrix generation time was an expected by-product but ended up being the more useful outcome. But since that wasn't the objective, there is still scope for optimising the short code. Expanding some of the lines in the short code might well reduce matrix generation time. Certain calculations (AccumulatedNewCapacity for e.g.) are done repeatedly in the short code whereas the long code just does them once. Another instance could be to add a DiscountingFactor which is calculated repeatedly in the short code at the moment.