OSeMOSYS Short Results as Individual CSV files

[willu47]

• Adds functionality to write out CSV file for each results parameter for short and long versions of OSeMOSYS formulation
• Demonstrates the short and long versions of OSeMOSYS produce identical results

Table Format

Each table is produced using the following syntax:

table <name>
    {indexes: conditional expression}
    OUT "CSV"
    "<path to csv file>":
    <index>~<column label in csv file>...
    <calculated results parameter expression>~VALUE

For example:

table AnnualEmissions
    {r in REGION, e in EMISSION, y in YEAR:
        sum{t in TECHNOLOGY, l in TIMESLICE, m in MODE_OF_OPERATION:
            EmissionActivityRatio[r,t,e,m,y] <> 0}
            RateOfActivity[r,l,t,m,y] > 0}
    OUT "CSV"
    "results/AnnualEmissions.csv":
    r~REGION, e~EMISSION, y~YEAR,
    sum{t in TECHNOLOGY, l in TIMESLICE, m in MODE_OF_OPERATION:
        EmissionActivityRatio[r,t,e,m,y] <> 0}
        EmissionActivityRatio[r,t,e,m,y] * RateOfActivity[r,l,t,m,y] * YearSplit[l,y]~VALUE;

This produces csv files in the results directory with headers of the index; REGION, EMISSION, YEAR, VALUE for the above example.

The conditional expressions in the initial index have been added to prevent writing of zero values, which reduces file size.

Identical results

The tests have been restructured and can be run using the Python package pytest. The tests contain a single fixture which sets up and runs the model, dumping the results into a temporary directory. Each test then compares the data written out from the model for a single parameter (corresponding to a single CSV file) with a canonical version hardcoded in the body of the test code. The tests can be viewed in tests/test_gnu_mathprog.py.

[tniet]

Hi @willu47 - is this in addition to the SelectedResults.csv output or is this instead of that? A lot of OSeMOSYS analysis out there is based on the existing output structure so I'm not sure we want to replace it, but adding the tables would be a good addition.

[willu47]

Hi @tniet, these new tables are an add on at the moment. They make automated testing and processing of the results much easier...

[tniet]

Hi @willu47 - that's probably good, at least for now.

[willu47]

@tniet or @abhishek0208 - This is now ready for review and to merge into master if you are happy

Note that with these changes to the results writing, we are now able to demonstrate that both long and short versions of the code produce identical results. Actually, there are some small rounding differences between the two formulations, but this does not make a large difference.

[tniet]

Hi @willu47 - Looks good to me. Shall I do the merge?

And are there issues we can close from this as well? I think it closes #28 as well as #26?

[willu47]

Hi @tniet - nope, neither of those issues can be closed - #28 actually requires the re-introduction of the TechWithCapacityToMeetPeakTS parameter together with the constraint which used it. And the use conditional operators in the results only effects the writing of results, not the generation of equations, so #26 is still open. I'll check through the other issues and tag any relevant ones in the description above.

[tniet]

Hi @willu47 - Sounds good. Not sure where those two numbers for issues came from now that I look at it again. I'll do the merge.