A Python script to convert OSeMOSYS CPLEX outputs to CBC or CSV format

[willu47]

This script allows you to easily convert a CPLEX output file which is in a wide format, to that of CBC, which is close to 'tidy' format.

In CPLEX format, the year dimension is inferred from the position of the values:

variable_name,<dimensions...>,value1, value2,...,value50

In CBC format, the year is explicit but it has some peculiarities:

0 variable_name,(dimensions,year1),value1 0
0 variable_name,(dimensions,year2),value2 0
...
0 variable_name,(dimensions,year50),value50 0

In CSV format, the year is also explicit, and we remove the spaces and extra characters from the CBC format to give something with three columns that can be imported using an CSV parser:

variable_name,"dimensions,year1",value1
variable_name,"dimensions,year2",value2
...
variable_name,"dimensions,year50",value50

The script includes a funky command line interface:

$ python convert_cplex_to_cbc.py --help
usage: convert_cplex_to_cbc.py [-h] [-s START_YEAR] [-e END_YEAR]
                               [--csv | --cbc]
                               cplex_file output_file

Convert OSeMOSYS CPLEX files into different formats

positional arguments:
  cplex_file            The filepath of the OSeMOSYS cplex output file
  output_file           The filepath of the converted file that will be
                        written

optional arguments:
  -h, --help            show this help message and exit
  -s START_YEAR, --start_year START_YEAR
                        Output only the results from this year onwards
  -e END_YEAR, --end_year END_YEAR
                        Output only the results upto and including this year
  --csv                 Output file in comma-separated-values format
  --cbc                 Output file in CBC format, (default option)

There are also some unit tests in the script file, which you can run using pytest. E.g.

$ pip install pytest
$ pytest convert_cplex_to_cbc.py
=================================================== test session starts ===================================================
platform darwin -- Python 3.7.1, pytest-4.3.1, py-1.8.0, pluggy-0.9.0
rootdir: /Users/will2/repository/osemosys/OSeMOSYS_GNU_MathProg, inifile:
plugins: cov-2.6.1
collected 6 items                                                                                                         

scripts/convert_cplex_to_cbc.py ......                                                                              [100%]

================================================ 6 passed in 0.04 seconds =================================================

[willu47]

Ah, okay, so it should probably raise an error if its passed a file with the wrong format. Also, it requires a CPLEX solution which has already been processed by the transformation script. I'll add these bits into the readme.