osemosys2iamc

Use this Python package to convert OSeMOSYS results to the IAMC format

Acknowledgements

This work was financially supported by:

• The European Climate and Energy Modelling Forum (ECEMF) has received funding from the European Union’s Horizon 2020 Research and Innovation programme under the grant agreement No 101022622
• The IAM COMPACT project has received funding from the European Union’s HORIZON EUROPE Research and Innovation Programme under grant agreement No 101056306

<img src="docs/images/ecemf.png" alt="European Climate and Energy Modelling Forum Logo" style="display: inline-block; margin: 100 auto; height: 100px"><img src="docs/images/iamcompact.png" alt="IAM COMPACT Logo" style="display: inline-block; margin: 100 auto; height: 100px">

Install from Python Packaging Index (PyPI)

pip install osemosys2iamc

Run the package

$ osemosys2iamc --help
osemosys2iamc <inputs_path> <results_path> <config_path> <output_path>

inputs_path: Path to a folder of csv files (OSeMOSYS inputs). File names should correspond to OSeMOSYS parameter names. results_path: Path to a folder of csv files (OSeMOSYS results). File names should correspond to OSeMOSYS variable names. config_path: Path to the configuration file (see below) output_path: Path to the .xlsx file you wish to write out

The IAMC format

The IAMC format was developed by the Integrated Assessment Modeling Consortium (IAMC) and is used in many model comparison projects at the global and national level. It can be used for integrated-assessment models, energy-systems scenarios and analysis of specific sectors like transport, industry or buildings.

The format is a tabular structure with the columns model, scenario, region, variable, unit, and a time domain. Each project defines "codelists" to be used across modelling teams for comparison and analysis of results.

The most recent high-profile application of the IAMC format is the AR6 Scenario Explorer hosting the scenario ensemble supporting the quantitative assessment in the contribution by Working Group III to the IPCC's Sixth Assessment Report (AR6).

Please refer to the Horizon 2020 project openENTRANCE for more information about the format and its usage in that project.

Writing a configuration file

Write a configuration file in YAML format. A simple configuration file with two result variables looks like this:

model: OSeMBE v1.0.0
scenario: DIAG-C400-lin-ResidualFossil
region: iso2_x, iso3_x, from_csv, or a name of a country/region [substitute x with start, end, or a positive number]
results:
- iamc_variable: 'Carbon Capture|Biomass'
  tech_emi: ['(?=^.{2}(BM))^.{4}(CS)']
  emissions: [CO2]
  unit: kt CO2/yr
  transform: abs
  osemosys_param: AnnualTechnologyEmission
- iamc_variable: 'Capital Investment'
  capacity: ['^.*$']
  unit: MUSD
  osemosys_param: CapitalInvestment

The first section of the configuration file with the keys model, scenario, and region are used to define the metadata for the IAMC template.

model and scenario are user-defined, taking any word or phrase and having no set format. region may be defined using 4 methods:

• iso2_x - using the ISO 3166-1 alpha-2 country codes located in the names of technologies/fuels/emissions (depending on the file) at the location given by the user.*
• iso3_x - using the ISO 3166-1 alpha-3 country codes located in the names of technologies/fuels/emissions (depending on the file) at the location given by the user.*
• from_csv - using the region as defined in the REGION column of the CSV file
• Any other word or phrase the user defines (other than an invalid iso option) will be used as the region for all variables. eg. Austria, Nepal, Ethiopia, Guyana, etc.

The x in the above ISO options is to be replaced by:

• start, if the codes are at the beginning of the names
• end, if the codes are at the end of the names, or
• a positive number indicating the position of the first letter of the code in the name. eg. iso2_5 will target the 'GH' in 'POWRGHSOL'

The second section, results, is where you describe each of the IAMC variables and provide instructions to osemosys2iamc on how to compute the values.

iamc_variable - this should match one of the IAMC variable names unit - provide the units of the OSeMOSYS results transform - only abs is currently available. This returns the absolute value of the results osemosys_param - provide the name of the result file from which the script should extract the result data

One or more of the following filter keys. These filter the results by one or more columns found in OSeMOSYS results. Following the fitering, the remaining columns except region and year are discarded and rows are summed.

tech_emi - filter the results by TECHNOLOGY and EMISSION columns using the provide regular expression and an emissions entry emissions - a list of emissions to filter the results by the EMISSION column fuel - filter by the FUEL column capacity - filter the TECHNOLOGY column primary_technology - filter the TECHNOLOGY column (can be replaced by capacity key) excluded_prod_tech - filter the TECHNOLOGY column (can be replaced by capacity key) el_prod_technology - filter the TECHNOLOGY column (can be replaced by capacity key) demand - filters by the FUEL column (final energy)

The value for each of these keys is a list of regular expressions. These regular expressions are used to filter the rows of data in the chosen column to those that match the regular expression.

Writing regular expressions can be tricky, but there are useful tools to help. Below we provide some examples:

^.{2}(WI) match rows with any two characters followed by WI

(?=^.{2}(HF))^((?!00).)*$ match rows with HF in 3rd and 4th position which do not include 00

(?=^.{2}(NG))^((?!(00)|(CS)).)*$ match rows with NG in 3rd and 4th position that do not include 00 or CS

^.{6}(I0) match rows which contain any 6 characters followed by IO in the 7th and 8th position

Putting this together, the following entry extracts results from the result file ProductionByTechnologyAnnual.csv, filters out the rows by matching values in the TECHNOLOGY column with a list of 6 regular expressions (this is an OR operation) and assigns the unit PJ/yr and adds the aggregated (summing over region and year) total under Primary Energy in the IAMC template format.

- iamc_variable: 'Primary Energy'
  primary_technology: ['^.{6}(I0)','^.{6}(X0)','^.{2}(HY)','^.{2}(OC)','^.{2}(SO)','^.{2}(WI)']
  unit: PJ/yr
  osemosys_param: ProductionByTechnologyAnnual

List of relevant IAMC variables for OSeMOSYS

Primary Energy

Primary Energy Primary Energy|Biomass Primary Energy|Coal Primary Energy|Fossil Primary Energy|Gas Primary Energy|Geothermal Primary Energy|Hydro Primary Energy|Non-Biomass Renewables Primary Energy|Nuclear Primary Energy|Ocean Primary Energy|Oil Primary Energy|Other Primary Energy|Secondary Energy Trade Primary Energy|Solar Primary Energy|Wind

Priority Variables