search

OSeMOSYS / otoole

OSeMOSYS Tools for Energy
https://otoole.readthedocs.io
MIT License
23 stars 17 forks source link

[Bug]: Data conversion to Excel #171

Closed trevorb1 closed 1 year ago

trevorb1 commented 1 year ago

The Issue

If I convert from CSV to excel, and a parameter is not indexed over years but the last column is MODE_OF_OPERATION, the mode gets pivoted on

For example, TechnologyToStorage is given as: image

Expected Behavior

The pivot should only happen on the years column. The output should look like

REGION TECH STORAGE MODE VALUE
Simplicity HYD2 DAM 1 1
Simplicity HYD2 DAM 2 0

Steps To Reproduce

  1. Clone Simplicity with git clone https://github.com/OSeMOSYS/simplicity.git
  2. Run otoole convert csv excel data data.xlsx config.yaml
  3. View both the TechnologyToStorage and TechnologyFromStorage sheets, which are not indexed over year in the CSV files

Log output

OtooleIndexError: TechnologyFromStorage -> Indices inconsistent between config and data. Config indices are ['REGION', 'TECHNOLOGY', 'STORAGE', 'MODE_OF_OPERATION']. Data indices are ['REGION', 'TECHNOLOGY', 'STORAGE', 'YEAR'].

Operating System

Linux

What version of otoole are you running?

Error is introduced in version 1.0.2

Possible Solution

No response

Anything else?

No response

trevorb1 commented 1 year ago

Ive narrowed down the issue to the writing out of data to excel. Reading in the data works fine and writing to CSV and datafile work fine

trevorb1 commented 1 year ago

This was actually an issue with version 1.0.0 as well. But in version 1.0.2, we introduce explicit checking for indices in the data against the config file. So thats why in version 1.0.2 the error is now flagged

trevorb1 commented 1 year ago

This wasn't getting flagged by other parameters before (such as operational life which also not indexed over years), since the logic checked for the number of indices associated with the parameter before pivoting