search

nismod / smif

Simulation Modelling Integration Framework
http://www.itrc.org.uk
MIT License
22 stars 6 forks source link

Incomprehensible error when reading a scenario file which doesn't match dimensions #320

Closed willu47 closed 5 years ago

willu47 commented 5 years ago

When using the following dimension configuration:

name: population
description: UK population, total people
provides:
  - name: population
    description: ''
    dims:
      - lad_uk_2016
    dtype: float
    unit: people
variants:
  - name: pop_baseline
    description: Baseline population for the UK
    data:
      population: population__lad.csv
  - name: pop_high
    description: High population for the UK
    data:
      population: tbd

where population__lad.csv contains the incorrect header such as timestep, region, value instead of timestep, lad_uk_2016, value the following error is raised:

Traceback (most recent call last):
  File "/home/vagrant/nismod/lib/python3.5/site-packages/pandas/core/indexes/base.py", line 2656, in get_loc
    return self._engine.get_loc(key)
  File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'lad_uk_2016'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vagrant/nismod/lib/python3.5/site-packages/smif/controller/scheduler.py", line 178, in add
    self._run(job_graph, job_graph_id)
  File "/home/vagrant/nismod/lib/python3.5/site-packages/smif/controller/scheduler.py", line 246, in _run
    model.before_model_run(data_handle)
  File "models/energy_demand/run.py", line 282, in before_model_run
    pop_array_by = data_handle.get_base_timestep_data('population')
  File "/home/vagrant/nismod/lib/python3.5/site-packages/smif/data_layer/data_handle.py", line 430, in get_base_timestep_data
    return self.get_data(input_name, RelativeTimestep.BASE)
  File "/home/vagrant/nismod/lib/python3.5/site-packages/smif/data_layer/data_handle.py", line 298, in get_data
    data = self._get_scenario(dep, timestep)
  File "/home/vagrant/nismod/lib/python3.5/site-packages/smif/data_layer/data_handle.py", line 372, in _get_scenario
    timestep
  File "/home/vagrant/nismod/lib/python3.5/site-packages/smif/data_layer/store.py", line 502, in read_scenario_variant_data
    return self.data_store.read_scenario_variant_data(key, spec, timestep)
  File "/home/vagrant/nismod/lib/python3.5/site-packages/smif/data_layer/file/file_data_store.py", line 85, in read_scenario_variant_data
    data = self._read_data_array(path, spec, timestep)
  File "/home/vagrant/nismod/lib/python3.5/site-packages/smif/data_layer/file/file_data_store.py", line 387, in _read_data_array
    dataframe.set_index(spec.dims, inplace=True)
  File "/home/vagrant/nismod/lib/python3.5/site-packages/pandas/core/frame.py", line 4178, in set_index
    level = frame[col]._values
  File "/home/vagrant/nismod/lib/python3.5/site-packages/pandas/core/frame.py", line 2927, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/vagrant/nismod/lib/python3.5/site-packages/pandas/core/indexes/base.py", line 2658, in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
  File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'lad_uk_2016'