search

gem / oq-engine

OpenQuake Engine: a software for Seismic Hazard and Risk Analysis
https://github.com/gem/oq-engine/#openquake-engine
GNU Affero General Public License v3.0
383 stars 277 forks source link

Problem loading avg_losses-stats #4986

Closed ptormene closed 5 years ago

ptormene commented 5 years ago 
Unable to complete data extraction 
svir.tasks.extract_npz_task.ExtractFailed: Unable to extract http://localhost:8800/v1/calc/1/extract/losses_by_asset with parameters None (Internal Server Error):
KeyError: "Can't open attribute (can't locate attribute: 'stats')"
  File "/home/paolo/projects/oq-engine/openquake/server/views.py", line 679, in extract
    aw = _extract(ds, what + query_string)
  File "/home/paolo/projects/oq-engine/openquake/calculators/extract.py", line 178, in __call__
    return ArrayWrapper.from_(data)
  File "/home/paolo/projects/oq-engine/openquake/baselib/hdf5.py", line 425, in from_
    array, attrs = (), dict(obj)
  File "/home/paolo/projects/oq-engine/openquake/calculators/extract.py", line 606, in extract_losses_by_asset
    stats = dstore['avg_losses-stats'].attrs['stats']
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/home/paolo/py36/lib/python3.6/site-packages/h5py/_hl/attrs.py", line 60, in __getitem__
    attr = h5a.open(self._id, self._e(name))
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5a.pyx", line 77, in h5py.h5a.open

Plugin side I have a note stating that it is fine to extract 'losses_by_asset' for both output types 'losses_by_asset' and 'avg_losses-stats'.

This is also breaking the integration tests (see https://travis-ci.org/gem/oq-irmt-qgis/jobs/568489251#L766)

ptormene commented 5 years ago

This is probably closely related (extracting 'agg_losses'):

Unable to extract http://localhost:8800/v1/calc/1/extract/agg_losses/nonstructural with parameters None (Internal Server Error):
KeyError: "Can't open attribute (can't locate attribute: 'stats')"
  Fi[...]

[...]le "/home/paolo/projects/oq-engine/openquake/server/views.py", line 679, in extract
    aw = _extract(ds, what + query_string)
  File "/home/paolo/projects/oq-engine/openquake/calculators/extract.py", line 170, in __call__
    data = self[k](dstore, v)
  File "/home/paolo/projects/oq-engine/openquake/calculators/extract.py", line 533, in extract_agg_losses
    stats = dstore['avg_losses-stats'].attrs['stats']
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/home/paolo/py36/lib/python3.6/site-packages/h5py/_hl/attrs.py", line 60, in __getitem__
    attr = h5a.open(self._id, self._e(name))
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5a.pyx", line 77, in h5py.h5a.open
ptormene commented 5 years ago

I've also tried to call the extract without passing the loss_type, as it is done for other output types, but it does not work either:

Unable to extract http://localhost:8800/v1/calc/1/extract/agg_losses with parameters None (Internal Server Error):
ValueError: loss_type not passed in agg_losses/<loss_type>
  File "/home/paolo/projects/oq-engine/openquake/server/views.py", line 679, in extract
    aw = _extract(ds, what + query_string)
  File "/home/paolo/projects/oq-engine/openquake/calculators/extract.py", line 175, in __call__
    data = self[key](dstore, '')
  File "/home/paolo/projects/oq-engine/openquake/calculators/extract.py", line 527, in extract_agg_losses
    raise ValueError('loss_type not passed in agg_losses/<loss_type>')