--export-outputs fails at losses_by_event

[dynaryu]

(oqv37) hyeuk@hyeuk-GA:~/Projects/York/Rp2500/output_37n/tmp$ oq --version
3.7.0-git9725e6c86a

(oqv37) hyeuk@hyeuk-GA:~/Projects/York/Rp2500/output_37n$ oq engine --export-outputs 263 ./tmp
Exporting agglosses...
./tmp/agglosses_263.csv
Exporting events...
./tmp/events_263.csv
Exporting fullreport...
There is no exporter for fullreport, csv,xml
Exporting gmf_data...
./tmp/gmf-data_263.csv
./tmp/sitemesh_263.csv
Exporting input...
There is no exporter for input, csv,xml
Exporting losses_by_asset...
./tmp/losses_by_asset-rlz-000_263.csv
Exporting losses_by_event...
Traceback (most recent call last):
  File "/home/hyeuk/openquake/src/oq-master/openquake/engine/export/core.py", line 65, in export_from_db
    exported = export(output_key, dstore)
  File "/home/hyeuk/openquake/src/oq-master/openquake/baselib/general.py", line 551, in __call__
    return self[key](obj, *args, **kw)
  File "/home/hyeuk/openquake/src/oq-master/openquake/calculators/export/risk.py", line 260, in export_losses_by_event
    table = add_columns(aw.to_table(), **columns)
  File "/home/hyeuk/openquake/src/oq-master/openquake/baselib/hdf5.py", line 553, in to_table
    out.append(values + tuple(val))
TypeError: 'numpy.float32' object is not iterable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hyeuk/miniconda3/envs/oqv37/bin/oq", line 11, in <module>
    load_entry_point('openquake.engine', 'console_scripts', 'oq')()
  File "/home/hyeuk/openquake/src/oq-master/openquake/commands/__main__.py", line 54, in oq
    parser.callfunc()
  File "/home/hyeuk/openquake/src/oq-master/openquake/baselib/sap.py", line 181, in callfunc
    return self.func(**vars(namespace))
  File "/home/hyeuk/openquake/src/oq-master/openquake/baselib/sap.py", line 251, in main
    return func(**kw)
  File "/home/hyeuk/openquake/src/oq-master/openquake/commands/engine.py", line 251, in engine
    hc_id, os.path.expanduser(target_dir), exports or 'csv,xml'):
  File "/home/hyeuk/openquake/src/oq-master/openquake/engine/export/core.py", line 102, in export_outputs
    dskey, job_id, datadir, target_dir, export_types):
  File "/home/hyeuk/openquake/src/oq-master/openquake/engine/export/core.py", line 125, in export_output
    yield from export_from_db(outkey, calc_id, datadir, target_dir)
  File "/home/hyeuk/openquake/src/oq-master/openquake/engine/export/core.py", line 72, in export_from_db
    (output_key + (version, tb_str, err)))
openquake.engine.export.core.DataStoreExportError: Could not export losses_by_event in csv
  File "/home/hyeuk/openquake/src/oq-master/openquake/engine/export/core.py", line 65, in export_from_db
    exported = export(output_key, dstore)
  File "/home/hyeuk/openquake/src/oq-master/openquake/baselib/general.py", line 551, in __call__
    return self[key](obj, *args, **kw)
  File "/home/hyeuk/openquake/src/oq-master/openquake/calculators/export/risk.py", line 260, in export_losses_by_event
    table = add_columns(aw.to_table(), **columns)
  File "/home/hyeuk/openquake/src/oq-master/openquake/baselib/hdf5.py", line 553, in to_table
    out.append(values + tuple(val))
'numpy.float32' object is not iterable

[micheles]

I cannot reproduce the bug with the current master. I try it with the ebrisk demo. If you can reproduce it with the current master, please send the files you are using.

[dynaryu]

When I used oq engine --run job.ini --exports csv, there is no output starting with losses_by_event, but somehow oq engine --export-outputs tries to export it and failed as above. Note that the simulation that I ran was a scenario risk calculation where I imported a single gmf. Hope it helps to replicate the error. Otherwise, I'm happy to pass the input data.

[micheles]

You must give me the inputs because I cannot reproduce the issue. Even better, send me a reduced version of the calculation that it is small enough to be used as a test case.

[dynaryu]

I figured out the reason. If the keyword of "aggregate_by" exists in the job.ini for the scenario risk calculation, then somehow the engine did not export result by event.

[micheles]

aggregate_by is implemented only for ebrisk. I will add a check.