search

openPMD / openPMD-api

:floppy_disk: C++ & Python API for Scientific I/O
https://openpmd-api.readthedocs.io
GNU Lesser General Public License v3.0
138 stars 51 forks source link

HDF5 Exception Message on Broken Mesh Attribute #1527

Closed ax3l closed 3 months ago

ax3l commented 1 year ago

With @RemiLehe today, we looked at a file that had an unsupported 128bit long double data type in the mesh position attribute.

We noticed that the exception when skipping the mesh during read was not pointing at the position attribute name yet, which we might be able to add.

The exception traceback we got was

openPMD series: test_output_gaussian_00000
openPMD standard: 1.1.0
openPMD extensions: 0

data author: unknown
data created: 2023-09-06 14:49:05 -0700
data backend: HDF5
generating machine: unknown
generating software: openPMD-api (version: 0.15.0)
generating software dependencies: unknown

number of iterations: 1 (fileBased)
HDF5-DIAG: Error detected in HDF5 (1.14.1-2) thread 1:
  #000: H5Tcset.c line 57 in H5Tget_cset(): operation not defined for data type class
    major: Datatype
    minor: Feature is unsupported
  all iterations: [AbstractIOHandlerImpl] IO Task READ_ATT failed with exception. Clearing IO queue and passing on the exception.
Cannot read mesh with name 'laserEnvelope' and will skip it due to read error:
Read Error in backend HDF5
Object type:    Attribute
Error type: UnexpectedContent
Further description:    [HDF5] Unsupported simple attribute type 216172782113784119 for position.
(Info for debugging: order 0
prec 64
ebias 1023
fields 63 52 11 0 52norm 0
cset -1
sign -1

x-ref https://github.com/LASY-org/lasy/pull/179

ax3l commented 1 year ago

ah, actually the exception has the name:

Further description: [HDF5] Unsupported simple attribute type 216172782113784119 for position.

franzpoeschel commented 10 months ago

Error recovery is currently not yet as fine-grained as it would ideally be, the currently-released version is the first approach of openPMD-api at error recovery after all. We are currently planning to refactor (to a large part rewrite) the handling of scientific default values such as the position. Currently, there is basically explicit logic for each single attribute that has a definition in the openPMD standard, which is not really ideal nor is it flexible. This issue is linked to https://github.com/openPMD/openPMD-api/issues/1439 since a rewrite should consider this point. Ref. the issue description:

  1. Reading defaults would happen at the same time as now (during parsing). It would deal leniently with missing / wrong values, e.g. by just setting the default, or ignoring it.
ax3l commented 3 months ago

The error message looks reasonable for now and points to the attribute named position being wrong. Tracking in #1439 for the underlying issue.