search

HDFGroup / vol-rest

HDF5 REST VOL Connector
Other
5 stars 8 forks source link

Segmentation fault on datatype conversion read #6

Closed jpswinski closed 9 months ago

jpswinski commented 3 years ago

It appears that when reading a dataset using REST VOL, if the size of the datatype supplied to the read does not match the size of the datatype of the dataset, then the code crashes. I was able to read the local file's dataset using libhdf5 with no issues, and the same dataset read from the HSDS file crashed. I did not investigate further.

Here is the stack trace:

#0  __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:262
#1  0x00007faa8f23ed1e in H5D__scatter_mem () from /usr/local/bin/../lib/libhdf5_vol_rest.so
#2  0x00007faa8f240bd2 in H5Dscatter () from /usr/local/bin/../lib/libhdf5_vol_rest.so
#3  0x00007faa8f17d173 in RV_dataset_read () from /usr/local/bin/../lib/libhdf5_vol_rest.so
#4  0x00007faa8f4200dd in H5VL_dataset_read () from /usr/local/bin/../lib/libhdf5_vol_rest.so
#5  0x00007faa8f23b21a in H5Dread () from /usr/local/bin/../lib/libhdf5_vol_rest.so
jpswinski commented 3 years ago

Another related issue came up recently when we tried to perform a data type conversion using the REST-VOL and the conversion was from a larger datatype (double precision floating point) to a smaller datatype (32-bit integer). The code did not crash, but returned garbage.

It appears that the REST-VOL does not perform any data type conversions at all, and provides no error to the calling function when a read requires a conversion.

jreadey commented 1 year ago

HSDS does not currently support type conversion. If we want to do this on the client side the rest-vol would need to do so. In the meantime we should disable type conversion within the rest vol to avoid a segmentation fault.

mattjala commented 9 months ago

Type conversion is handled in #72.