search

SlideRuleEarth / sliderule

Server and client framework for on-demand science data processing in the cloud
https://slideruleearth.io
Other
29 stars 12 forks source link

REST VOL crashes on datatype conversion read #42

Closed jpswinski closed 2 years ago

jpswinski commented 3 years ago

When reading a dataset using REST VOL, if the size of the datatype supplied to the read does not match the size of the datatype of the dataset, then the code crashes.

Here is a stack trace of the crash:

#0  __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:262
#1  0x00007faa8f23ed1e in H5D__scatter_mem () from /usr/local/bin/../lib/libhdf5_vol_rest.so
#2  0x00007faa8f240bd2 in H5Dscatter () from /usr/local/bin/../lib/libhdf5_vol_rest.so
#3  0x00007faa8f17d173 in RV_dataset_read () from /usr/local/bin/../lib/libhdf5_vol_rest.so
#4  0x00007faa8f4200dd in H5VL_dataset_read () from /usr/local/bin/../lib/libhdf5_vol_rest.so
#5  0x00007faa8f23b21a in H5Dread () from /usr/local/bin/../lib/libhdf5_vol_rest.so
#6  0x00005620a7b08761 in H5Lib::read (url=0x7faa78006ba0 "hsds:///hsds/ATL03/ATL03_20181019065445_03150111_003_01.h5", 
    datasetname=0x7faa78006ec0 "/gt1r/geolocation/segment_ph_cnt", valtype=RecordObject::REAL, col=0, startrow=0, numrows=-1)
    at /home/ubuntu/sliderule/packages/h5/H5Lib.cpp:387
#7  0x00005620a7b06af7 in H5DatasetDevice::H5DatasetDevice (this=0x7faa7800c850, L=0x7faa740088c8, _role=DeviceObject::READER, 
    filename=0x7faa78007718 "hsds:///hsds/ATL03/ATL03_20181019065445_03150111_003_01.h5", dataset_name=0x7faa78006b28 "/gt1r/geolocation/segment_ph_cnt", id=0, 
    raw_mode=false, datatype=RecordObject::REAL) at /home/ubuntu/sliderule/packages/h5/H5DatasetDevice.cpp:127
#8  0x00005620a7b06505 in H5DatasetDevice::luaCreate (L=0x7faa740088c8) at /home/ubuntu/sliderule/packages/h5/H5DatasetDevice.cpp:68
#9  0x00007faa8f7dd4d5 in ?? () from /lib/x86_64-linux-gnu/liblua5.3.so.0
#10 0x00007faa8f7ea225 in ?? () from /lib/x86_64-linux-gnu/liblua5.3.so.0
#11 0x00007faa8f7dd8c8 in ?? () from /lib/x86_64-linux-gnu/liblua5.3.so.0
#12 0x00007faa8f7dd8f5 in ?? () from /lib/x86_64-linux-gnu/liblua5.3.so.0
#13 0x00007faa8f7dccf7 in ?? () from /lib/x86_64-linux-gnu/liblua5.3.so.0
#14 0x00007faa8f7ddb9f in ?? () from /lib/x86_64-linux-gnu/liblua5.3.so.0
#15 0x00007faa8f7d943e in lua_pcallk () from /lib/x86_64-linux-gnu/liblua5.3.so.0
#16 0x00005620a7aaee0a in LuaEngine::directThread (parm=0x7faa7400b510) at /home/ubuntu/sliderule/packages/core/LuaEngine.cpp:454
#17 0x00007faa8f81f609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#18 0x00007faa8eaaa293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

To reproduce the crash, change the following line so that the "INTEGER" type is instead "REAL": https://github.com/ICESat2-SlideRule/sliderule/blob/42b41d8ef925befecbd4f077f40dab37d8075600/plugins/icesat2/tests/api.py#L72

Note that when the asset is atl03-local, and the read does not go through REST VOL but instead the regular libhdf5 library, then the code does not crash and works as expected.

jpswinski commented 3 years ago

See: HDFGroup/vol-rest/issues/6