mannau / h5

Interface to the HDF5 Library
Other
70 stars 22 forks source link

Vlen-integer is not correctly written or read #47

Open kommaklar opened 7 years ago

kommaklar commented 7 years ago

I am trying to write data into a hdf5 file using h5. The data consists of vectors of integers with different lengths - each integer in a vector is < 69. These vectors are in a list even though the documentation for writeDataSet says that only vector, array or matrix are supported. However, the code runs without errors. But when reading the data back some integers in some vectors are wrong (absolute values > 69). The length of the vectors is fine however.

I have attached a zip file with sample data and some code to reproduce the problem. The problem does not occur every time. But in a fresh R session usually the first attempt fails - afterwards, i.e. when executing h5Bug.R again and again any one of them can and will fail on two different systems. Using all data I have an error rate of about 0.9% for 11 million of these vectors. I have never used hdf5 nor h5 before so maybe I am doing something wrong. h5bug.zip

mannau commented 7 years ago

Correct - there seems to be a bug:

library(h5)
load("testData.Rda")
testFile <- h5file("test.h5", mode = "a")
testFile["test"] <- testData
h5close(testFile)

testFileRead <- h5file("test.h5", mode = "r")
identical(testFileRead["test"], testData)
[1] FALSE
h5close(testFileRead)