search

grimbough / rhdf5

Package providing an interface between HDF5 and R
http://bioconductor.org/packages/rhdf5
60 stars 21 forks source link

Error creating a dataset with chunks of size 1 #63

Open grimbough opened 4 years ago

grimbough commented 4 years ago

It's not really sensible to allow singleton chunks, but users on the SIB advanced single-cell RNA course found it was crashing. This should be looked into.

grimbough commented 4 years ago

This causes a memory explosion, which is probably why the students saw R crash:

n <- 2048*200*6
fullImg <- array(runif(n), dim = c(2048, 200, 6))
myfile <- "/tmp/myhdf5file.h5"
if(file.exists(myfile)) { file.remove(myfile) }
h5createFile(myfile)
h5createDataset(myfile, "test", dim(fullImg), storage.mode = "double", chunk = c(1,1, 1))
h5write(fullImg, myfile, name="test")