I'm working on read a ultra large hdf5 dataset via HDF.PInvoke recently. Due to the reason of .NET array can not be allocated in size greater than 2GB, so if the data inside the hdf5 file its size is greater than 2GB, then the program crashed.
The dataset which its size less than 2GB could be read via the demo code in wiki Cookbook : Datasets, and I guess i could read the dataset which its size greater than 2GB chunk by chunk via HDF.PInvoke, but after google search around it, i was not able to figure out how to implements such code correctly.
Here is the DEMO data in ultra large size that i currently working on: download the P_1900 | Human Prostate dataset from this repository.
Hi,
Many thanks for this great data format library.
I'm working on read a ultra large hdf5 dataset via HDF.PInvoke recently. Due to the reason of .NET array can not be allocated in size greater than 2GB, so if the data inside the hdf5 file its size is greater than 2GB, then the program crashed.
The dataset which its size less than 2GB could be read via the demo code in wiki Cookbook : Datasets, and I guess i could read the dataset which its size greater than 2GB chunk by chunk via HDF.PInvoke, but after google search around it, i was not able to figure out how to implements such code correctly.
Here is the DEMO data in ultra large size that i currently working on: download the P_1900 | Human Prostate dataset from this repository.