Hdf5 files not being released on bigdataviewer window closing (Windows)

[NicoKiaru]

@martinschorb noticed in https://github.com/bigdataviewer/bigdataviewer-playground/issues/77 that hdf5 files are not released from bigdataviewer when the window is closed.

In FIJI:

• open an xml / hdf5 with Plugins>BigDataViewer> Open Xml/Hdf5
• try to delete the hdf5 file -> not possible because it's accessed by java (ok)
• close the bdv window
• try to delete the hdf5 file -> still not possible because it's accessed by java (not released)
• only closing fiji releases the file

Actually it seems to be OS dependent:

• On Mac it looks like release is not important. (deleting works).
• On Windows, files are not release the file when you close the window.

[martinschorb]

Update: probably it is not OS dependent but rather at the file-system level. Our network storage would not allow deletion of open files no matter what OS is accessing.

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/bigdataviewer-bigstitcher-file-handle/65680/2

[mkitti]

I wonder if this is still the case after https://github.com/bigdataviewer/bigdataviewer-core/pull/131 since I added some calls to H5Fclose there. Whether they are being invoked may be another matter.

[mkitti]

Perhaps to help narrow this down, are you able to open the same HDF5 file with multiple FIJI clients? On Windows? On Mac? Over a network file system?

[NicoKiaru]

Thanks for looking into this! I'll try to do a few tests tomorrow

[lguerard]

Hi both,

I can confirm that this is still happening. The only way I found so far to clear up the H5 handle is to completely close Fiji which is quite annoying...

[mkitti]

Is the primary issue due to file locking?

One way to address this would be to prevent HDF5 from locking the file in the first place.

This is an API call that accomplishes this: https://portal.hdfgroup.org/display/HDF5/H5P_SET_FILE_LOCKING

For users, there is the HDF5_USE_FILE_LOCKING environment variable: https://portal.hdfgroup.org/display/HDF5/Environment+Variables+Used+by+HDF5 https://portal.hdfgroup.org/display/knowledge/How+to+disable+file+locking+-+flock

launching ImageJ using something like this might prevent the issue. HDF5_USE_FILE_LOCKING=FALSE ImageJ-*

It would be helpful if reports from users would specify details about the operating environment. For example, what operating system and version is being used where the issue is occurring.

[mkitti]

It is not clear to me if HDF5_USE_FILE_LOCKING will have an effect when using the library currently packaged in the JHDF5 jar file since that uses HDF5 version 1.10.5 rather than the needed 1.10.7. It may be necessary to try pointing JHDF5 to another shared library by providing the -Dnative.libpath.jhdf5=/path/to/jhdf5.so. We may be able to point directly to the libhdf5_java.so provided by some operating systems. For example, Ubuntu provides HDF5 version 1.10.7. https://packages.ubuntu.com/jammy/amd64/libhdf5-jni/filelist

https://unlimited.ethz.ch/display/JHDF/JHDF5+FAQ

[lguerard]

Hi @mkitti,

This is happening on Windows 10 using an up to date Fiji, hope that helps !