BIC-MNI / libminc

libminc is the core library and API of the MINC toolkit
Other
19 stars 29 forks source link

Limit HDF5 file versioning to 1.8.x #97

Closed gdevenyi closed 6 years ago

gdevenyi commented 6 years ago

ITKv5 recently pulled in HDF5 1.10.x. When they did, the MINC files they started generating were no longer readable with minc-toolkit because of HDF5 on-disk versioning change.

This change adapts the existing HDF5 versioning to recognize the new defs from 1.10.x and use them if defined, limiting the on-disk format to the 1.8.x series.

Implemented according to "What does this change mean to an HDF5 application?" at: https://www.hdfgroup.org/2018/04/why-should-i-care-about-the-hdf5-1-10-2-release/

gdevenyi commented 6 years ago

From what I can find, this is the fix for the problem, however when I apply this patch to the ITK libminc and rebuild, it doesn't fix the problem.

Not sure if I'm missing something?

gdevenyi commented 6 years ago

Further docs: https://portal.hdfgroup.org/display/HDF5/H5P_SET_LIBVER_BOUNDS

gdevenyi commented 6 years ago

This is error I get when I try to use minc-toolkit (built against a HDF5 1.8.x) to read a MINC file generated by ITK+MINC built against 1.10.x (with included patches).

12:10:42 [devgab@cicws24:/tmp] $ mincheader /tmp/test.mnc
/opt/quarantine/minc-toolkit/1.9.16/src/minc-toolkit-v2/libminc/libsrc/netcdf_convenience.c:573 (from miopen): Unable to open file '/tmp/test.mnc'
mincinfo: Error opening file "/tmp/test.mnc"
12:11:06 [devgab@cicws24:/tmp] $ mincstats /tmp/test.mnc
/opt/quarantine/minc-toolkit/1.9.16/src/minc-toolkit-v2/libminc/libsrc/netcdf_convenience.c:573 (from miopen): Unable to open file '/tmp/test.mnc'
ncvarid: ncid -1: NetCDF: Not a valid ID

It looks like the miopen is falling through right to the end based on strace:

open("/tmp/test.mnc", O_RDONLY)         = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=22257321, ...}) = 0
mmap(NULL, 528384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb73e132000
lseek(3, 0, SEEK_SET)                   = 0
read(3, "\211HDF\r\n\32\n", 8)          = 8
lseek(3, 0, SEEK_SET)                   = 0
read(3, "\211HDF\r\n\32\n\3", 9)        = 9
munmap(0x7fb73e132000, 528384)          = 0
close(3)                                = 0
open("/tmp/test.mnc", O_RDONLY)         = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=22257321, ...}) = 0
lseek(3, 0, SEEK_SET)                   = 0
read(3, "\211HDF\r\n\32\n", 8)          = 8
close(3)                                = 0
open("/tmp/test.mnc", O_RDONLY)         = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=22257321, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb73e20b000
read(3, "\211HDF\r\n\32\n\3\10\10\0\0\0\0\0\0\0\0\0\377\377\377\377\377\377\377\377\251\236S\1"..., 4096) = 4096
close(3)                                = 0
munmap(0x7fb73e20b000, 4096)            = 0
open("/tmp/test.mnc", O_RDONLY)         = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=22257321, ...}) = 0
brk(0x8ac000)                           = 0x8ac000
lseek(3, 0, SEEK_SET)                   = 0
read(3, "\211HDF\r\n\32\n", 8)          = 8
lseek(3, 0, SEEK_SET)                   = 0
read(3, "\211HDF\r\n\32\n\3", 9)        = 9
close(3)                                = 0
write(2, "/opt/quarantine/minc-toolkit/1.9"..., 111/opt/quarantine/minc-toolkit/1.9.16/src/minc-toolkit-v2/libminc/libsrc/netcdf_convenience.c:573 (from miopen): ) = 111
write(2, "Unable to open file '/tmp/test.m"..., 35Unable to open file '/tmp/test.mnc') = 35
write(2, "\n", 1
)                       = 1
write(2, "mincinfo: Error opening file \"/t"..., 45mincinfo: Error opening file "/tmp/test.mnc"
) = 45
exit_group(1)                           = ?
+++ exited with 1 +++
gdevenyi commented 6 years ago

Confirmed that I'm not forcing an 1.8.x file on save yet. I was above to use h5format_convert to get a file readable by minc-toolkit.

gdevenyi commented 6 years ago

Looks like I needed to move some of the version sets to right after the file object create.

This patch applied to ITK's libminc now produces MINC files that are 1.8.x compatible

vfonov commented 6 years ago

thank you for looking into this

gdevenyi commented 6 years ago

While this fixes the 1.8 support for libminc, it looks like there's a ITK glue layer which is still causing issues.

I had HDF5_USE_18_API_DEFAULT set on on ITK cmake accidentally which masked that the problem still exists.

Looking at the MINC-ITK CXX glue layers now...