Supported HDF5 version

[aminiussi]

Hi,

We tried to build with HDF5 1.14.x and it either fails at compile time or crashes at runtime (depending on platforms). Was it tested with that 1.14.0 version ?

Thanks

[milancurcic]

I have not tested with HDF5 1.14.x.

On my system, the latest provided by the OS (Ubuntu 22.04 LTS) and which I use to build neural-fortran with GCC is 1.10.7.

The HDF5 that I've built from source with ifort and ifx is 1.12.2, and that works as well.

What error messages do you have and on what specific platforms?

[aminiussi]

11:51:50 [alainm@pollux rel]# rm -rf *
11:51:55 [alainm@pollux rel]# cmake ..  -DCMAKE_BUILD_TYPE=Release
-- The C compiler identification is IntelLLVM 2023.1.0
-- The Fortran compiler identification is IntelLLVM 2023.1.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /data/spack/opt/spack/linux-centos7-x86_64/gcc-12.2.0/intel-oneapi-compilers-2023.1.0-5jqpzcsl5p6f2zxlpcksdjugsgqcryog/compiler/2023.1.0/linux/bin/icx - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: /data/spack/opt/spack/linux-centos7-x86_64/gcc-12.2.0/intel-oneapi-compilers-2023.1.0-5jqpzcsl5p6f2zxlpcksdjugsgqcryog/compiler/2023.1.0/linux/bin/ifx - skipped
-- Configuring build for parallel execution
-- h5fortran 4.6.3  CMake 3.26.2
-- checking that C and Fortran compilers can link
-- checking that C and Fortran compilers can link - OK
icx: error: unknown argument: '-show'
-- Looking for H5_HAVE_FILTER_SZIP
-- Looking for H5_HAVE_FILTER_SZIP - not found
-- Looking for H5_HAVE_FILTER_DEFLATE
-- Looking for H5_HAVE_FILTER_DEFLATE - found
-- Looking for H5_HAVE_PARALLEL
-- Looking for H5_HAVE_PARALLEL - not found
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.7")  
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
/data/spack/opt/spack/linux-centos7-x86_64/gcc-12.2.0/intel-oneapi-compilers-2023.1.0-5jqpzcsl5p6f2zxlpcksdjugsgqcryog/compiler/2023.1.0/linux/compiler/lib/intel64_lin/for_main.o: In function `main':
for_main.c:(.text+0x19): undefined reference to `MAIN__'
-- Performing Test HDF5_C_links
-- Performing Test HDF5_C_links - Success
-- Performing Test HDF5_Fortran_links
-- Performing Test HDF5_Fortran_links - Failed
-- Could NOT find HDF5 (missing: HDF5_links) (Required is at least version "1.8.7")
CMake Error at rel/_deps/h5fortran-src/CMakeLists.txt:29 (message):
  HDF5 not found or working on the system.  First build HDF5 by:

    cmake -S scripts -B scripts/build -DCMAKE_INSTALL_PREFIX=~/mylibs
    cmake --build scripts/build

    Then build h5fortran:

    cmake -B build -DCMAKE_PREFIX_PATH=~/mylibs
    cmake --build build

-- Configuring incomplete, errors occurred!
11:52:25 [alainm@pollux rel]#

[aminiussi]

Note that HDF5 was built with cmake, here is the organisation of the libs:

-rw-r--r-- 1 spack spack    84794 May 17  2023 jarhdf5-1.14.1.jar
-rw-r--r-- 1 spack spack 10477364 May 17  2023 libhdf5.a
-rw-r--r-- 1 spack spack   852704 May 17  2023 libhdf5_cpp.a
lrwxrwxrwx 1 spack spack       18 May 17  2023 libhdf5_cpp.so -> libhdf5_cpp.so.310
lrwxrwxrwx 1 spack spack       22 May 17  2023 libhdf5_cpp.so.310 -> libhdf5_cpp.so.310.0.1
-rwxr-xr-x 1 spack spack   562568 May 17  2023 libhdf5_cpp.so.310.0.1
-rw-r--r-- 1 spack spack   145624 May 17  2023 libhdf5_f90cstub.a
lrwxrwxrwx 1 spack spack       23 May 17  2023 libhdf5_f90cstub.so -> libhdf5_f90cstub.so.310
lrwxrwxrwx 1 spack spack       27 May 17  2023 libhdf5_f90cstub.so.310 -> libhdf5_f90cstub.so.310.1.0
-rwxr-xr-x 1 spack spack   146656 May 17  2023 libhdf5_f90cstub.so.310.1.0
-rw-r--r-- 1 spack spack   369962 May 17  2023 libhdf5_fortran.a
lrwxrwxrwx 1 spack spack       22 May 17  2023 libhdf5_fortran.so -> libhdf5_fortran.so.310
lrwxrwxrwx 1 spack spack       26 May 17  2023 libhdf5_fortran.so.310 -> libhdf5_fortran.so.310.1.0
-rwxr-xr-x 1 spack spack   365984 May 17  2023 libhdf5_fortran.so.310.1.0
-rw-r--r-- 1 spack spack   232988 May 17  2023 libhdf5_hl.a
-rw-r--r-- 1 spack spack    14982 May 17  2023 libhdf5_hl_cpp.a
lrwxrwxrwx 1 spack spack       21 May 17  2023 libhdf5_hl_cpp.so -> libhdf5_hl_cpp.so.310
lrwxrwxrwx 1 spack spack       25 May 17  2023 libhdf5_hl_cpp.so.310 -> libhdf5_hl_cpp.so.310.0.1
-rwxr-xr-x 1 spack spack    20072 May 17  2023 libhdf5_hl_cpp.so.310.0.1
-rw-r--r-- 1 spack spack    39290 May 17  2023 libhdf5_hl_f90cstub.a
lrwxrwxrwx 1 spack spack       26 May 17  2023 libhdf5_hl_f90cstub.so -> libhdf5_hl_f90cstub.so.310
lrwxrwxrwx 1 spack spack       30 May 17  2023 libhdf5_hl_f90cstub.so.310 -> libhdf5_hl_f90cstub.so.310.0.1
-rwxr-xr-x 1 spack spack    42352 May 17  2023 libhdf5_hl_f90cstub.so.310.0.1
-rw-r--r-- 1 spack spack   159550 May 17  2023 libhdf5_hl_fortran.a
lrwxrwxrwx 1 spack spack       25 May 17  2023 libhdf5_hl_fortran.so -> libhdf5_hl_fortran.so.310
lrwxrwxrwx 1 spack spack       29 May 17  2023 libhdf5_hl_fortran.so.310 -> libhdf5_hl_fortran.so.310.0.1
-rwxr-xr-x 1 spack spack   148440 May 17  2023 libhdf5_hl_fortran.so.310.0.1
lrwxrwxrwx 1 spack spack       17 May 17  2023 libhdf5_hl.so -> libhdf5_hl.so.310
lrwxrwxrwx 1 spack spack       21 May 17  2023 libhdf5_hl.so.310 -> libhdf5_hl.so.310.0.1
-rwxr-xr-x 1 spack spack   156928 May 17  2023 libhdf5_hl.so.310.0.1
lrwxrwxrwx 1 spack spack       19 May 17  2023 libhdf5_java.so -> libhdf5_java.so.310
lrwxrwxrwx 1 spack spack       23 May 17  2023 libhdf5_java.so.310 -> libhdf5_java.so.310.0.1
-rwxr-xr-x 1 spack spack   638472 May 17  2023 libhdf5_java.so.310.0.1
-rw-r--r-- 1 spack spack     4252 May 17  2023 libhdf5.settings
lrwxrwxrwx 1 spack spack       14 May 17  2023 libhdf5.so -> libhdf5.so.310
lrwxrwxrwx 1 spack spack       18 May 17  2023 libhdf5.so.310 -> libhdf5.so.310.1.0
-rwxr-xr-x 1 spack spack  5093128 May 17  2023 libhdf5.so.310.1.0
-rw-r--r-- 1 spack spack   601550 May 17  2023 libhdf5_tools.a
lrwxrwxrwx 1 spack spack       20 May 17  2023 libhdf5_tools.so -> libhdf5_tools.so.310
lrwxrwxrwx 1 spack spack       24 May 17  2023 libhdf5_tools.so.310 -> libhdf5_tools.so.310.0.1
-rwxr-xr-x 1 spack spack   322792 May 17  2023 libhdf5_tools.so.310.0.1
drwxr-sr-x 2 spack spack      157 May 17  2023 pkgconfig
-rw-r--r-- 1 spack spack    62531 May  2  2023 slf4j-api-2.0.6.jar
-rw-r--r-- 1 spack spack     4071 May  2  2023 slf4j-nop-2.0.6.jar
-rw-r--r-- 1 spack spack    15239 May  2  2023 slf4j-simple-2.0.6.jar

[the-ethan-hunt]

@aminiussi were you able to secure a work-around for this? Facing a similar issue when working on TACC with the module list as:

$ module list
Currently Loaded Modules:
 1) intel/24.0   2) impi/21.11   3) autotools/1.4   4) cmake/3.28.1   5) xalt/3.0.1   6) TACC   7) hdf5/1.14.4

However, upon the cmake command cmake .. -DSERIAL=1, I encounter this error:

Could NOT find HDF5 (missing: HDF5_links) (found suitable version "1.14.3", minimum required is "1.8.7")
CMake Error at build/_deps/h5fortran-src/CMakeLists.txt:29 (message):
  HDF5 not found or working on the system.  First build HDF5 by:

    cmake -S scripts -B scripts/build -DCMAKE_INSTALL_PREFIX=~/mylibs
    cmake --build scripts/build

    Then build h5fortran:

    cmake -B build -DCMAKE_PREFIX_PATH=~/mylibs
    cmake --build build

[milancurcic]

Thanks @the-ethan-hunt. Do you need the HDF5 part for your work? It's used only to read Keras saved models. I'm planning to make it optional at build time.

[milancurcic]

If you don't need HDF5, try building the branch from this PR: https://github.com/modern-fortran/neural-fortran/pull/187

[the-ethan-hunt]

Thanks a lot @milancurcic ! That worked out well!

[milancurcic]

Neural-fortran main branch no longer relies on HDF5. The Keras HDF5 adapter has moved here: https://github.com/neural-fortran/nf-keras-hdf5. I will close this issue assuming it's no longer a problem.