HDF5 version/linking problem?

[fnollet]

Hi

Trying to compile on RHEL 6 64 bit. I am using a more recent compiler (gcc 4.9.4) and have been helped a lot already with https://github.com/nanoporetech/scrappie/issues/5 But I still seem to have issues with hdf5.

I have hdf5 1.8.16 installed in a dedicated path and added this to the CMakeLists.txt file:

include_directories("/tools/bioinfo/app/atlas-3.10.2/include")
include_directories("/tools/bioinfo/app/hdf5-1.8.16/include")
link_directories("/tools/bioinfo/app/atlas-3.10.2/lib")
link_directories("/tools/bioinfo/app/hdf5-1.8.16/lib")

But I still seem to have compile issues:

CMakeFiles/scrappie.dir/src/scrappie_raw.c.o: In function `main_raw':
scrappie_raw.c:(.text+0xcd9): undefined reference to `H5check_version'
scrappie_raw.c:(.text+0xcec): undefined reference to `H5Fopen'
scrappie_raw.c:(.text+0xd5e): undefined reference to `H5Fclose'
scrappie_raw.c:(.text+0xdb3): undefined reference to `H5check_version'
scrappie_raw.c:(.text+0xdc8): undefined reference to `H5Fcreate'
CMakeFiles/scrappie.dir/src/scrappie_events.c.o: In function `main_events':
scrappie_events.c:(.text+0xd69): undefined reference to `H5check_version'
scrappie_events.c:(.text+0xd7c): undefined reference to `H5Fopen'
scrappie_events.c:(.text+0xdee): undefined reference to `H5Fclose'
scrappie_events.c:(.text+0xe43): undefined reference to `H5check_version'
scrappie_events.c:(.text+0xe58): undefined reference to `H5Fcreate'
CMakeFiles/scrappie.dir/src/fast5_interface.c.o: In function `read_float_attribute':
fast5_interface.c:(.text+0x28): undefined reference to `H5Aopen'
fast5_interface.c:(.text+0x33): undefined reference to `H5open'
fast5_interface.c:(.text+0x39): undefined reference to `H5T_NATIVE_FLOAT_g'
fast5_interface.c:(.text+0x45): undefined reference to `H5Aread'
fast5_interface.c:(.text+0x4c): undefined reference to `H5Aclose'
CMakeFiles/scrappie.dir/src/fast5_interface.c.o: In function `get_raw_scaling':
fast5_interface.c:(.text+0xd4): undefined reference to `H5Gopen2'
fast5_interface.c:(.text+0xf4): undefined reference to `H5Aopen'
fast5_interface.c:(.text+0x103): undefined reference to `H5open'
fast5_interface.c:(.text+0x109): undefined reference to `H5T_NATIVE_FLOAT_g'
fast5_interface.c:(.text+0x115): undefined reference to `H5Aread'
fast5_interface.c:(.text+0x11c): undefined reference to `H5Aclose'
fast5_interface.c:(.text+0x13d): undefined reference to `H5Aopen'
fast5_interface.c:(.text+0x14c): undefined reference to `H5open'
fast5_interface.c:(.text+0x152): undefined reference to `H5T_NATIVE_FLOAT_g'
fast5_interface.c:(.text+0x15e): undefined reference to `H5Aread'
fast5_interface.c:(.text+0x165): undefined reference to `H5Aclose'
fast5_interface.c:(.text+0x186): undefined reference to `H5Aopen'
fast5_interface.c:(.text+0x195): undefined reference to `H5open'
fast5_interface.c:(.text+0x19b): undefined reference to `H5T_NATIVE_FLOAT_g'
fast5_interface.c:(.text+0x1a7): undefined reference to `H5Aread'
fast5_interface.c:(.text+0x1ae): undefined reference to `H5Aclose'
fast5_interface.c:(.text+0x1cf): undefined reference to `H5Aopen'
fast5_interface.c:(.text+0x1de): undefined reference to `H5open'
fast5_interface.c:(.text+0x1e4): undefined reference to `H5T_NATIVE_FLOAT_g'
fast5_interface.c:(.text+0x1f0): undefined reference to `H5Aread'
fast5_interface.c:(.text+0x1f7): undefined reference to `H5Aclose'
fast5_interface.c:(.text+0x208): undefined reference to `H5Gclose'
CMakeFiles/scrappie.dir/src/fast5_interface.c.o: In function `read_raw':
fast5_interface.c:(.text+0x368): undefined reference to `H5check_version'
fast5_interface.c:(.text+0x374): undefined reference to `H5Fopen'
fast5_interface.c:(.text+0x389): undefined reference to `H5Eset_auto2'
fast5_interface.c:(.text+0x3a3): undefined reference to `H5Lget_name_by_idx'
fast5_interface.c:(.text+0x3e3): undefined reference to `H5Lget_name_by_idx'
fast5_interface.c:(.text+0x421): undefined reference to `H5Dopen2'
fast5_interface.c:(.text+0x437): undefined reference to `H5Dget_space'
fast5_interface.c:(.text+0x451): undefined reference to `H5Sget_simple_extent_dims'
fast5_interface.c:(.text+0x468): undefined reference to `H5open'
fast5_interface.c:(.text+0x46e): undefined reference to `H5T_NATIVE_FLOAT_g'
fast5_interface.c:(.text+0x480): undefined reference to `H5Dread'
fast5_interface.c:(.text+0x4a5): undefined reference to `H5Sclose'
fast5_interface.c:(.text+0x4ad): undefined reference to `H5Dclose'
fast5_interface.c:(.text+0x4bc): undefined reference to `H5Fclose'
CMakeFiles/scrappie.dir/src/fast5_interface.c.o: In function `write_annotated_events':
fast5_interface.c:(.text+0x83c): undefined reference to `H5Tcreate'
fast5_interface.c:(.text+0x84b): undefined reference to `H5open'
fast5_interface.c:(.text+0x851): undefined reference to `H5T_NATIVE_UINT64_g'
fast5_interface.c:(.text+0x85f): undefined reference to `H5Tinsert'
fast5_interface.c:(.text+0x864): undefined reference to `H5open'
fast5_interface.c:(.text+0x86a): undefined reference to `H5T_NATIVE_FLOAT_g'
fast5_interface.c:(.text+0x87b): undefined reference to `H5Tinsert'
fast5_interface.c:(.text+0x880): undefined reference to `H5open'
fast5_interface.c:(.text+0x886): undefined reference to `H5T_NATIVE_FLOAT_g'
fast5_interface.c:(.text+0x897): undefined reference to `H5Tinsert'
fast5_interface.c:(.text+0x89c): undefined reference to `H5open'
fast5_interface.c:(.text+0x8a2): undefined reference to `H5T_NATIVE_FLOAT_g'
fast5_interface.c:(.text+0x8b3): undefined reference to `H5Tinsert'
fast5_interface.c:(.text+0x8b8): undefined reference to `H5open'
fast5_interface.c:(.text+0x8be): undefined reference to `H5T_NATIVE_INT_g'
fast5_interface.c:(.text+0x8cf): undefined reference to `H5Tinsert'
fast5_interface.c:(.text+0x8de): undefined reference to `H5Tcreate'
fast5_interface.c:(.text+0x8ed): undefined reference to `H5open'
fast5_interface.c:(.text+0x8f3): undefined reference to `H5T_STD_U64LE_g'
fast5_interface.c:(.text+0x901): undefined reference to `H5Tinsert'
fast5_interface.c:(.text+0x906): undefined reference to `H5open'
fast5_interface.c:(.text+0x90c): undefined reference to `H5T_IEEE_F32LE_g'
fast5_interface.c:(.text+0x91d): undefined reference to `H5Tinsert'
fast5_interface.c:(.text+0x922): undefined reference to `H5open'
fast5_interface.c:(.text+0x928): undefined reference to `H5T_IEEE_F32LE_g'
fast5_interface.c:(.text+0x939): undefined reference to `H5Tinsert'
fast5_interface.c:(.text+0x93e): undefined reference to `H5open'
fast5_interface.c:(.text+0x944): undefined reference to `H5T_IEEE_F32LE_g'
fast5_interface.c:(.text+0x955): undefined reference to `H5Tinsert'
fast5_interface.c:(.text+0x95a): undefined reference to `H5open'
fast5_interface.c:(.text+0x960): undefined reference to `H5T_STD_I32LE_g'
fast5_interface.c:(.text+0x971): undefined reference to `H5Tinsert'
fast5_interface.c:(.text+0x98c): undefined reference to `H5Screate_simple'
fast5_interface.c:(.text+0x9a4): undefined reference to `H5open'
fast5_interface.c:(.text+0x9aa): undefined reference to `H5P_CLS_DATASET_CREATE_ID_g'
fast5_interface.c:(.text+0x9af): undefined reference to `H5Pcreate'
fast5_interface.c:(.text+0x9c1): undefined reference to `H5Pset_shuffle'
fast5_interface.c:(.text+0x9cc): undefined reference to `H5Pset_deflate'
fast5_interface.c:(.text+0x9de): undefined reference to `H5Pset_chunk'
fast5_interface.c:(.text+0x9fb): undefined reference to `H5Dcreate2'
fast5_interface.c:(.text+0xa1f): undefined reference to `H5Dwrite'
fast5_interface.c:(.text+0xa34): undefined reference to `H5Dclose'
fast5_interface.c:(.text+0xa3c): undefined reference to `H5Sclose'
fast5_interface.c:(.text+0xa43): undefined reference to `H5Tclose'
fast5_interface.c:(.text+0xa4a): undefined reference to `H5Tclose'
fast5_interface.c:(.text+0xaaf): undefined reference to `H5Pclose'
collect2: error: ld returned 1 exit status
make[2]: *** [scrappie] Error 1
make[1]: *** [CMakeFiles/scrappie.dir/all] Error 2
make: *** [all] Error 2

To I need to do something else? Is my version of hdf5 ok?

Thanks for you help.

Filip Nollet

[tmassingham-ont]

Hello. It looks like it is not linking with the HDF5 library; the name of library to link against is determined by the block https://github.com/nanoporetech/scrappie/blob/master/CMakeLists.txt#L111 Try commenting this out and explicitly adding

set(HDF5 libname)

where libname might be 'hdf5' if your /tools/bioinfo/app/hdf5-1.8.16/lib directory contains a libhdf5.so file, similar trimming for other libraries (e.g. libhdf5_serial.so -> hdf5_serial).

N.B. we have encountered issues with the HDF5 library on Redhat platforms not being compiled for multithreaded use. If errors are encountered reading from the fast5 files, running Scrappie in single-threaded mode is recommended (see https://github.com/nanoporetech/scrappie/blob/master/README.md#running)

[fnollet]

Hi

So your trick helped partially. I also had to fix this line:

check_include_file ("hdf5.h" HDF5_STANDARD)

to

check_include_file ("/tools/bioinfo/app/hdf5-1.8.16/include/hdf5.h" HDF5_STANDARD)

Also for RHEL 6 users still out there building this from scratch: CUnit is not available (from RedHat repo's) as a precompiled package so I compiled a version myself. This article helped a lot: http://blog.cleverelephant.ca/2014/12/building-cunit-from-source.html Main tip here was to use version 2.1-2 (not 2.1-3 as is has a missing file). Also requires a more recently version of automake/autoconf (1.14.1 & 2.69) and gcc (4.9.4) and some voodoo with several files there.

Also had to add then the lib/include dirs to the CMakeLists.txt: e.g. (in our case)

include_directories("/tools/bioinfo/app/CUnit-2.1-2/include")
link_directories("/tools/bioinfo/app/CUnit-2.1-2/lib")

This made scrappie compile. I will ask my user to verify multithreading. Thanks for tip anyway and all the help!!!

[tmassingham-ont]

Glad it helped. Many thanks for the additional info -- I'll see if I can improve the documentation and CMakeLists.txt file.