search

aldanor / hdf5-rust

HDF5 for Rust
https://docs.rs/hdf5
Apache License 2.0
310 stars 84 forks source link

Compiling HDF5 on M1 mac 11.7.9 using custom HDF5 libs #256

Closed pavanakumar closed 1 year ago

pavanakumar commented 1 year ago

Using a custom HDF5 installation (v1.10.5) on M1 mac running 11.7.9 gives the following error

  Looking for HDF5 library binary...
  Attempting to load: "/Users/mpkumar/bin/openmpi/hdf5-1.10.5-ompi-4.1.5-native/lib/libhdf5.dylib"

  --- stderr
  dyld: lazy symbol binding failed: Symbol not found: _H5_init_library
    Referenced from: /Users/mpkumar/bin/openmpi/hdf5-1.10.5-ompi-4.1.5-native/lib/libhdf5.dylib
    Expected in: flat namespace

  dyld: Symbol not found: _H5_init_library
    Referenced from: /Users/mpkumar/bin/openmpi/hdf5-1.10.5-ompi-4.1.5-native/lib/libhdf5.dylib
    Expected in: flat namespace

Note that HDF5 was installed using the following configuration

CC=mpicc CXX=mpic++ CFLAGS="-Wno-error=implicit-function-declaration" ./configure --enable-parallel --prefix=/Users/mpkumar/bin/openmpi/hdf5-1.10.5-ompi-4.1.5-native; make -j4 install

I set the HDF5_DIR env var for Rust-hdf5 to detect the library.

Using the homebrew hdf5 1.10 works without problems but I need to use a custom HDF5 installation that is compiled with OpenMPI (v4.1.5).

mulimoen commented 1 year ago

I can't pinpoint what is wrong and I don't have a mac to test it, but I have a hunch it might be arch related. Are you compiling for x86 or arm? Which rust toolchain are you using?

pavanakumar commented 1 year ago

This is an arm build and the library is consistent with the arch. Am using cargo build using the following TOML file,

[package]
name = "hdf_demo"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
hdf5 = "0.8.1"
ndarray = "0.15.6"

And building the code gives the output

$> cargo build
   Updating crates.io index
   Compiling autocfg v1.1.0
   Compiling cfg-if v1.0.0
   Compiling memchr v2.6.3
   Compiling regex-syntax v0.7.5
   Compiling libc v0.2.147
   Compiling proc-macro2 v1.0.66
   Compiling unicode-ident v1.0.11
   Compiling syn v1.0.109
   Compiling libloading v0.7.4
   Compiling parking_lot_core v0.8.6
   Compiling instant v0.1.12
   Compiling scopeguard v1.2.0
   Compiling num-traits v0.2.16
   Compiling matrixmultiply v0.3.7
   Compiling aho-corasick v1.0.5
   Compiling num-integer v0.1.45
   Compiling lock_api v0.4.10
   Compiling smallvec v1.11.0
   Compiling hdf5-types v0.8.1
   Compiling quote v1.0.33
   Compiling paste v1.0.14
   Compiling rawpointer v0.2.1
   Compiling ascii v1.1.0
   Compiling hdf5 v0.8.1
   Compiling parking_lot v0.11.2
   Compiling regex-automata v0.3.8
   Compiling num-complex v0.4.4
   Compiling bitflags v1.3.2
   Compiling lazy_static v1.4.0
   Compiling ndarray v0.15.6
   Compiling hdf5-derive v0.8.1
   Compiling regex v1.9.5
   Compiling hdf5-sys v0.8.1
error: failed to run custom build command for `hdf5-sys v0.8.1`

Caused by:
  process didn't exit successfully: `/Users/mpkumar/AnamikaDSL_sandbox/tapestry/hdf_demo/target/debug/build/hdf5-sys-8e792d70e44ed19e/build-script-build` (signal: 6, SIGABRT: process abort signal)
  --- stdout
  Setting HDF5 root from environment variable:
      HDF5_DIR = "/Users/mpkumar/bin/openmpi/hdf5-1.10.10-ompi-4.1.5-native"
  Custom HDF5_DIR provided; rpath can be set via:
      RUSTFLAGS="-C link-args=-Wl,-rpath,/Users/mpkumar/bin/openmpi/hdf5-1.10.10-ompi-4.1.5-native/lib"
  On some OS X installations, you may also need to set:
      DYLD_FALLBACK_LIBRARY_PATH="/Users/mpkumar/bin/openmpi/hdf5-1.10.10-ompi-4.1.5-native/lib"
  Parsing HDF5 config from:
      "/Users/mpkumar/bin/openmpi/hdf5-1.10.10-ompi-4.1.5-native/include/H5pubconf.h"
  Looking for HDF5 library binary...
  Attempting to load: "/Users/mpkumar/bin/openmpi/hdf5-1.10.10-ompi-4.1.5-native/lib/libhdf5.dylib"

  --- stderr
  dyld: lazy symbol binding failed: Symbol not found: _H5_init_library
    Referenced from: /Users/mpkumar/bin/openmpi/hdf5-1.10.10-ompi-4.1.5-native/lib/libhdf5.dylib
    Expected in: flat namespace

  dyld: Symbol not found: _H5_init_library
    Referenced from: /Users/mpkumar/bin/openmpi/hdf5-1.10.10-ompi-4.1.5-native/lib/libhdf5.dylib
    Expected in: flat namespace

warning: build failed, waiting for other jobs to finish...
pavanakumar commented 1 year ago

Ok I found a solution to this problem. After downloading the HDF5 sources we must do the following to the extracted sources,

$> autoreconf --force --install --verbose
...
$> ./autogen.sh

In addition use the following flags while configuring the source,

$> ./configure .... --disable-dependency-tracking --disable-silent-rules --prefix=...

After compiling and installing the HDF5 library I can now successfully run the Rust code. Hope someone with a similar problem finds this useful.