dyld: Library not loaded: @rpath/libhdf5.103.dylib

[chemgeeklian]

Because I manually installed armadillo before installing it with brew, after I updating (reinstall) bmdca and run bmdca, it throws

dyld: Library not loaded: @rpath/libhdf5.103.dylib
  Referenced from: /usr/local/lib/libarmadillo.9.87.2.dylib
  Reason: image not found
Abort trap: 6

I tried a few things that don't help:

1. Manually reinstall armadillo from http://arma.sourceforge.net
2. Reinstall armadillo through brew by running brew reinstall armadillo --build-from-source
3. Reinstall bmdca after step 1 and 2.

Finally it turns out to be a version issue of libhdf5. I guess it happens when I upgraded brew it also got upgraded. But bmdca still uses the old version.

So I link the latest version of libhdf5 that I had installed with brew to the old version: sudo ln -s /usr/local/lib/libhdf5.200.dylib /usr/local/lib/libhdf5.103.dylib

(you can check your latest version of libhdf5 by running ls /usr/local/lib/libhdf5*)

And it worked.

I think this issue is rare, only happens when you manually installed armadillo before installing brew. But is it possible for bmDCA to do version check to avoid this error?

Ref:https://github.com/mlpack/mlpack/issues/1799

[sudorook]

I'm guessing that there is a second installation of hdf5 somewhere, maybe /usr/lib. Did you also install a version of hdf5 when you originally manually installed armadillo? You can search your whole filesystem by running: find / -type f -name "libhdf*". I suspect that when compiling armadillo manually, it links against one version, and when using bmDCA, it looks for the other.

It may also help to know which version of Armadillo gets used when compiling bmDCA. What is the output of echo $PKG_CONFIG_PATH and echo $LD_LIBRARY_PATH?

This issue is solely to do with installing Armadillo --- bmDCA doesn't use hdf5. The src/Makefile.am file uses the Armadillo-provided DARMA_DONT_USE_HDF5 flag. (I set it this way because I was having trouble statically linking against hdf5 in msys2 for windows installations.) This may explain why creating a symlink to a different version didn't cause any errors. Doing that won't always work and will likely have to be done again whenever hdf5 is updated.

[chemgeeklian]

OK find / -type f -name "libhdf*" gives

/usr/local/Cellar/hdf5/1.12.0/lib/libhdf5hl_fortran.a
/usr/local/Cellar/hdf5/1.12.0/lib/libhdf5_hl.200.dylib
/usr/local/Cellar/hdf5/1.12.0/lib/libhdf5.a
/usr/local/Cellar/hdf5/1.12.0/lib/libhdf5hl_fortran.200.dylib
/usr/local/Cellar/hdf5/1.12.0/lib/libhdf5_fortran.a
/usr/local/Cellar/hdf5/1.12.0/lib/libhdf5_fortran.200.dylib
/usr/local/Cellar/hdf5/1.12.0/lib/libhdf5_hl.a
/usr/local/Cellar/hdf5/1.12.0/lib/libhdf5_hl_cpp.a
/usr/local/Cellar/hdf5/1.12.0/lib/libhdf5_cpp.a
/usr/local/Cellar/hdf5/1.12.0/lib/libhdf5_hl_cpp.200.dylib
/usr/local/Cellar/hdf5/1.12.0/lib/libhdf5_cpp.200.dylib
/usr/local/Cellar/hdf5/1.12.0/lib/libhdf5.200.dylib

Does this indicate my hdf5 was installed manually?

Both echo $PKG_CONFIG_PATH and echo $LD_LIBRARY_PATH outputs nothing. But I guess the manually installed armadillo will be used in priority. After I uninstalled this armadillo, bmDCA throws error during installation because it "cannot find armadillo", even if I have installed the brew one.

[sudorook]

Looks like hdf5 was installed via homebrew. Homebrew by default installs programs in the '/usr/local/Cellar' directory, and no other program I'm aware of puts files there. I wonder where cmake was finding the 'libhdf5.103.dylib' file...

A more permissive search (looks for anything, not just files):

find / -name "libhdf*"

---

Anyway, yes, the system will default to the manually installed armadillo over the brew-installed one. The reason is that the linker will look for libraries in the /usr/lib and then /usr/local/lib directories. By default, it won't look in '/usr/local/Cellar`, where the homebrew armadillo files are located.

To use the homebrew-installed armadillo, the PKG_CONFIG_PATH and LD_LIBRARY_PATH variables should be updated. Doing this is a bit tedious, so to help with that, I created a file in this repository called 'rcparams' in the tools/ directory. If you copy the contents of the file to your shell rc file and then reload your shell, the PKG_CONFIG_PATH and LD_LIBRARY_PATH environmental variables will be updated. If using bash, edit your ~/.bashrc file, and if using zsh, edit the ~/.zshrc file. Use echo $SHELL if unsure of what shell you're using.

After editing the files, run source ~/.bashrc (or source ~/.zshrc as appropriate) or close and open the terminal. If you echo $PKG_CONFIG_PATH and echo $LD_LIBRARY_PATH, the variables should no longer be empty and instead contain a ':'-delimited list of paths, including a directory for '/usr/local/Cellar/armadillo' (name may be slightly different -- I write from memory).

(If you are using bash and run into an issue where the variables don't get updated when starting a terminal, it could be because the .bashrc isn't being run. I don't know why this happens, but if it does, add the line [ -f $HOME/.bashrc ] && . $HOME/.bashrc to the ~/.bash_profile file and try reloading your shell again.)

Then, recompile and reinstall bmdca. Autotools should be able to find the brew-installed armadillo now.

Using the homebrew armadillo will probably be less hassle in the long run. You won't have to worry about libraries it depends on (hdf5, among others) getting updated in the background and breaking it. You'll only have to deal with updates to armadillo possibly breaking bmdca. Just recompile if that happens.

[chemgeeklian]

I followed your steps, now it works.

While installing bmDCA, I noticed ~/.bashrc was automatically written by contents in your 'rcparams' (maybe not the exact content) , but ~/.zshrc is not. My shell is zsh, and I guess this is the cause of this issue?

[sudorook]

There isn't code that automatically adds stuff to the bashrc file. It has to be edited manually. I suppose I could bundle a script that does something like:

if [[ "$(echo "$SHELL")" = "/bin/zsh" ]]; then
  cat tools/rcparams >> ~/.zshrc
elif [[ "$(echo "$SHELL")" = "/bin/bash" ]]; then
  cat tools/rcparams >> ~/.bashrc
fi

But yeah, if that stuff was in bashrc when it's needed in zshrc, that would explain the issue.