"make install" fails with "undefined symbol: sf_open"

[bugaev]

How to reproduce the error message

cd cmake-build && cmake .. && make install

Started compiling: "libs/external/instruments_ext.xtm"...
Loading xtmaudiobuffer library... done in 0.000000 seconds
Loading xtmaudio_dsp library... done in 0.000000 seconds
Loading xtminstruments library... done in 0.000000 seconds
sys:load notification instruments already loaded 
Bad: /foo/bar/extempore/libs/aot-cache/xtmsndfile.so: undefined symbol: sf_open
Error: could not open  dynamic library

Additional info

patchelf --print-needed ../libs/aot-cache/xtmsndfile.so --> libc.so.6 i.e., libsndfile dependency is missing in the needed section of xtmsndfile.so

uname -a:
Linux foo 4.14.54 #1-NixOS SMP Sun Jul 8 13:30:53 UTC 2018 x86_64 GNU/Linux

make extempore is successful.

[benswift]

Thanks for the report.

I must confess that I've never used (and only have basic knowledge of) NixOS. But a couple of things which might help you figure out what's going wrong. xtmsndfile.so doesn't depend (statically) on libsndfile.so, it's all done at run-time (Extempore really pushes the late-binding thing as far as it'll sensibly go, and perhaps farther :)

The Extempore build process just dumps all the "external" shared libs in the libs/aot-cache/ folder, and bind-dylib checks in there first (a poor man's "prepend to LD_LIBRARY_PATH"). It's all quite filthy in some ways, but it's been (reasonably) reliable across macOS/Win/Linux (at least, Debian and derivatives). Further off this path it's quite likely that some of the hacks won't work.

If we can make things work in a NixOS-y way which doesn't break the other platforms (especially Windows, which is obviously just a whole different world with this stuff) then I'm open to ideas/pull requests.

The one final caveat is that it's a while since I worked on this stuff and without diving back in some of the above might be wrong :( But the above is my recollection about how it works.

[bugaev]

Thank you, @benswift, for the overview of Extempore's inner workings. That helped me in diagnosing and solving the problem. Can you recommend us some more in-depth source of information on what is going under the hood?

Anyways, the problem was on NixOS's side. Invocation of gcc implicitly adds compiler flags for shared libraries "hardening". The one that adds "bind_now" attribute to the library was a culprit.

I will describe the whole build procedure here to save other NixOS users their time:

nix-shell -p cmakeCurses alsaLib libsndfile libGLU_combined xorg.libX11 xorg.libXrandr vulkan-loader xorg.libXinerama xorg.libXcursor zlib gtest
export hardeningDisable=all
git clone https://github.com/digego/extempore && cd extempore &&
git checkout 4a6be4ef813a44d377a47de75bc8782c4f92a864 &&
mkdir cmake-build && cd cmake-build
cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DCMAKE_INSTALL_PREFIX=~/INSTALLED/EXTEMPORE ..
make && make install

Success! Closing the issue.

[digego]

Yep, setting bindnow would definitely cause you some problems :)

On 9 October 2018 10:48:10 am AEST, Viatcheslav Bugaev notifications@github.com wrote:

Thank you, @benswift, for the overview of Extempore's inner workings. That helped me in diagnosing and solving the problem. Can you recommend us some more in-depth source of information on what is going under the hood?

Anyways, the problem was on NixOS's side. Invocation of gcc implicitly adds compiler flags for shared libraries "hardening". The one that adds "bind_now" attribute to the library was a culprit.

I will describe the whole build procedure here to save other NixOS users their time:

nix-shell -p cmakeCurses alsaLib libsndfile libGLU_combined xorg.libX11
xorg.libXrandr vulkan-loader xorg.libXinerama xorg.libXcursor zlib
gtest
export hardeningDisable=all
git clone https://github.com/digego/extempore && cd extempore &&
git checkout 4a6be4ef813a44d377a47de75bc8782c4f92a864 &&
mkdir cmake-build && cd cmake-build
cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=1
-DCMAKE_INSTALL_PREFIX=~/INSTALLED/EXTEMPORE ..
make && make install

Success! Closing the issue.

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/digego/extempore/issues/329#issuecomment-428023577

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[benswift]

No worries, glad it worked out. Yep, it's good to leave this here for the next NixOS user who hits this issue.