Closed Kuxe closed 2 years ago
Thanks for reporting.
I tried to reproduce your findings and found no unexpected behavior.
It seems like you did a local install of glbinding in the directory ../install
, which should result in a directory structure that contains all executables (e.g., the examples and the tools) in the install directory, and both libraries of glbinding in the lib
subdirectory. With this setup in mind, CMake correctly added an RPATH of $ORIGIN/lib
.
However, what I expect you wanted to achieve is a directory structure where executables and libraries are all placed in the install directory directly, where an RPATH of $ORIGIN
would be appropriate.
Both approaches are feasible and currently our CMake setup is configured for the former approach. If you want to install your software and glbinding using the latter approach, feel free to adjust the CMake variables in your copy of glbinding (see https://github.com/cginternals/glbinding/blob/master/CMakeLists.txt#L181 ff). However, I think we're happy with our setup so far, so I don't think we need to adjust the directories here in this repository.
If, for whatever reason, there is an actual mismatch of the configured install location (should be lib
) and the RPATH (should be $ORIGIN/lib
), this may an actual bug and we may want to fix this.
Hi, thanks for the explanation. I was writing a follow up simultaneously and here it is:
I'm think that this is actually not a glbinding-issue due to this SO-post, although that post is about rpath
and not runpath
. Nevertheless, the only difference between rpath
and runpath
is their relation to LD_LIBRARY_PATH
so there is no reason so suspect that the SO-post does not hold for runpath
too.
So in summary it is fine that
ldd libglbinding-aux.so
reports
[...]
libglbinding.so.3 => not found
[...]
The important thing is that it is resolved when my executable executes then libglbinding-aux.so.3
should resolve libglbinding.so.3
- which it does from parents runpath according to LD_DEBUG=libs
I am dabbling in these matters and wrongly thought of it as a glbinding problem but now I'm leaning towards that this is a CMake issue with RUNTIME_DEPENDENCY_SET
. Sorry!
Should anyone share my confusion, then you might be interested in the issue I created over at Kitware's gitlab for CMake.
Closing.
I previously asserted this:
Nevertheless, the only difference between rpath and runpath is their relation to LD_LIBRARY_PATH so there is no reason so suspect that the SO-post does not hold for runpath too.
This is incorrect and I'd like to continue discussing my problem, reopening this issue.
Using the directories specified in the DT_RUNPATH dynamic section attribute of the binary if present. Such directories are searched only to find those objects required by DT_NEEDED (direct dependencies) entries and do not apply to those objects' children, which must themselves have their own DT_RUNPATH entries. This is unlike DT_RPATH, which is applied to searches for all children in the dependency tree.
I have the following directory after running cmake --build . --target install
<CMAKE_INSTALL_PREFIX>
└── bin
├── libglbinding-aux.so.3
├── libglbinding.so.3
└── myexecutable
where in my setup glbinding is an IMPORTED library (via the conan package-manager, using generator CMakeDeps
. This is why I am reluctant to modifying my instance of glbinding, since I'd rather just download a precompiled binary and avoid building it from source). Since the libraries are IMPORTED they are searched for when using RUNTIME_DEPENDENCY_SET
.
CMake does this (I consulted CMake developers, see https://gitlab.kitware.com/cmake/cmake/-/issues/23103, and my stance is that CMake is not at fault here):
myexecutable
and discovers that libglbinding-aux.so.3
is a NEEDED dependency.myexecutable
for the NEEDED dependency libglbinding-aux.so.3
libglbinding-aux.so.3
since it indeed is available at RUNPATH of myexecutable
. All good so far!libglbinding-aux.so.3
and discovers that libglbinding.so.3
is a NEEDED dependency.libglbinding-aux.so.3
for the NEEDED dependency libglbinding.so.3
.libglbinding.so.3
of libglbinding-aux.so.3
since no such file exists in $ORIGIN/lib
libglbinding.so.3
as an UNRESOLVED dependency and the install-step fails.Correct me if I am wrong, but I believe this problem would be observed in all of the following (common) cases:
// CASE lib
.
├── lib
│ ├── libglbinding.so.3
│ └── libglbinding-aux.so.3
└── myexecutable
// CASE bin+lib
.
├── bin
│ └── myexecutable
└── lib
├── libglbinding.so.3
└── libglbinding-aux.so.3
// CASE bin (my use-case!)
bin
├── libglbinding.so.3
├── libglbinding-aux.so.3
└── myexecutable
With all this in mind, I think that
Both approaches are feasible
is false. My conclusion is that $ORIGIN/lib
approach is not feasible and only $ORIGIN
is feasible. To me it currently seems that the only conan-compatible solution (that does not rely on some manual patching via patchelf
) is an upstream-change in https://github.com/cginternals/glbinding where glbinding-aux.so.3
- and glbinding.so.3
-RUNPATHs are changed from $ORIGIN/lib
to $ORIGIN
. What do you think?
I posted a more generic question on stackoverflow that directly relates to this discussion, to benefit other people that share the same confusion as I do.
Thanks for the additional input, I tried to wrap my head around all of this.
What I took away:
There are two settings to find dynamic dependencies: the RPATH
and the RUNPATH
, both for the same overall goal but with slightly changed behavior. The main difference is, that the RPATH
is inherited by dependencies, the RUNPATH
is not. The overall order of processing and the mutual exclusivity don't seem to matter for this issue.
What surprised me most is that we only configure RPATH
s and don't configure CMake to use RUNPATH
tags (no --enable-new-dtags
), but the paths still end up in the RUNPATH
.
With this in mind, it may be totally possible we may have ignored the RUNPATH
behavior as we wrote our setup back in 2011-2013. On the other hand, I find clues we may have searched for a setup that should work with using either RPATH
or RUNPATH
- so lets assume this for the rest of my post.
You proposed three cases for an executable and libraries setup:
RPATH
to $ORIGIN/lib
. To confirm your proposal, I, too, expect $ORIGIN
is the needed RPATH
and RUNPATH
for your use case.Let's further assume two different setups I want to go through:
glbinding
with an RPATH
of $ORIGIN/lib
glbinding
with an RPATH
of $ORIGIN
When using an RPATH
of $ORIGIN/lib
we assume the CASE lib to be the install structure:
// CASE lib
.
├── lib
│ ├── libglbinding.so.3
│ └── libglbinding-aux.so.3
└── myexecutable
What we further expect is, that all executables and libraries are build with an RPATH
of $ORIGIN/lib
, including myexecutable
, which will hold if a downstream project is set up using cmake-init as well.
Using this setup, myexecutable
will search for libglbinding-aux.so.3
and finds it in $ORIGIN/lib
, then sees another dependency libglbinding.so.3
. Depending on whether RPATH
or RUNPATH
is actually use, the linker will use the RPATH
of myexecutable
or the RUNPATH
of libglbinding-aux.so.3
and find a value of $ORIGIN/lib
either way. What we assume is, that $ORIGIN
is resolved to be the directory of myexecutable
in both cases! From the docs I read This expands to the directory containing the program or shared object.
, so maybe we have an error here. On the other hand, we had no perceivable errors on our end for any downstream dependency and (local) deployment so far.
Second case: When using an RPATH
of $ORIGIN
we assume the CASE bin to be the install structure:
// CASE bin
.
├── libglbinding.so.3
├── libglbinding-aux.so.3
└── myexecutable
What we similarly then expect is, that all executables and libraries are build with an RPATH
of $ORIGIN
. This setup can resolve all dependencies easily, disregarding the choice of RPATH
or RUNPATH
.
I conclude, that an RPATH
setting of $ORIGIN
would result in a simpler setup with more external use cases in mind. On the other hand, this would clutter up the directory of the executables.
Another solution may be the use of different RPATH
s depending on whether we link an executable or a library, i.e., depending on the install directory.
For reference, I add my linker search protocol for a local install with an RPATH
setting of $ORIGIN/lib
that actually is stored in the RUNPATH
of the executable and libraries.
scheibel@scheibel-ThinkPad-W550s:~/dev/glbinding/glbinding-install$ LD_DEBUG=libs ./cubescape
106539: find library=libglfw.so.3 [0]; searching
106539: search path=/home/scheibel/dev/glbinding/glbinding-install/lib/tls/haswell/x86_64:/home/scheibel/dev/glbinding/glbinding-install/lib/tls/haswell:/home/scheibel/dev/glbinding/glbinding-install/lib/tls/x86_64:/home/scheibel/dev/glbinding/glbinding-install/lib/tls:/home/scheibel/dev/glbinding/glbinding-install/lib/haswell/x86_64:/home/scheibel/dev/glbinding/glbinding-install/lib/haswell:/home/scheibel/dev/glbinding/glbinding-install/lib/x86_64:/home/scheibel/dev/glbinding/glbinding-install/lib (RUNPATH from file ./cubescape)
106539: trying file=/home/scheibel/dev/glbinding/glbinding-install/lib/tls/haswell/x86_64/libglfw.so.3
106539: trying file=/home/scheibel/dev/glbinding/glbinding-install/lib/tls/haswell/libglfw.so.3
106539: trying file=/home/scheibel/dev/glbinding/glbinding-install/lib/tls/x86_64/libglfw.so.3
106539: trying file=/home/scheibel/dev/glbinding/glbinding-install/lib/tls/libglfw.so.3
106539: trying file=/home/scheibel/dev/glbinding/glbinding-install/lib/haswell/x86_64/libglfw.so.3
106539: trying file=/home/scheibel/dev/glbinding/glbinding-install/lib/haswell/libglfw.so.3
106539: trying file=/home/scheibel/dev/glbinding/glbinding-install/lib/x86_64/libglfw.so.3
106539: trying file=/home/scheibel/dev/glbinding/glbinding-install/lib/libglfw.so.3
106539:
106539: find library=libglbinding-aux.so.3 [0]; searching
106539: search path=/home/scheibel/dev/glbinding/glbinding-install/lib (RUNPATH from file ./cubescape)
106539: trying file=/home/scheibel/dev/glbinding/glbinding-install/lib/libglbinding-aux.so.3
106539:
106539: find library=libcpplocate.so.2 [0]; searching
106539: search path=/home/scheibel/dev/glbinding/glbinding-install/lib (RUNPATH from file ./cubescape)
106539: trying file=/home/scheibel/dev/glbinding/glbinding-install/lib/libcpplocate.so.2
106539:
106539: find library=libglbinding.so.3 [0]; searching
106539: search path=/home/scheibel/dev/glbinding/glbinding-install/lib (RUNPATH from file ./cubescape)
106539: trying file=/home/scheibel/dev/glbinding/glbinding-install/lib/libglbinding.so.3
106539:
106539: find library=libstdc++.so.6 [0]; searching
106539: search path=/home/scheibel/dev/glbinding/glbinding-install/lib (RUNPATH from file ./cubescape)
106539: trying file=/home/scheibel/dev/glbinding/glbinding-install/lib/libstdc++.so.6
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libstdc++.so.6
106539:
106539: find library=libm.so.6 [0]; searching
106539: search path=/home/scheibel/dev/glbinding/glbinding-install/lib (RUNPATH from file ./cubescape)
106539: trying file=/home/scheibel/dev/glbinding/glbinding-install/lib/libm.so.6
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libm.so.6
106539:
106539: find library=libgcc_s.so.1 [0]; searching
106539: search path=/home/scheibel/dev/glbinding/glbinding-install/lib (RUNPATH from file ./cubescape)
106539: trying file=/home/scheibel/dev/glbinding/glbinding-install/lib/libgcc_s.so.1
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libgcc_s.so.1
106539:
106539: find library=libc.so.6 [0]; searching
106539: search path=/home/scheibel/dev/glbinding/glbinding-install/lib (RUNPATH from file ./cubescape)
106539: trying file=/home/scheibel/dev/glbinding/glbinding-install/lib/libc.so.6
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libc.so.6
106539:
106539: find library=libdl.so.2 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libdl.so.2
106539:
106539: find library=libX11.so.6 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libX11.so.6
106539:
106539: find library=libpthread.so.0 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libpthread.so.0
106539:
106539: find library=libxcb.so.1 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libxcb.so.1
106539:
106539: find library=libXau.so.6 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libXau.so.6
106539:
106539: find library=libXdmcp.so.6 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libXdmcp.so.6
106539:
106539: find library=libbsd.so.0 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libbsd.so.0
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libpthread.so.0
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libc.so.6
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libbsd.so.0
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libXdmcp.so.6
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libXau.so.6
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libxcb.so.1
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libdl.so.2
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libX11.so.6
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libgcc_s.so.1
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libm.so.6
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libstdc++.so.6
106539:
106539:
106539: calling init: /home/scheibel/dev/glbinding/glbinding-install/lib/libglbinding.so.3
106539:
106539:
106539: calling init: /home/scheibel/dev/glbinding/glbinding-install/lib/libcpplocate.so.2
106539:
106539:
106539: calling init: /home/scheibel/dev/glbinding/glbinding-install/lib/libglbinding-aux.so.3
106539:
106539:
106539: calling init: /home/scheibel/dev/glbinding/glbinding-install/lib/libglfw.so.3
106539:
106539:
106539: initialize program: ./cubescape
106539:
106539:
106539: transferring control: ./cubescape
106539:
106539: find library=libXxf86vm.so.1 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libXxf86vm.so.1
106539:
106539: find library=libXext.so.6 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libXext.so.6
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libXext.so.6
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libXxf86vm.so.1
106539:
106539: find library=libXi.so.6 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libXi.so.6
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libXi.so.6
106539:
106539: find library=libXrandr.so.2 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libXrandr.so.2
106539:
106539: find library=libXrender.so.1 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libXrender.so.1
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libXrender.so.1
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libXrandr.so.2
106539:
106539: find library=libXcursor.so.1 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libXcursor.so.1
106539:
106539: find library=libXfixes.so.3 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libXfixes.so.3
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libXfixes.so.3
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libXcursor.so.1
106539:
106539: find library=libXinerama.so.1 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libXinerama.so.1
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libXinerama.so.1
106539:
106539: find library=libX11-xcb.so.1 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libX11-xcb.so.1
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libX11-xcb.so.1
106539:
106539: find library=libGL.so.1 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libGL.so.1
106539:
106539: find library=libGLdispatch.so.0 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libGLdispatch.so.0
106539:
106539: find library=libGLX.so.0 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libGLX.so.0
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libGLdispatch.so.0
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libGLX.so.0
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libGL.so.1
106539:
106539: find library=libGLX_nvidia.so.0 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libGLX_nvidia.so.0
106539:
106539: find library=libnvidia-tls.so.460.91.03 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libnvidia-tls.so.460.91.03
106539:
106539: find library=libnvidia-glcore.so.460.91.03 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libnvidia-glcore.so.460.91.03
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libnvidia-tls.so.460.91.03
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libnvidia-glcore.so.460.91.03
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libGLX_nvidia.so.0
106539:
106539: /lib/x86_64-linux-gnu/libnvidia-glcore.so.460.91.03: error: symbol lookup error: undefined symbol: ErrorF (fatal)
106539: find library=libxcb-glx.so.0 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libxcb-glx.so.0
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libxcb-glx.so.0
106539:
106539: find library=libdrm.so.2 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libdrm.so.2
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libdrm.so.2
106539:
106539: find library=liballocator.so.0 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: search path=/lib/x86_64-linux-gnu/tls/haswell/x86_64:/lib/x86_64-linux-gnu/tls/haswell:/lib/x86_64-linux-gnu/tls/x86_64:/lib/x86_64-linux-gnu/tls:/lib/x86_64-linux-gnu/haswell/x86_64:/lib/x86_64-linux-gnu/haswell:/lib/x86_64-linux-gnu/x86_64:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu/tls/haswell/x86_64:/usr/lib/x86_64-linux-gnu/tls/haswell:/usr/lib/x86_64-linux-gnu/tls/x86_64:/usr/lib/x86_64-linux-gnu/tls:/usr/lib/x86_64-linux-gnu/haswell/x86_64:/usr/lib/x86_64-linux-gnu/haswell:/usr/lib/x86_64-linux-gnu/x86_64:/usr/lib/x86_64-linux-gnu:/lib/tls/haswell/x86_64:/lib/tls/haswell:/lib/tls/x86_64:/lib/tls:/lib/haswell/x86_64:/lib/haswell:/lib/x86_64:/lib:/usr/lib/tls/haswell/x86_64:/usr/lib/tls/haswell:/usr/lib/tls/x86_64:/usr/lib/tls:/usr/lib/haswell/x86_64:/usr/lib/haswell:/usr/lib/x86_64:/usr/lib (system search path)
106539: trying file=/lib/x86_64-linux-gnu/tls/haswell/x86_64/liballocator.so.0
106539: trying file=/lib/x86_64-linux-gnu/tls/haswell/liballocator.so.0
106539: trying file=/lib/x86_64-linux-gnu/tls/x86_64/liballocator.so.0
106539: trying file=/lib/x86_64-linux-gnu/tls/liballocator.so.0
106539: trying file=/lib/x86_64-linux-gnu/haswell/x86_64/liballocator.so.0
106539: trying file=/lib/x86_64-linux-gnu/haswell/liballocator.so.0
106539: trying file=/lib/x86_64-linux-gnu/x86_64/liballocator.so.0
106539: trying file=/lib/x86_64-linux-gnu/liballocator.so.0
106539: trying file=/usr/lib/x86_64-linux-gnu/tls/haswell/x86_64/liballocator.so.0
106539: trying file=/usr/lib/x86_64-linux-gnu/tls/haswell/liballocator.so.0
106539: trying file=/usr/lib/x86_64-linux-gnu/tls/x86_64/liballocator.so.0
106539: trying file=/usr/lib/x86_64-linux-gnu/tls/liballocator.so.0
106539: trying file=/usr/lib/x86_64-linux-gnu/haswell/x86_64/liballocator.so.0
106539: trying file=/usr/lib/x86_64-linux-gnu/haswell/liballocator.so.0
106539: trying file=/usr/lib/x86_64-linux-gnu/x86_64/liballocator.so.0
106539: trying file=/usr/lib/x86_64-linux-gnu/liballocator.so.0
106539: trying file=/lib/tls/haswell/x86_64/liballocator.so.0
106539: trying file=/lib/tls/haswell/liballocator.so.0
106539: trying file=/lib/tls/x86_64/liballocator.so.0
106539: trying file=/lib/tls/liballocator.so.0
106539: trying file=/lib/haswell/x86_64/liballocator.so.0
106539: trying file=/lib/haswell/liballocator.so.0
106539: trying file=/lib/x86_64/liballocator.so.0
106539: trying file=/lib/liballocator.so.0
106539: trying file=/usr/lib/tls/haswell/x86_64/liballocator.so.0
106539: trying file=/usr/lib/tls/haswell/liballocator.so.0
106539: trying file=/usr/lib/tls/x86_64/liballocator.so.0
106539: trying file=/usr/lib/tls/liballocator.so.0
106539: trying file=/usr/lib/haswell/x86_64/liballocator.so.0
106539: trying file=/usr/lib/haswell/liballocator.so.0
106539: trying file=/usr/lib/x86_64/liballocator.so.0
106539: trying file=/usr/lib/liballocator.so.0
106539:
106539: find library=libxcb-dri3.so.0 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libxcb-dri3.so.0
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libxcb-dri3.so.0
106539:
106539: find library=libxcb-present.so.0 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libxcb-present.so.0
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libxcb-present.so.0
106539:
106539: find library=libxcb-sync.so.1 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: trying file=/lib/x86_64-linux-gnu/libxcb-sync.so.1
106539:
106539:
106539: calling init: /lib/x86_64-linux-gnu/libxcb-sync.so.1
106539:
106539: find library=liballocator.so.0 [0]; searching
106539: search cache=/etc/ld.so.cache
106539: search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib (system search path)
106539: trying file=/lib/x86_64-linux-gnu/liballocator.so.0
106539: trying file=/usr/lib/x86_64-linux-gnu/liballocator.so.0
106539: trying file=/lib/liballocator.so.0
106539: trying file=/usr/lib/liballocator.so.0
106539:
OpenGL Version: 4.6
OpenGL Vendor: NVIDIA Corporation
OpenGL Renderer: Quadro K620M/PCIe/SSE2
Press i or d to either increase or decrease number of cubes.
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libXcursor.so.1 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libXfixes.so.3 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libXrandr.so.2 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libXinerama.so.1 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libXrender.so.1 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libXxf86vm.so.1 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libXi.so.6 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libGL.so.1 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libGLX.so.0 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libGLdispatch.so.0 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libGLX_nvidia.so.0 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libXext.so.6 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libnvidia-glcore.so.460.91.03 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libX11-xcb.so.1 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libdrm.so.2 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libxcb-dri3.so.0 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libxcb-present.so.0 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libxcb-sync.so.1 [0]
106539:
106539:
106539: calling fini: ./cubescape [0]
106539:
106539:
106539: calling fini: /home/scheibel/dev/glbinding/glbinding-install/lib/libglfw.so.3 [0]
106539:
106539:
106539: calling fini: /home/scheibel/dev/glbinding/glbinding-install/lib/libglbinding-aux.so.3 [0]
106539:
106539:
106539: calling fini: /home/scheibel/dev/glbinding/glbinding-install/lib/libcpplocate.so.2 [0]
106539:
106539:
106539: calling fini: /home/scheibel/dev/glbinding/glbinding-install/lib/libglbinding.so.3 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libstdc++.so.6 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libm.so.6 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libgcc_s.so.1 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libX11.so.6 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libdl.so.2 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libpthread.so.0 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libnvidia-tls.so.460.91.03 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libxcb-glx.so.0 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libxcb.so.1 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libXdmcp.so.6 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libbsd.so.0 [0]
106539:
106539:
106539: calling fini: /lib/x86_64-linux-gnu/libXau.so.6 [0]
106539:
scheibel@scheibel-ThinkPad-W550s:~/dev/glbinding/glbinding-install$
What I took away: There are two settings to find dynamic dependencies: the RPATH and the RUNPATH, both for the same overall goal but with slightly changed behavior. The main difference is, that the RPATH is inherited by dependencies, the RUNPATH is not. The overall order of processing and the mutual exclusivity don't seem to matter for this issue.
Yes, this is my understanding too
What surprised me most is that we only configure RPATHs and don't configure CMake to use RUNPATH tags (no --enable-new-dtags), but the paths still end up in the RUNPATH.
I have at some point in time read that you don't always need to pass --enable-new-dtags
since it is the default on newer linkers, but I am unable to locate any source for this statement.
From the docs I read
This expands to the directory containing the program or shared object.
, so maybe we have an error here.
I think this is the error. When the shared object looks for its NEEDED dependencies, then $ORIGIN
resolves to the directory containing shared object. Even if the shared object is loaded by an executable, $ORIGIN
will be the directory of the shared object when the shared object looks for its NEEDED dependencies. This means that if ./libglbinding-aux.so
has a RUNPATH $ORIGIN/lib
, then ./libglbinding-aux.so
expects to find it's dependency libglbinding.so
at ./lib/libglbinding.so
. This is why ldd ./libglbinding.so
gives
libglbinding.so.3 => not found
You should be able to observe this output from ldd
too. I think this is something that all downstream project will observe, if they use RUNPATH.
On the other hand, we had no perceivable errors on our end for any downstream dependency and (local) deployment so far.
I can't explain this. Maybe most systems still wind up with RPATHs instead of RUNPATHs in binaries by default, or this problem is "hidden" if myexecutable
(or similarily for other projects) has libglbinding.so
and libglbinding-aux.so
as NEEDED dependencies since both of these are actually resolvable by myexecutable
. I am just guessing now.
I conclude, that an RPATH setting of $ORIGIN would result in a simpler setup with more external use cases in mind. On the other hand, this would clutter up the directory of the executables.
I agree that it would result in simpler setup with more external use cases in mind, however I disagree with that it would clutter up the directory of executables.
The "lib"-case (I understand this is your case)
// CASE lib
.
├── lib
│ ├── libglbinding.so.3
│ └── libglbinding-aux.so.3
└── myexecutable
works very well if libraries has RUNPATH $ORIGIN
and the executable has runpath $ORIGIN/lib
, since the libraries will resolve their dependencies by looking in the same directory that they themselves live in, whereas the executable will look in the ./lib
-dir for glbinding.so
(and possibly glbinding-aux.so
).
Point is, you can keep your way of organizing things AND change RUNPATH from $ORIGIN/lib
-> $ORIGIN
for your libraries (not executables!) and everything will still work, even for downstream projects.
I think the only project structure that will break with this change is:
.
├── lib
│ ├── libglbinding-aux.so.3 // can resolve libglbinding.so.3 since RUNPATH is $ORIGIN/lib
│ └── lib
│ └── libglbinding.so.3
└── myexecutable // can resolve libglbinding-aux.so.3 and libglbinding.so.3 since RUNPATH is $ORIGIN/lib:$ORIGIN/lib/lib
which is nonsensical anyhow.
Another solution may be the use of different RPATHs depending on whether we link an executable or a library, i.e., depending on the install directory.
I think this is the solution. The executables can have RUNPATH $ORIGIN/lib
and libraries can have RUNPATH $ORIGIN
.
Thank you for taking time to answer! I very much appreciate your engaged answers 👍 .
About the output from ~/dev/glbinding/glbinding-install$ LD_DEBUG=libs ./cubescape
, note that the only RUNPATH that is ever used to resolve anything is the "RUNPATH from file ./cubescape". The RUNPATH from glbinding-aux.so
is never used. I believe this happens because cubescape
depend directly on both glbinding.so
and glbinding-aux.so
, and these are resolved via cubscape
's RUNPATH before the RUNPATHs of glbinding-aux.so
is used. I am highly unsure if this is actually what happes, I am just guessing.
Nevertheless, ldd ./lib/glbinding-aux.so
will probably output libglbinding.so.3 => not found
, since then the RUNPATH of cubescape
won't be used to resolve any dependency.
Yeah, it was a nice deep-dive - thanks for the insights, links, discussion, and endurance - and you're totally right, libglbinding.so.3
could not be found:
~/dev/glbinding/glbinding-install$ ldd lib/libglbinding-aux.so
linux-vdso.so.1 (0x00007ffc14319000)
libglbinding.so.3 => not found
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f43e5ee2000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f43e5ec7000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f43e5ea4000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f43e5cb2000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f43e5b63000)
/lib64/ld-linux-x86-64.so.2 (0x00007f43e6239000)
It seems as if we need to change our setup to accommodate for both RPATH
and RUNPATH
.
I have built glbinding from source:
I cd into the install directory
cd ../install
and run:output:
This is because the runpath is set to
$ORIGIN/lib
, where$ORIGIN
forlibglbinding-aux.so
is it's absolute path (in my case~/glbinding/install/lib/
) and hence$ORIGIN/lib
is~/glbinding/install/lib/lib
. No wonderlibglbinding-aux.so
can't findlibglbinding.so.3
, since there is no~/glbinding/install/lib/lib/libglbinding.so.3
(note thelib/lib
!)I'd expect
to report
libglbinding.so.3
as found in~/glbinding/install/lib/libglbinding.so.3
Question(s):
$ORIGIN/lib
as opposed to just$ORIGIN
? If runpath ofglbinding-aux.so.3
was just$ORIGIN
, thenldd libglbinding-aux.so
would've foundlibglbinding.so.3
.Background: I use CMake to build an executable that depends on glbinding. I install the executable and all of it's runtime dependencies like so:
but this step fails since
libglbinding-aux.so
can't resolve it's dependency onlibglbinding.so.3
since no such file exists in RUNPATH. I can workaround this withpatchelf --set-rpath "\$ORIGIN" libglbinding-aux.so.3
, but I'd rather not do this since it's brittle and requires developer-machines to installpatchelf
.