RPATH needs to be used for dynamic linking components

[Serafadam]

When trying to link against shared lib targets (for example rtabmap::utilite) one needs to set --disable-new-dtags for target GCC linker options or otherwise shared libs cannot be found at runtime (rtabmap::core works normally, but rtabmap::utilite cannot be found even though it is in the same directory). This also causes some issues when trying to build on MacOS/Clang setup. Note - this happens when RTABMap is not installed on system but built as part of another project, in this case included via Hunter

[matlabbe]

Is this issue only on Mac OSX? On linux, I have included in another project the library from the build directory (not installed) and it worked. When launching the third party app, we should still add in the PATH the bin directory though to be executed.

[Serafadam]

This also happened on Linux, currently there have been some changes in the main codebase but I can point to some older branch that still has those parts of code. Here we set up a target that links to RTABMap's libraries and can be used to link to our examples. But if we want to create the example as in here, then everything build correctly but linker cannot find rtabmap::utilite target (this can be investigated with ld), even though rtabmap_utilite.so is in the same directory as rtabmap::core which is found properly. With --disable-new-dtags it works properly but it seems not all compilers have this option.

[matlabbe]

I'll try to create a minimal example on my side to reproduce the problem, based on your examples. I won't be able to work on this the next 2 weeks. I'll keep you updated afterwards.

[matlabbe]

I tried to dig more in the cmake/linker lore to know what is happening. That tag --disable-new-dtags is used to disable usage of RUNPATH and use old RPATH on recent compilers. Here is a video (found from this thread Setting RPATH and not RUNPATH for executable) explaining the difference between RPATH and RUNPATH (watch from 45:10). Basically, LD_LIBRARY_PATH is looked after RPATH in old behavior but before RUNPATH in new behavior. Another related post: https://stackoverflow.com/questions/52018092/how-to-set-rpath-and-runpath-with-gcc-ld

That explains why on Ubuntu 22.04, I started to see rtabmap launched from build directory loading shared rtabmap's library installed in ROS binaries directory /opt/ros/humble/lib/... instead of those built in same directory than the executable. I thought it was a new behavior of recent OS, but actually it can use libraries built in same directory, it was just seeing the same ones installed by ROS because ROS adds them to the LD_LIBRARY_PATH. So ideally, keep LD_LIBRARY_PATH empty unless you really want to use specific libraries, and RUNPATH should then find the correct libraries from the build directory. Because ROS seems to update LD_LIBRARY_PATH anyway, a workaround when developping in build directories is to add :: at the begging of LD_LIBRARY_PATH so that current directory of where you launch an executable is looked first:

echo "export LD_LIBRARY_PATH=::$LD_LIBRARY_PATH" >> ~/.bashrc

For reference:

• Command to see if RPATH or RUNPATH is used in an executable, and check if the path is good (here interestingly I was linking to an OpenCV version built from source and installed in its own build directory):

  $ cd ~/workspace/rtabmap/build/bin
  $ readelf -d ./rtabmap |head -20
  [...]
   0x000000000000001d (RUNPATH) Library runpath: [/home/mathieu/workspace/rtabmap/build/bin:/home/mathieu/workspace/opencv/build/install/lib:/opt/ros/humble/lib/x86_64-linux-gnu:]
  [...]

In conclusion, as RUNPATH seems the default new approach, I would not set --disable-new-dtags by default. So I added in this commit https://github.com/introlab/rtabmap/commit/f73e9c02b3b7d624c454b2ab096e11ce6171a972 a new cmake option called BUILD_WITH_RPATH_NOT_RUNPATH that you can activate for your internal build:

$ cd ~/workspace/rtabmap/build/
$ cmake -DBUILD_WITH_RPATH_NOT_RUNPATH=ON ..
[...]
-- --------------------------------------------
-- Info :
--   RTAB-Map Version =     0.21.6
--   CMAKE_VERSION =        3.22.1
--   CMAKE_INSTALL_PREFIX = /home/mathieu/workspace/rtabmap/build_cv_latest/install
--   CMAKE_BUILD_TYPE =     Release
--   CMAKE_INSTALL_LIBDIR = lib
--   BUILD_APP =            ON
--   BUILD_TOOLS =          ON
--   BUILD_EXAMPLES =       ON
--   BUILD_SHARED_LIBS =    ON
--   CMAKE_CXX_FLAGS =  -fmessage-length=0 -fopenmp -std=c++17
--   LINKER FLAGS =    --disable-new-dtags                      <-------------------- new
[...]
$ make
$ cd bin
$ readelf -d ./rtabmap |head -20
[...]
 0x000000000000000f (RPATH)  Library rpath: [/home/mathieu/workspace/rtabmap/build/bin:/home/mathieu/workspace/opencv/build/install/lib:/opt/ros/humble/lib/x86_64-linux-gnu:]
[...]

cheer, Mathieu