search

conan-io / conan-center-index

Recipes for the ConanCenter repository
https://conan.io/center
MIT License
972 stars 1.78k forks source link

[package] onetbb/2021.10.0: tbbmalloc is not linked #22984

Open maichmueller opened 8 months ago

maichmueller commented 8 months ago

Description

EDIT: Linking onetbb fails for libtbbmalloc. My CMake configuration reads:

find_package(TBB REQUIRED)
...
target_link_libraries(
        train
        PRIVATE
        onetbb::onetbb
)

Running the executable throws the following error:

train: error while loading shared libraries: libtbbmalloc_debug.so.2: cannot open shared object file: No such file or directory

and

train: error while loading shared libraries: libtbbmalloc.so.2: cannot open shared object file: No such file or directory

respectively depending on the config.

CMake-Conan handles the conan installation via the provided conan_provider.cmake. My conandata.yml includes the aforementioned package among others and conanfile.py is defaulted.

Checking the executable in debug and release with ldd shows the following:

ldd train                                                                                                                                       
...
    libtbb_debug.so.12 => /u/michael.aichmueller/.conan2/p/b/onetb22d5b511a1c11/p/lib/libtbb_debug.so.12 (0x00007f95cb9fa000)
    libtbbmalloc_proxy_debug.so.2 => /u/michael.aichmueller/.conan2/p/b/onetb22d5b511a1c11/p/lib/libtbbmalloc_proxy_debug.so.2 (0x00007f95cb9f3000)
    libtbbmalloc_debug.so.2 => not found

and

ldd train                                                                                                                                                   
...
    libtbbmalloc_proxy.so.2 => /u/michael.aichmueller/.conan2/p/onetb75ff22d6ae745/p/lib/libtbbmalloc_proxy.so.2 (0x00007ff4e3936000)
    libtbb.so.12 => /u/michael.aichmueller/.conan2/p/onetb75ff22d6ae745/p/lib/libtbb.so.12 (0x00007ff4e38df000)
    libtbbmalloc.so.2 => not found

But all the libs are available in /u/michael.aichmueller/.conan2/p/onetb75ff22d6ae745/p/lib (release) and /u/michael.aichmueller/.conan2/p/b/onetb22d5b511a1c11/p/lib (debug):

ls /u/michael.aichmueller/.conan2/p/b/onetb22d5b511a1c11/p/lib/                                                                                       
libtbbbind_2_5_debug.so       libtbb_debug.so        libtbbmalloc_debug.so       libtbbmalloc_proxy_debug.so
libtbbbind_2_5_debug.so.3     libtbb_debug.so.12     libtbbmalloc_debug.so.2     libtbbmalloc_proxy_debug.so.2
libtbbbind_2_5_debug.so.3.10  libtbb_debug.so.12.10  libtbbmalloc_debug.so.2.10  libtbbmalloc_proxy_debug.so.2.10

ls /u/michael.aichmueller/.conan2/p/onetb75ff22d6ae745/p/lib                                                                                          
libtbbbind_2_5.so    libtbbbind_2_5.so.3.10  libtbbmalloc_proxy.so.2     libtbbmalloc.so    libtbbmalloc.so.2.10  libtbb.so.12
libtbbbind_2_5.so.3  libtbbmalloc_proxy.so   libtbbmalloc_proxy.so.2.10  libtbbmalloc.so.2  libtbb.so             libtbb.so.12.10

I don't understand how linking tbbmalloc can succeed, but running the executable fails to find this lib in the same folder as the other tbb libs.

Package and Environment Details

Conan profile

conan profile show Host profile: [settings] arch=x86_64 build_type=Release compiler=gcc compiler.cppstd=gnu17 compiler.libcxx=libstdc++11 compiler.version=11 os=Linux

Build profile: [settings] arch=x86_64 build_type=Release compiler=gcc compiler.cppstd=gnu17 compiler.libcxx=libstdc++11 compiler.version=11 os=Linux

Steps to reproduce

conandata.yml

# This file is managed by Conan, contents will be overwritten.
# To keep your changes, remove these comment lines, but the plugin won't be able to modify your requirements

requirements:
  - "onetbb/2021.10.0"
  - "range-v3/0.12.0"
  - "boost/1.80.0"
  - "nlohmann_json/3.11.2"
  - "tclap/1.2.5"
  - "fmt/[>=10.2.1]"
  - "spdlog/[>=1.12.0]"

conanfile.py

# This file is managed by Conan, contents will be overwritten.
# To keep your changes, remove these comment lines, but the plugin won't be able to modify your requirements

from conan import ConanFile
from conan.tools.cmake import cmake_layout, CMakeToolchain

class ConanApplication(ConanFile):
    package_type = "application"
    settings = "os", "compiler", "build_type", "arch"
    generators = "CMakeDeps"

    def layout(self):
        cmake_layout(self)

    def generate(self):
        tc = CMakeToolchain(self)
        tc.user_presets_path = False
        tc.generate()

    def requirements(self):
        requirements = self.conan_data.get('requirements', [])
        for requirement in requirements:
            self.requires(requirement)

Run: path/to/cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_C_COMPILER=/usr/bin/gcc -DCMAKE_CXX_COMPILER=/usr/bin/g++ -G Ninja -DCMAKE_PROJECT_TOP_LEVEL_INCLUDES=conan_provider.cmake -DCONAN_COMMAND=path/to/conan

Logs

Click to expand log ``` /u/michael.aichmueller/cmake-3.26.5-linux-x86_64/bin/cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_MAKE_PROGRAM=ninja -DCMAKE_C_COMPILER=/usr/bin/gcc -DCMAKE_CXX_COMPILER=/usr/bin/g++ -G Ninja -DCMAKE_PROJECT_TOP_LEVEL_INCLUDES=conan_provider.cmake -DCONAN_COMMAND=/work/rleap1/michael.aichmueller/miniconda/bin/conan -S /work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network -B /work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/cmake-build-debug-gcc-lactose -- CPM: Adding package microtar@27076 (27076e1) -- CMake-Conan: first find_package() found. Installing dependencies with Conan -- CMake-Conan: Checking if a default profile exists /u/michael.aichmueller/.conan2/profiles/default -- CMake-Conan: cmake_system_name=Linux -- CMake-Conan: cmake_system_processor=x86_64 -- CMake-Conan: CMake compiler=GNU -- CMake-Conan: CMake compiler version=11.4.0 -- CMake-Conan: [settings] compiler=gcc -- CMake-Conan: [settings] compiler.version=11 -- CMake-Conan: Creating profile /work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/cmake-build-debug-gcc-lactose/conan_host_profile -- CMake-Conan: Profile: [settings] arch=x86_64 os=Linux compiler=gcc compiler.version=11 compiler.libcxx=libstdc++11 build_type=Debug [conf] tools.cmake.cmaketoolchain:generator=Ninja tools.build:compiler_executables={"c":"/usr/bin/gcc","cpp":"/usr/bin/g++"} -- CMake-Conan: Installing single configuration Debug -- CMake-Conan: conan install /work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network -of=/work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/cmake-build-debug-gcc-lactose/conan --profile:host=default;--profile:host=/work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/cmake-build-debug-gcc-lactose/conan_host_profile;--profile:build=default;--build=missing;-g;CMakeDeps ======== Input profiles ======== Profile host: [settings] arch=x86_64 build_type=Debug compiler=gcc compiler.cppstd=gnu17 compiler.libcxx=libstdc++11 compiler.version=11 os=Linux [conf] tools.cmake.cmaketoolchain:generator=Ninja tools.build:compiler_executables={'c': '/usr/bin/gcc', 'cpp': '/usr/bin/g++'} Profile build: [settings] arch=x86_64 build_type=Release compiler=gcc compiler.cppstd=gnu17 compiler.libcxx=libstdc++11 compiler.version=11 os=Linux ======== Computing dependency graph ======== Graph root conanfile.py: /work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/conanfile.py Requirements ... hwloc/2.9.3#7b004fab2336e27d686e9ae08dcf4b0d - Cache ... onetbb/2021.10.0#159e2dde755615b9f4d43b6868cdd1d2 - Cache ... Build requirements b2/4.10.1#1b290403d8648c79f468f5a6496f829a - Cache meson/1.2.2#aace9dcc1db58fa42ecb5292f724092d - Cache ninja/1.11.1#77587f8c8318662ac8e5a7867eb4be21 - Cache pkgconf/2.1.0#27f44583701117b571307cf5b5fe5605 - Cache Resolved version ranges ... ======== Computing necessary packages ======== Requirements ... c2a87ebbbc9c1 - Cache hwloc/2.9.3#7b004fab2336e27d686e9ae08dcf4b0d:56fabb2f37a6bb653f7e6dace9a7d53edb7ea31a#0498c2022f27709979bbe5847e6e7080 - Cache ... onetbb/2021.10.0#159e2dde755615b9f4d43b6868cdd1d2:8b2b2c1c14c6fcd8e1404345bdd476ed368ce915#8e82421c33b31696d7be11226265eaa5 - Cache ... Build requirements Skipped binaries b2/4.10.1, meson/1.2.2, ninja/1.11.1, pkgconf/2.1.0 ======== Installing packages ======== ... hwloc/2.9.3: Already installed! (3 of 11) ... onetbb/2021.10.0: Already installed! (11 of 11) ======== Finalizing install (deploy, generators) ======== conanfile.py: Writing generators to /work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/cmake-build-debug-gcc-lactose/conan/build/Debug/generators conanfile.py: Generator 'CMakeDeps' calling 'generate()' conanfile.py: Calling generate() conanfile.py: Generators folder: /work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/cmake-build-debug-gcc-lactose/conan/build/Debug/generators conanfile.py: CMakeToolchain generated: conan_toolchain.cmake conanfile.py: Preset 'conan-debug' added to CMakePresets.json. Invoke it manually using 'cmake --preset conan-debug' if using CMake>=3.23 conanfile.py: If your CMake version is not compatible with CMakePresets (<3.23) call cmake like: 'cmake -G Ninja -DCMAKE_TOOLCHAIN_FILE=/work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/cmake-build-debug-gcc-lactose/conan/build/Debug/generators/conan_toolchain.cmake -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_BUILD_TYPE=Debug' conanfile.py: CMakeToolchain generated: CMakePresets.json conanfile.py: Generating aggregated env files conanfile.py: Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh'] Install finished successfully -- CMake-Conan: CONAN_GENERATORS_FOLDER=/work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/cmake-build-debug-gcc-lactose/conan/build/Debug/generators -- CMake-Conan: CONANFILE=/work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/conanfile.py ... -- CMake-Conan: find_package(Git) found, 'conan install' already ran -- CMake-Conan: find_package(TBB) found, 'conan install' already ran -- Conan: Component target declared 'TBB::tbb' -- Conan: Component target declared 'TBB::tbbmalloc' -- Conan: Component target declared 'TBB::tbbmalloc_proxy' -- Conan: Target declared 'onetbb::onetbb' -- CMake-Conan: find_package(hwloc) found, 'conan install' already ran -- Conan: Target declared 'hwloc::hwloc' ... -- The following REQUIRED packages have been found: ... * TBB -- Configuring done (4.5s) -- Generating done (0.1s) -- Build files have been written to: /work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/cmake-build-debug-gcc-lactose [Finished] ```
maichmueller commented 8 months ago

If I add the conan package folder for tbb to LD_LIBRARY_PATH it does find tbbmalloc as well

LD_LIBRARY_PATH=/u/michael.aichmueller/.conan2/p/onetb75ff22d6ae745/p/lib/:$LD_LIBRARY_PATH ldd train                                    
...
    libtbbmalloc_proxy.so.2 => /u/michael.aichmueller/.conan2/p/onetb75ff22d6ae745/p/lib/libtbbmalloc_proxy.so.2 (0x00007efd7b721000)
    libtbb.so.12 => /u/michael.aichmueller/.conan2/p/onetb75ff22d6ae745/p/lib/libtbb.so.12 (0x00007efd7b6ca000)
    libtbbmalloc.so.2 => /u/michael.aichmueller/.conan2/p/onetb75ff22d6ae745/p/lib/libtbbmalloc.so.2 (0x00007efd7b10d000)

Oddly enough, it seems that tbbmalloc is not mentioned in the dynamic section of the executable for some reason:

readelf -d ./train | grep 'NEEDED'                                                                                                                    
 0x0000000000000001 (NEEDED)             Shared library: [libtorch_cpu.so]
 0x0000000000000001 (NEEDED)             Shared library: [libc10.so]
 0x0000000000000001 (NEEDED)             Shared library: [libtorch.so]
 0x0000000000000001 (NEEDED)             Shared library: [libtbbmalloc_proxy.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libtbb.so.12]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]

The ninja build file segement pertaining to the train executable does list all tbb libs to link against with the correct paths:

Click to expand log ``` build train: CXX_EXECUTABLE_LINKER__train_Release CMakeFiles/train.dir/src/train.cpp.o CMakeFiles/train.dir/src/helpers.cpp.o CMakeFiles/train.dir/src/train_dl.cpp.o CMakeFiles/train.dir/src/train_rl.cpp.o CMakeFiles/train.dir/src/train_ol.cpp.o | libdataset.a libexperiments.a libformalism.a libmodel.a libpddl.a libplanners.a libdataset.a libplanners.a libmodel.a /work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/deps/libtorch_cpu/lib/libtorch.so /work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/deps/libtorch_cpu/lib/libc10.so /work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/deps/libtorch_cpu/lib/libc10.so /work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/deps/libtorch_cpu/lib/libkineto.a libalgorithms.a libformalism.a libmicrotar.a libmurmurhash3.a /u/michael.aichmueller/.conan2/p/spdloefc7a16752d1b/p/lib/libspdlog.a /u/michael.aichmueller/.conan2/p/fmt5520fd882199a/p/lib/libfmt.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_log_setup.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_log.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_locale.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_iostreams.a /u/michael.aichmueller/.conan2/p/bzip24ed69d4fecc16/p/lib/libbz2.a /u/michael.aichmueller/.conan2/p/zlibcdd5270f9ab9f/p/lib/libz.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_graph.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_fiber_numa.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_fiber.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_contract.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_wave.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_unit_test_framework.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_type_erasure.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_timer.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_thread.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_random.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_prg_exec_monitor.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_nowide.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_json.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_filesystem.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_coroutine.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_chrono.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_wserialization.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_test_exec_monitor.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_stacktrace_noop.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_stacktrace_basic.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_stacktrace_backtrace.a /u/michael.aichmueller/.conan2/p/libbab546f21710147/p/lib/libbacktrace.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_stacktrace_addr2line.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_serialization.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_regex.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_program_options.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_math_tr1l.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_math_tr1f.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_math_tr1.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_math_c99l.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_math_c99f.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_math_c99.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_exception.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_date_time.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_context.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_container.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_atomic.a /u/michael.aichmueller/.conan2/p/onetb75ff22d6ae745/p/lib/libtbbmalloc_proxy.so /u/michael.aichmueller/.conan2/p/onetb75ff22d6ae745/p/lib/libtbbmalloc.so /u/michael.aichmueller/.conan2/p/onetb75ff22d6ae745/p/lib/libtbb.so || libalgorithms.a libdataset.a libexperiments.a libformalism.a libmicrotar.a libmodel.a libmurmurhash3.a libpddl.a libplanners.a FLAGS = -O3 -DNDEBUG -flto=auto -fno-fat-lto-objects LINK_LIBRARIES = -Wl,-rpath,/u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib:/u/michael.aichmueller/.conan2/p/bzip24ed69d4fecc16/p/lib:/u/michael.aichmueller/.conan2/p/zlibcdd5270f9ab9f/p/lib:/u/michael.aichmueller/.conan2/p/libbab546f21710147/p/lib:/u/michael.aichmueller/.conan2/p/range0301bf3d76d5d/p/lib:/u/michael.aichmueller/.conan2/p/fmt5520fd882199a/p/lib:/u/michael.aichmueller/.conan2/p/spdloefc7a16752d1b/p/lib:/u/michael.aichmueller/.conan2/p/onetb75ff22d6ae745/p/lib:/work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/deps/libtorch_cpu/lib libdataset.a libexperiments.a libformalism.a libmodel.a libpddl.a libplanners.a libdataset.a libplanners.a libmodel.a /work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/deps/libtorch_cpu/lib/libtorch.so -Wl,--no-as-needed,"/work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/deps/libtorch_cpu/lib/libtorch_cpu.so" -Wl,--as-needed /work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/deps/libtorch_cpu/lib/libc10.so -Wl,--no-as-needed,"/work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/deps/libtorch_cpu/lib/libtorch.so" -Wl,--as-needed /work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/deps/libtorch_cpu/lib/libc10.so /work/rleap1/michael.aichmueller/projects/hpl/Relational-Neural-Network/deps/libtorch_cpu/lib/libkineto.a libalgorithms.a libformalism.a libmicrotar.a libmurmurhash3.a /u/michael.aichmueller/.conan2/p/spdloefc7a16752d1b/p/lib/libspdlog.a /u/michael.aichmueller/.conan2/p/fmt5520fd882199a/p/lib/libfmt.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_log_setup.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_log.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_locale.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_iostreams.a /u/michael.aichmueller/.conan2/p/bzip24ed69d4fecc16/p/lib/libbz2.a /u/michael.aichmueller/.conan2/p/zlibcdd5270f9ab9f/p/lib/libz.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_graph.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_fiber_numa.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_fiber.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_contract.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_wave.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_unit_test_framework.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_type_erasure.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_timer.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_thread.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_random.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_prg_exec_monitor.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_nowide.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_json.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_filesystem.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_coroutine.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_chrono.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_wserialization.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_test_exec_monitor.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_stacktrace_noop.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_stacktrace_basic.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_stacktrace_backtrace.a /u/michael.aichmueller/.conan2/p/libbab546f21710147/p/lib/libbacktrace.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_stacktrace_addr2line.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_serialization.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_regex.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_program_options.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_math_tr1l.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_math_tr1f.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_math_tr1.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_math_c99l.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_math_c99f.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_math_c99.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_exception.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_date_time.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_context.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_container.a /u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib/libboost_atomic.a /u/michael.aichmueller/.conan2/p/onetb75ff22d6ae745/p/lib/libtbbmalloc_proxy.so /u/michael.aichmueller/.conan2/p/onetb75ff22d6ae745/p/lib/libtbbmalloc.so /u/michael.aichmueller/.conan2/p/onetb75ff22d6ae745/p/lib/libtbb.so -lm -lrt -lpthread -ldl LINK_PATH = -L/u/michael.aichmueller/.conan2/p/boostb11903ce0da08/p/lib -L/u/michael.aichmueller/.conan2/p/bzip24ed69d4fecc16/p/lib -L/u/michael.aichmueller/.conan2/p/zlibcdd5270f9ab9f/p/lib -L/u/michael.aichmueller/.conan2/p/libbab546f21710147/p/lib -L/u/michael.aichmueller/.conan2/p/range0301bf3d76d5d/p/lib -L/u/michael.aichmueller/.conan2/p/fmt5520fd882199a/p/lib -L/u/michael.aichmueller/.conan2/p/spdloefc7a16752d1b/p/lib -L/u/michael.aichmueller/.conan2/p/onetb75ff22d6ae745/p/lib OBJECT_DIR = CMakeFiles/train.dir POST_BUILD = : PRE_LINK = : TARGET_FILE = train TARGET_PDB = train.dbg ```
jcar87 commented 8 months ago

If I remember correctly, tbbmalloc is not linked directly, and would be brought in by tbbmalloc_proxy instead.

I can find some evidence here:

I'd have thought that its use was opt-in as well, rather than by default - ie, I dont think your executable should link against tbbmalloc_proxy either.

maichmueller commented 8 months ago

hi @jcar87 ,

thanks for looking into this.

The behaviour of find_package(TBB REQUIRED) without specifying components is to select all components tbb, tbbmalloc, tbbmalloc_proxy according to the official README. The conan package mimics this behaviour from what I can tell and also links all three components into onetbb::onetbb:

    ########## AGGREGATED GLOBAL TARGET WITH THE COMPONENTS #####################
    set_property(TARGET onetbb::onetbb PROPERTY INTERFACE_LINK_LIBRARIES TBB::tbbmalloc_proxy APPEND)
    set_property(TARGET onetbb::onetbb PROPERTY INTERFACE_LINK_LIBRARIES TBB::tbbmalloc APPEND)
    set_property(TARGET onetbb::onetbb PROPERTY INTERFACE_LINK_LIBRARIES TBB::tbb APPEND)

This means that target_link_libraries(train PRIVATE onetbb::onetbb) brings in all components.

To be fair, I do not know what exactly I needed from TBB to use parallel std algorithms with GCC so I went with the default. But even if only e.g. tbb was needed, I would expect that the linkage provided by the conan-package for onetbb works for all components. I do not quite understand why building and linking against all 3 components works as expected, but only tbbmalloc is not found at runtime (despite residing in the same folder as the other tbb libs). Is this an RPATH issue?

jcar87 commented 8 months ago

Hi @maichmueller - I see, thanks for linking to the documentation.

the behaviour of find_package(TBB REQUIRED) is to find all components, but you still need to choose which targets to link against. From what I can see, the onetbb::onetbb target is generated by Conan's CMakeDeps generator but has no equivalent in the TBB documentation. We do have targets for the ones mentioned by the TBB documentation.

I think your problem has to do with linking against the conan-only onetbb::onetbb, when you should link against TBB::tbb , and only if you know you need the malloc_proxy, link against that one too.

find_package(TBB REQUIRED)
...
target_link_libraries(
        train
        PRIVATE
        onetbb::onetbb
)

should be:

find_package(TBB REQUIRED)
...
target_link_libraries(
        train
        PRIVATE
        TBB::tbb
)

I do not quite understand why building and linking against all 3 components works as expected, but only tbbmalloc is not found at runtime (despite residing in the same folder as the other tbb libs). Is this an RPATH issue?

I suspect that the way tbbmalloc is loaded at runtime is different than regular libraries. I can see some evidence here: https://github.com/oneapi-src/oneTBB/blob/9e401d119c9422775b2267cbf133f0446493ddf7/src/tbbmalloc/tbbmalloc.cpp#L27 , where dlopen() is used. dlopen may (or may not) follow slightly different rules or nuances than regular library loading. We could have a look, however I suspect in your case the solution is as above and only link against TBB::tbb

maichmueller commented 8 months ago

Hi @jcar87 ,

thanks for your help on this. I linked as you said now only to TBB::tbb which works out of the box. I am happy to leave it at that for my use case, but think that any user wishing to link against tbbmalloc would run into this problem again.

For completeness sake, I installed TBB via apt for some tests and noticed that my package would compensate for the missing libtbbmalloc by simply picking it up from the system-wide installation while libtbb would still be selected from the conan folder. Not sure which extra settings allow finding it in the system-wide install though and whether they are set by the package or whether using standard ubuntu search paths is already enough.