conan-io / conan

Conan - The open-source C and C++ package manager
https://conan.io
MIT License
8.14k stars 970 forks source link

[question] CONAN_LIB::smth links transitively with other CONAN_LIBs #17008

Open HazyMrf opened 3 days ago

HazyMrf commented 3 days ago

What is your question?

Hello! I'm writing conanfile.py to pull gperftools into my project. I need my libraries to be built static as .a files so I cannot use default package. By default tcmalloc from gperftools has dependencies (as target_link_libraries) from libraries called spinlock, sysinfo and logging. Right now I'm able to compile my project by doing smth like

target_link_libraries(APP PRIVATE
    CONAN_LIB::gperftools_tcmalloc
    CONAN_LIB::gperftools_spinlock
    CONAN_LIB::gperftools_sysinfo
    CONAN_LIB::gperftools_logging
)

What I want, is just write

target_link_libraries(APP PRIVATE
    CONAN_LIB::gperftools_tcmalloc
)

Here is my conanfile.py:

from conans import ConanFile, CMake, tools
from conan.tools.files import get

class LightgbmConan(ConanFile):
    name = "gperftools"
    version = "2.15-full"
    url = "https://github.com/gperftools/gperftools"
    description = "Conan package for tcmalloc and its cpu/heap profiler."
    settings = "os", "compiler", "build_type", "arch"
    generators = "cmake"

    # Options here ...

    def _add_component(self, lib):
        self.cpp_info.components[lib].libs = [lib]
        self.cpp_info.components[lib].set_property("pkg_config_name", f"lib{lib}")

    def source(self):
        get(self, "https://github.com/gperftools/gperftools/releases/download/gperftools-2.15/gperftools-2.15.tar.gz", strip_root=True)

    def build(self):
        cmake = CMake(self)

      # A lot of code here

        cmake.configure()
        cmake.build()

    def package(self):
        self.copy("*.h", dst='include/gperftools', src='src/gperftools')
        self.copy("*.h", dst='include/google', src='src/google')
        self.copy("*.a", dst="lib", keep_path=False)

    def package_info(self):
        for lib in ["tcmalloc_and_profiler", "tcmalloc", "tcmalloc_minimal", "profiler",
                    "sysinfo", "spinlock", "logging", "fake_stacktrace_scope"]:
            self._add_component(lib)

        LIBSTACKTRACE = ['spinlock', 'sysinfo', 'logging']

        self.cpp_info.components['tcmalloc_minimal'].requires = LIBSTACKTRACE
        self.cpp_info.components['tcmalloc_minimal'].libs += LIBSTACKTRACE

        self.cpp_info.components['tcmalloc'].requires = LIBSTACKTRACE
        self.cpp_info.components['tcmalloc'].libs += LIBSTACKTRACE

        self.cpp_info.components['tcmalloc_and_profiler'].requires = LIBSTACKTRACE
        self.cpp_info.components['tcmalloc_and_profiler'].libs += LIBSTACKTRACE

        self.cpp_info.components['profiler'].requires = LIBSTACKTRACE + ['fake_stacktrace_scope']
        self.cpp_info.components['profiler'].libs += LIBSTACKTRACE + ['fake_stacktrace_scope']

        for component in self.cpp_info.components.values():
            if self.settings.os in ["Linux", "FreeBSD"]:
                component.system_libs.extend(["pthread", "m"])
                component.cflags.append("-pthread")
                component.cxxflags.append("-pthread")
                component.requires.append("libunwind::libunwind")

        self.cpp_info.includedirs = ["include"]

        if self.settings.os == 'Linux':
            self.requires("libunwind/1.6.2")
HazyMrf commented 3 days ago

So I understand I need to do some manipulations with .requires field, but I just don't understand how to achieve the desired result :( . I see that m and pthread are linked together with every component, but I cannot link themselves between themselves

memsharded commented 3 days ago

Hi @HazyMrf

Thanks for your question.

Hello! I'm writing conanfile.py to pull gperftools into my project. I need my libraries to be built static as .a files so I cannot use default package.

I am not sure what you mean. I have checked the gperftools package recipe and it defaults to building static libraries:

 default_options = {
        "shared": False,

(as most Conan package recipes in ConanCenter do)

from conans import ConanFile, CMake, tools

It seems that you are using very legacy Conan 1.X syntax. Even if using Conan 1.X, the imports that you should be using should be:

from conan import ConanFile
from conan.tools.cmake import CMake

But from conans should not longer be used. This applies also to generators = "cmake", this shouldn't be used anymore.

Same with self.copy(), it shouldn't be used anymore, instead use from conan.tools.files import copy and ``copy(self, ...```

In general the recommendation would be to use the ConanCenter recipes, and modify/tune from there what is needed, these recipes are already modernized so they work both with Conan 1 and Conan 2.

HazyMrf commented 2 days ago

Hello @memsharded, thank you for such a quick response. The reasons I decided to write my own conanfile are:

  1. gperftools libraries in conan-center package are dynamically linked with libc++/libstdc++, meaning even though they are static, they have smth like libc++.so in dependencies. My project cannot allow that, I need to enforce static linking. I actually solved this problem by deleting this line https://github.com/conan-io/conan-center-index/blob/master/recipes/gperftools/all/conanfile.py#L200
  2. This problem I couldn't solve myself, that's why I decided to write my own conanfile. When I build static libraries and I just set flags build_heap_profiler=True and build_heap_checker=True and then link with CONAN_LIB::gperftools_tcmalloc everything works just fine, but when I add build_cpu_profiler=True, I cannot link with CONAN_LIB::gperftools_tcmalloc anymore:
    ld.lld-18: error: unable to find library -lprofiler
    ld.lld-18: error: unable to find library -ltcmalloc_and_profiler
    clang++-18: error: linker command failed with exit code 1 (use -v to see invocation)

When I try to link with CONAN_LIB::gperftools_tcmalloc_and_profiler, CMake just cannot find these libraries:

  Target "processor" links to:

    CONAN_LIB::gperftools_tcmalloc_and_profiler

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.
HazyMrf commented 2 days ago

BTW when I open conanbuildinfo.txt I see this:

[libs_gperftools]
tcmalloc_minimal
tcmalloc
profiler
tcmalloc_and_profiler

[system_libs_gperftools]
pthread
m

So basically library CONAN_LIB::gperftools_tcmalloc_and_profiler must exist, but yet it doesn't

At the same time, when I try to manually inspect installed libraries, I see that there are no profilers

user@machine:~/project/build-release-clang$ ls /home/user/.conan/data/gperftools/2.15-patched/my_artifactory/package/05f8d434f3281f6460cdb60743163e815604864d/lib
libtcmalloc.a  libtcmalloc_minimal.a