conan-io / conan

Conan - The open-source C and C++ package manager
https://conan.io
MIT License
8.12k stars 968 forks source link

[question] Keying packages on libstdc++ version? Autodetecting it? #16802

Open nextsilicon-itay-bookstein opened 1 month ago

nextsilicon-itay-bookstein commented 1 month ago

What is your question?

Hi! I'm trying to push packages to a conan artifactory instance and reuse them in compatible environments. To that end I added keying for the packages through settings_user.yml. Specifically, I added keying on (distro, distro_version). However I found that's not enough since we have easybuild-based environments which provide libstdc++ versions that are newer than the distro's, and then packages produced in that environment cannot be used on the vanilla distro itself because they demand newer symbols from libstdc++.

I thought I might further key the package on a compiler.libcxx.version setting field I would add, and was wondering if that makes sense and what's the best practice here.

I was also wondering if I could somehow extend the APIs available at profile templating time in order to be able to call some code to autodetect this property, instead of doing it outside and passing it in via an environment variable. WDYT?

Lastly, I was wondering if more things (like the distro module and maybe also subprocess) could be made available during template formatting to make this keying easier?

Have you read the CONTRIBUTING guide?

jcar87 commented 1 month ago

Hi @nextsilicon-itay-bookstein - thanks for reaching out.

Typically when the compiler is gcc, compiler.version should effectively "track" the version of libstdc++ (as it goes hand in hand with a particular major version of GCC, if I'm not mistaken). Is this not the case?

of course, it's a different story for glibc, or if the compiler is clang whilst using libstdc++

nextsilicon-itay-bookstein commented 1 month ago

I ended up relying on these two articles:

Because the property I'm trying to capture is "abi compat on dynamic linking", I did the following:

jcar87 commented 1 month ago

https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html

Looking at this -

GCC 10.1.0: libstdc++.so.6.0.28

GCC 11.1.0: libstdc++.so.6.0.29

GCC 12.1.0: libstdc++.so.6.0.30

GCC 13.1.0: libstdc++.so.6.0.31

GCC 13.2.0: libstdc++.so.6.0.32

GCC 14.1.0: libstdc++.so.6.0.33

It seems that versions of libstdc++ go hand in hand with the compiler version, up to the minor component of the version.

Hence I'm surprised - I would have imagined that settings like the following:

compiler=gcc
compiler.version=13.2

would already be enough to capture the binary compatibility around libstdc++ - especially considering that I'm unaware of getting g++ to use a different version of libstdc++ than the one that corresponds to the same version ! so there will always be a 1:1 mapping

nextsilicon-itay-bookstein commented 1 month ago

We're using clang with libstdc++ 🙂 Also I haven't checked what happens when you use a newer gcc on a system whose systemwide libstdc++ version is older. Maybe it always picks its own libstdc++ and then you're right for gcc. But as a general rule using a compiler that isn't the system compiler opens these cans of worms.

memsharded commented 3 weeks ago

As some extra hint, note that Conan 2 has the profile.py plugin, that can be used to check, complete (we use it for completing runtime_type based on build_type for MSVC), so it might also be used for injecting some functionality. I agree that it seems that profile templates would be a slightly better place, though it is true that too heavy system detection in profiles like using subprocess sounds too much, and might be an indicator that pre-defined profiles (maybe with profile.py checks) could be easier than auto-detecting profiles.