conan-io / conan

Conan - The open-source C and C++ package manager
https://conan.io
MIT License
8.25k stars 980 forks source link

Symbols not found when linking against library provided by conan #2658

Closed DEGoodmanWilson closed 6 years ago

DEGoodmanWilson commented 6 years ago

To help us debug your issue please explain:

I'm back with more questions. I apologize, as I cannot tell at all where the problem lies. In short, on Linux, using GCC (verified with gcc6, but I bet it exists with other versions, and maybe with clang too) (notably, this is a non-issue on OSX), if you take this sample code: https://github.com/DEGoodmanWilson/luna/tree/testing/5.0.0/examples/project_template (note I'm working in branch testing/5.0.0 and do a conan install . && cmake . && cmake --build ., you get the following linker errors:

main.cpp:(.text+0x188): undefined reference to `luna::set_error_logger(std::function<void (luna::log_level, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>)'
main.cpp:(.text+0x22b): undefined reference to `luna::server::create_router(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)'
main.cpp:(.text+0x2d6): undefined reference to `luna::router::handle_request(luna::request_method, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::function<luna::response (luna::request const&)>, std::vector<luna::parameter::validator, std::allocator<luna::parameter::validator> >)'
...and so on

OK, this looks like luna isn't getting linked in. But, running cmake in verbose mode: cmake -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON . I can see that the linker command line does include -lluna and the appropriate -L directives, and can verify that the paths provided with -L exist

Finally, a quick check of libluna.a reveals that these symbols are in fact getting exported: nm -g ~/.conan/data/luna/5.0.0/DEGoodmanWilson/testing/package/220f2ee147781d76ef20f65b797d3e21181cfd64/lib/libluna.a | c++filt | grep set_error_logger

00000000000001f0 T luna::set_error_logger(std::function<void (luna::log_level, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>)

The exported symbols don't have the additional __cxx11 namespace qualifiers on std, which raises my suspicions a bit, but it's clear we're reaching the end of my experience here.

Is there something I've left out of the build process for luna that is causing this?

pvicente commented 6 years ago

Hi @DEGoodmanWilson

I've been reproducing your problem as you've explained at https://github.com/DEGoodmanWilson/luna/issues/49 with lasote/conangcc6

Your problem is related with How to manage the GCC >= 5 ABI.

You can see the following warning in conan when running for the first time in lasote/conangcc6:

conan@a023e116e7d1:/Users/pvicente/tmp/luna/examples/project_template$  conan install . && cmake . && cmake --build .
Auto detecting your dev setup to initialize the default profile (/home/conan/.conan/profiles/default)
Found gcc 6.4
gcc>=5, using the major as version

************************* WARNING: GCC OLD ABI COMPATIBILITY ***********************

Conan detected a GCC version > 5 but has adjusted the 'compiler.libcxx' setting to
'libstdc++' for backwards compatibility.
Your compiler is likely using the new CXX11 ABI by default (libstdc++11).

If you want Conan to use the new ABI, edit the default profile at:

    ~/.conan/profiles/default

adjusting 'compiler.libcxx=libstdc++11'

************************************************************************************

Default settings
    os=Linux
    os_build=Linux
    arch=x86_64
    arch_build=x86_64
    compiler=gcc
    compiler.version=6
    compiler.libcxx=libstdc++
    build_type=Release
*** You can change them in /home/conan/.conan/profiles/default ***
*** Or override with -s compiler='other' -s ...s***

In summary conan is using the old abi compatibility for libstdc++ to maintain backwards compatibility.

I can see that your project is using set(CMAKE_CXX_STANDARD 14).

You have two options to change this behaviour:

Hope it helps to resolve your question. I've tried 1st option from the command line and it worked:

 conan install . -s compiler.libcxx=libstdc++11 --build=missing && cmake . && cmake --build .
...
/usr/bin/make -f tests/CMakeFiles/awesomesauce_tests.dir/build.make tests/CMakeFiles/awesomesauce_tests.dir/depend
make[2]: Entering directory '/Users/pvicente/tmp/luna/examples/project_template'
cd /Users/pvicente/tmp/luna/examples/project_template && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /Users/pvicente/tmp/luna/examples/project_template /Users/pvicente/tmp/luna/examples/project_template/tests /Users/pvicente/tmp/luna/examples/project_template /Users/pvicente/tmp/luna/examples/project_template/tests /Users/pvicente/tmp/luna/examples/project_template/tests/CMakeFiles/awesomesauce_tests.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/Users/pvicente/tmp/luna/examples/project_template'
/usr/bin/make -f tests/CMakeFiles/awesomesauce_tests.dir/build.make tests/CMakeFiles/awesomesauce_tests.dir/build
make[2]: Entering directory '/Users/pvicente/tmp/luna/examples/project_template'
make[2]: Nothing to be done for 'tests/CMakeFiles/awesomesauce_tests.dir/build'.
make[2]: Leaving directory '/Users/pvicente/tmp/luna/examples/project_template'
[100%] Built target awesomesauce_tests
make[1]: Leaving directory '/Users/pvicente/tmp/luna/examples/project_template'
/usr/bin/cmake -E cmake_progress_start /Users/pvicente/tmp/luna/examples/project_template/CMakeFiles 0

BTW I've realised that lasote/conangcc6 doesn't have pkg-config system package, which is neeeded to build conan-gnutls one of the project dependencies, so you'll have to install it to do this test. I'll open an issue to add it to all docker images.

Maybe it's worth to rebuild in CI all of your dependency tree with both settings:

-compiler.libcxx=libstdc++ (default) for gcc<5
-compiler.libcxx=libstdc++11 for gcc>=5

This problem is not happening in MacOSX because it's using clang and it uses libc++ which is fully compatible with C++11 and C++14 (in latest version).

See also some related issues:

DEGoodmanWilson commented 6 years ago

This makes a lot of sense, thank you so kindly for bringing this to my attention! I agree that the wisest course of action is probably to generate builds for both standard library versions. However, I suspect that a number of package maintainers are going to run into this issue, and it would be really awesome if ConanMultiPackager.add_common_builds() could help to automate this…would that be a welcome patch?

pvicente commented 6 years ago

Yes maybe that's a good patch to help to other people hitting the same problem.

I'm pretty sure that @lasote and @memsharded have some inputs about it.

CC: @uilianries

DEGoodmanWilson commented 6 years ago

Interestingly, I see that the option pure_c=False for add_common_builds() seems to add builds for both libstc++ and libstdc++11 already ;)

https://travis-ci.org/DEGoodmanWilson/base64/jobs/354380826

pvicente commented 6 years ago

I see ... good catch!!!

pure_c: (Default True) If your project is C++, pass the pure_c=False to add both combinations using libstdc and libstdc++11 for the setting compiler.libcxx. When True, the default profile value of libcxx will be applied.

So that's your solution then.

If everything is ok for you feel free to close this ticket.

Cheers, Pedro

uilianries commented 6 years ago

@DEGoodmanWilson Yes, you can check about this on Conan package tools:

https://github.com/conan-io/conan-package-tools#complete-conanmultipackager-methods-reference

By default it uses pure_c=True that means, do not link libstc++11

uilianries commented 6 years ago

If you are interested, Bincrafters created an wrapper for Conan Package Tools: https://github.com/bincrafters/bincrafters-package-tools

It invokes Conan package tools as well, but it solves mandatory variables, as CONAN_REFERENCE, CONAN_USERNAME ...

DEGoodmanWilson commented 6 years ago

So, I've run into an interesting problem, which is that now luna, which is building with pure_c=False, won't pull down any pure C library without rebuilding it. https://travis-ci.org/DEGoodmanWilson/luna/jobs/356970004

Is there any way around this? Do I need to build C libraries in three variants?

Interestingly, this doesn't seem to be an issue for gcc4.9, but it is an issue for clang.

uilianries commented 6 years ago

C libraries should not care about libcxx, Conan has no crystal ball to guess if the project is pure C, so you need to tell it:

class FoobarConan(Conanfile):
    ....
    def config_options(self):
        del self.settings.compiler.libcxx

After that, your package won't care about libcxx in it package id. As result, Luna will works fine even liking libc++, libstdc++11 or libstdc++

pvicente commented 6 years ago

Hi @DEGoodmanWilson

I've checked the new problem and I can see that dependency libgpg-error was built with pure_c=True (default). This package is C_only package and the recipe should have a method:

    def configure(self):
        del self.settings.compiler.libcxx

It's not well documented (I'll fix it) but you can see some references about it in docs. Adding this method conan will ignore the libcxx setting for that library and it won't generate any error when it's consumed for any c++ package which is building with libcxx setting.

The best approach is to revisit all your dependencies a update the recipes for C-only packages/libraries.

With the latest conan version 1.1.1 if you run conan new for c package only:

$ conan new -c hello/0.1

You can see that the template conanfile has this method.

18:13 $ cat conanfile.py
from conans import ConanFile, CMake, tools

class HelloConan(ConanFile):
    name = "hello"
    version = "0.1"
    license = "<Put the package license here>"
    url = "<Package recipe repository url here, for issues about the package>"
    description = "<Description of Hello here>"
    settings = "os", "compiler", "build_type", "arch"
    options = {"shared": [True, False]}
    default_options = "shared=False"
    generators = "cmake"

    def source(self):
        self.run("git clone https://github.com/memsharded/hello.git")
        self.run("cd hello && git checkout static_shared")
        # This small hack might be useful to guarantee proper /MT /MD linkage in MSVC
        # if the packaged project doesn't have variables to set it properly
        tools.replace_in_file("hello/CMakeLists.txt", "PROJECT(MyHello)", '''PROJECT(MyHello)
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()''')

    def build(self):
        cmake = CMake(self)
        cmake.configure(source_folder="hello")
        cmake.build()

        # Explicit way:
        # self.run('cmake %s/hello %s' % (self.source_folder, cmake.command_line))
        # self.run("cmake --build . %s" % cmake.build_config)

    def package(self):
        self.copy("*.h", dst="include", src="hello")
        self.copy("*hello.lib", dst="lib", keep_path=False)
        self.copy("*.dll", dst="bin", keep_path=False)
        self.copy("*.so", dst="lib", keep_path=False)
        self.copy("*.dylib", dst="lib", keep_path=False)
        self.copy("*.a", dst="lib", keep_path=False)

    def package_info(self):
        self.cpp_info.libs = ["hello"]

    def configure(self):
        del self.settings.compiler.libcxx

Hope it helps.

uilianries commented 6 years ago

@pvicente Which one should be used config_options or configure?

pvicente commented 6 years ago

configure for this case.

The doc about these methods says config_options should only be used for options of the package and in this case we want to remove a setting (not an option) which is always set so it should be manipulated in config method.

See example about how to manipulate them (settings and options) in other recipe 😉

Hope it helps.

uilianries commented 6 years ago

Oh, my fault, I use config_options to remove self.options.fPIC on Windows.

Thanks!

pvicente commented 6 years ago

But that's totally fine, it's the way to do it, fPIC is an option.

DEGoodmanWilson commented 6 years ago

So. Here's where things stand.

All the pure C dependencies have been built with

def configure(self):
        del self.settings.compiler.libcxx

in the Conanfile.

All the C++ dependencies have been built with pure_c=False in the build.py, generating builds for both versions of the GCC standard library.

pvicente commented 6 years ago

Hi @DEGoodmanWilson,

When I go to build an app that depends on Luna using GCC 6 using stdlibc++, I get the same linker errors. Why? The dependencies were built linking against stdlibc++, why would there be a problem?

It looks like the same problem than yesterday, it's consuming your library to be linked in your program with libstdc++ which will fail in gcc>=5 because it needs to link against libstdc++11. You can create a profile to consume your library with libstdc++11 for gcc>=5 or in the command line specify:

conan install . -s compiler.libcxx=libstdc++11 && cmake . && cmake --build .

When I got to build an app that depends on Luna using GCC 6 using stdlibc++11, I am told that the first pure C dependency encountered is missing prebuilt packages for the configuration, and need to be built from sources. Why?

I've tried to reproduce the problem with a bash script in examples/project_template/test.bash using lasote/conangcc6 docker image:

#!/usr/bin/env bash
# Remove default remote
conan remote remove conan-center

# Add your remotes to consume your libraries
conan remote add DEGoodmanWilson https://api.bintray.com/conan/degoodmanwilson/opensource
conan remote add conan-center https://conan.bintray.com
conan remote add bincrafters https://api.bintray.com/conan/bincrafters/public-conan

# Create luna library with libstdc++11 setting. All good here!!!
conan create ../../ DEGoodmanWilson/stable -s compiler.libcxx=libstdc++11

# Create app.
conan install . -s compiler.libcxx=libstdc++11  && cmake . && cmake --build .

# It fails with the following back_trace
#cpr/1.3.0@DEGoodmanWilson/stable: WARN: Can't find a 'cpr/1.3.0@DEGoodmanWilson/stable' package for the specified options and settings:
#- Settings: arch=x86_64, build_type=Release, compiler=gcc, compiler.libcxx=libstdc++11, compiler.version=6, os=Linux
#- Options: shared=False, use_ssl=True, OpenSSL:386=False, OpenSSL:no_asm=False, OpenSSL:no_bf=False, OpenSSL:no_cast=False, OpenSSL:no_des=False, OpenSSL:no_dh=False, OpenSSL:no_dsa=False, OpenSSL:no_hmac=False, OpenSSL:no_md2=False, OpenSSL:no_md5=False, OpenSSL:no_mdc2=False, OpenSSL:no_rc2=False, OpenSSL:no_rc4=False, OpenSSL:no_rc5=False, OpenSSL:no_rsa=False, OpenSSL:no_sha=False, OpenSSL:no_sse2=False, OpenSSL:no_threads=False, OpenSSL:no_zlib=False, OpenSSL:shared=False, libcurl:custom_cacert=False, libcurl:disable_threads=False, libcurl:fPIC=True, libcurl:shared=False, libcurl:with_largemaxwritesize=False, libcurl:with_ldap=False, libcurl:with_libidn=False, libcurl:with_libmetalink=False, libcurl:with_libpsl=False, libcurl:with_librtmp=False, libcurl:with_libssh2=False, libcurl:with_nghttp2=False, libcurl:with_openssl=True, zlib:shared=False
#- Package ID: 3c7387f53b67a8db30c50cbfa456d4e0d6216138

#ERROR: Missing prebuilt package for 'cpr/1.3.0@DEGoodmanWilson/stable'
#Try to build it from sources with "--build cpr"
#Or read "http://docs.conan.io/en/latest/faq/troubleshooting.html#error-missing-prebuilt-package"

Here the problem is with the build_require cpr/1.2.0@DEGoodmanWilson/stable for the test app which also needs to build with pure_c=False. https://github.com/DEGoodmanWilson/conan-cpr/blob/stable/1.3.0/build.py#L73

I've built conan-cpr in local container with:

conan@42e4ce731064:/Users/pvicente/tmp/luna/examples/project_template$ conan create ../../../conan-cpr DEGoodmanWilson/stable -s compiler.libcxx=libstdc++11

and running the script again the build for this test app works with gcc6:

conan@42e4ce731064:/Users/pvicente/tmp/luna/examples/project_template$ bash test.bash
...
PROJECT: Generated conaninfo.txt
-- The C compiler identification is GNU 6.4.0
-- The CXX compiler identification is GNU 6.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Current conanbuildinfo.cmake directory: /Users/pvicente/tmp/luna/examples/project_template
-- Conan: Compiler GCC>=5, checking major version 6
-- Conan: Checking correct version: 6
-- Conan: Using cmake global configuration
-- Conan: Adjusting default RPATHs Conan policies
-- Conan: Adjusting language standard
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/pvicente/tmp/luna/examples/project_template
Scanning dependencies of target awesomesauce
[ 16%] Building CXX object CMakeFiles/awesomesauce.dir/logger.cpp.o
[ 33%] Building CXX object CMakeFiles/awesomesauce.dir/main.cpp.o
[ 50%] Linking CXX executable bin/awesomesauce
[ 50%] Built target awesomesauce
Scanning dependencies of target awesomesauce_tests
[ 66%] Building CXX object tests/CMakeFiles/awesomesauce_tests.dir/main.cpp.o
[ 83%] Building CXX object tests/CMakeFiles/awesomesauce_tests.dir/basic.cpp.o
[100%] Linking CXX executable ../bin/awesomesauce_tests
[100%] Built target awesomesauce_tests

Hope it helps you.

DEGoodmanWilson commented 6 years ago

OK, I fixed cpr—I failed to update that dependency to use pure_c=False

After that, long story short: conan install . -s compiler.libcxx=libstdc++11 && cmake . && cmake --build . works great.

conan install . -s compiler.libcxx=libstdc++ && cmake . && cmake --build . gives me errors, but of an entirely different sort around my use of std::experimental::optional which I think I ought to be able to handle…anyway, it's not the same old linker errors ;)

closing…

DEGoodmanWilson commented 6 years ago

I've encountered one more problem. Everything is awesome in gcc land…but clang 4 is giving me the same linker errors now.

docker run -it lasote/conanclang40 /bin/bash
git clone https://github.com/DEGoodmanWilson/luna.git
cd luna/examples/project_template/
conan remote add vthiery https://api.bintray.com/conan/vthiery/conan-packages
conan remote add degoodmanwilson https://api.bintray.com/conan/degoodmanwilson/opensource
conan remote add bincrafters https://api.bintray.com/conan/bincrafters/public-conan
conan install . && cmake . && cmake --build .

yields:

[ 50%] Linking CXX executable bin/awesomesauce
CMakeFiles/awesomesauce.dir/logger.cpp.o: In function `access_logger(luna::request const&, luna::response const&)':
/home/conan/luna/examples/project_template/logger.cpp:(.text+0x1d0): undefined reference to `luna::to_string[abi:cxx11](luna::request_method)'
CMakeFiles/awesomesauce.dir/logger.cpp.o: In function `std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, luna::case_insensitive_comp_, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >::at(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const':
/home/conan/luna/examples/project_template/logger.cpp:(.text._ZNKSt3mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES5_N4luna22case_insensitive_comp_ESaISt4pairIKS5_S5_EEE2atERS9_[_ZNKSt3mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES5_N4luna22case_insensitive_comp_ESaISt4pairIKS5_S5_EEE2atERS9_]+0x80): undefined reference to `luna::case_insensitive_comp_::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const'

and so forth.

There's clearly some kind of linker error involving the standard library going on here, but once again I've run into a wall—I can't tell what's at issue.

DEGoodmanWilson commented 6 years ago

Interestingly conan is defaulting to libstc++ for clang, rather than libc++ as i would have expected. This is true for all the C++ dependencies I've built as well.

DEGoodmanWilson commented 6 years ago

OK, I tried the obvious solution that didn't occur to me on Friday:

conan install . -s compiler.libcxx=libc++

But that doesn't work either.

/home/conan/luna/examples/project_template/logger.cpp:(.text+0x1d0): undefined reference to `luna::to_string[abi:cxx11](luna::request_method)'
CMakeFiles/awesomesauce.dir/logger.cpp.o: In function `std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, luna::case_insensitive_comp_, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >::at(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const':

and so on.

pvicente commented 6 years ago

Hi @DEGoodmanWilson,

Apparently for clang -stdlib=libc++ needs to be passed in the linkage but CMake is not doing it with the generated conanbuildinfo.cmake:

conan install . -s compiler.libcxx=libc++ && cmake -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON . && cmake --build .

[ 50%] Linking CXX executable bin/awesomesauce
/usr/bin/cmake -E cmake_link_script CMakeFiles/awesomesauce.dir/link.txt --verbose=1
/usr/bin/clang++   -pthread      -rdynamic CMakeFiles/awesomesauce.dir/logger.cpp.o CMakeFiles/awesomesauce.dir/main.cpp.o  -o bin/awesomesauce  -L/home/conan/.conan/data/cpr/1.3.0/DEGoodmanWilson/stable/package/3438e1a30384eb49f379f2fb5162af029f454ea9/lib  -L/home/conan/.conan/data/libcurl/7.56.1/bincrafters/stable/package/518adacba6b7e42d168cb5770de606f97982bd37/lib  -L/home/conan/.conan/data/OpenSSL/1.0.2n/conan/stable/package/d3ab52c2b9e30f36b8789a53b9a306f9bdaf29db/lib  -L/home/conan/.conan/data/luna/5.0.0/DEGoodmanWilson/stable/package/b823981dcc3b8ce18e739a34c68aadc4a662b546/lib  -L/home/conan/.conan/data/base64/1.0.2/DEGoodmanWilson/stable/package/0f421c56ac4f8c45bfd8394ff96b259913656b71/lib  -L/home/conan/.conan/data/libmicrohttpd/0.9.51/DEGoodmanWilson/stable/package/babc31cef8b5043e4ec2d4cee7daff6e344c913d/lib  -L/home/conan/.conan/data/libmime/0.1.0/DEGoodmanWilson/stable/package/96b5ad879ea44dd7c820510e4f6bb907ae43b009/lib  -L/home/conan/.conan/data/gnutls/3.6.2/DEGoodmanWilson/stable/package/7f149a1ddca1d9ab94b27b208045949039e35d79/lib  -L/home/conan/.conan/data/jsonformoderncpp/3.1.2/vthiery/stable/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/lib  -L/home/conan/.conan/data/libgcrypt/1.7.3/DEGoodmanWilson/stable/package/96995c0d44dd600aef59af811e49c4ceaf8742e6/lib  -L/home/conan/.conan/data/libgpg-error/1.24/DEGoodmanWilson/stable/package/e5bde775ebac8b0a7cd94a071bb551fc164bae97/lib  -L/home/conan/.conan/data/libiconv/1.15/bincrafters/stable/package/a3c1782e6f0acdaccbc6b52d45f3ac8597be6c8a/lib  -L/home/conan/.conan/data/nettle/3.4/DEGoodmanWilson/stable/package/2f4611af8a31c11e5ec2a1ca417dbe99c5873bd5/lib  -L/home/conan/.conan/data/zlib/1.2.11/conan/stable/package/e5bde775ebac8b0a7cd94a071bb551fc164bae97/lib  -L/home/conan/.conan/data/gmp/6.1.1/DEGoodmanWilson/stable/package/b6cb11375855cfba4cc14c081dbbf73c1376ee74/lib -Wl,-rpath,/home/conan/.conan/data/cpr/1.3.0/DEGoodmanWilson/stable/package/3438e1a30384eb49f379f2fb5162af029f454ea9/lib:/home/conan/.conan/data/libcurl/7.56.1/bincrafters/stable/package/518adacba6b7e42d168cb5770de606f97982bd37/lib:/home/conan/.conan/data/OpenSSL/1.0.2n/conan/stable/package/d3ab52c2b9e30f36b8789a53b9a306f9bdaf29db/lib:/home/conan/.conan/data/luna/5.0.0/DEGoodmanWilson/stable/package/b823981dcc3b8ce18e739a34c68aadc4a662b546/lib:/home/conan/.conan/data/base64/1.0.2/DEGoodmanWilson/stable/package/0f421c56ac4f8c45bfd8394ff96b259913656b71/lib:/home/conan/.conan/data/libmicrohttpd/0.9.51/DEGoodmanWilson/stable/package/babc31cef8b5043e4ec2d4cee7daff6e344c913d/lib:/home/conan/.conan/data/libmime/0.1.0/DEGoodmanWilson/stable/package/96b5ad879ea44dd7c820510e4f6bb907ae43b009/lib:/home/conan/.conan/data/gnutls/3.6.2/DEGoodmanWilson/stable/package/7f149a1ddca1d9ab94b27b208045949039e35d79/lib:/home/conan/.conan/data/jsonformoderncpp/3.1.2/vthiery/stable/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/lib:/home/conan/.conan/data/libgcrypt/1.7.3/DEGoodmanWilson/stable/package/96995c0d44dd600aef59af811e49c4ceaf8742e6/lib:/home/conan/.conan/data/libgpg-error/1.24/DEGoodmanWilson/stable/package/e5bde775ebac8b0a7cd94a071bb551fc164bae97/lib:/home/conan/.conan/data/libiconv/1.15/bincrafters/stable/package/a3c1782e6f0acdaccbc6b52d45f3ac8597be6c8a/lib:/home/conan/.conan/data/nettle/3.4/DEGoodmanWilson/stable/package/2f4611af8a31c11e5ec2a1ca417dbe99c5873bd5/lib:/home/conan/.conan/data/zlib/1.2.11/conan/stable/package/e5bde775ebac8b0a7cd94a071bb551fc164bae97/lib:/home/conan/.conan/data/gmp/6.1.1/DEGoodmanWilson/stable/package/b6cb11375855cfba4cc14c081dbbf73c1376ee74/lib -lcpr -lcurl -lrt -lpthread -lssl -lcrypto -ldl -lluna -lbase64 -lmicrohttpd -lmime -lgnutls -lcipher -lmpi -lrandom -lcompat -lgcrypt -lgpg-error -liconv -lnettle -lhogweed -lz -lgmp -lgmpxx

Everything works like a charm if you replace your conanfile.txt with the following conanfile.py (with the same info than your conanfile.txt but it can also build your project with the conan's cmake helper).

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from conans import ConanFile, CMake

class TestPackageConan(ConanFile):
    settings = "os", "compiler", "build_type", "arch"
    generators = "cmake"
    requires = "luna/[~=5.0]@DEGoodmanWilson/stable", "jsonformoderncpp/[~= 3.1]@vthiery/stable"
    build_requires = "cpr/[~=1.3]@DEGoodmanWilson/stable", "Catch/[~=1.9]@bincrafters/stable"

    def build(self):
        cmake = CMake(self)
        cmake.verbose = True
        cmake.configure()
        cmake.build()

and to build the app you only need to:

conan install . && conan build .

See stdlib=libstdc++ (detected default for clang40)

[ 83%] Linking CXX executable bin/awesomesauce
/usr/bin/cmake -E cmake_link_script CMakeFiles/awesomesauce.dir/link.txt --verbose=1
/usr/bin/clang++   -pthread -m64 -stdlib=libstdc++ -O3 -DNDEBUG

In this case conan is using cmake helper to build the project and it's adjusting all the flags based on your cmake project settings set(CMAKE_CXX_STANDARD 14) to build ok with any compiler and defaults detected for each docker image.

You can also force to use a concrete version of libstdcxx and it should work too.

For clang:

conan install . -s compiler.libcxx=libc++ && conan build .

For gcc>=5: conan install . -s compiler.libcxx=libstdc++11 && conan build .

Hope it helps.

DEGoodmanWilson commented 6 years ago

This seems like a bug in Conan…I don't want to force my end users to have to use a conanfile.py if it's not necessary. I'll update the template for now, however. Thanks for tracking this down!

memsharded commented 6 years ago

It is not necessary to force the usage of conanfile.py. What is necessary is that together the provided CMakeLists.txt and the conanbuildinfo.cmake, define the correct configuration. It is important to know that by default, conanbuildinfo.cmake do not define any CMake configuration, and it relies on some CONAN_XXX variables to be defined. So the definition of which standard library to use will not be done by conanbuildinfo.cmake, unles CONAN_LIBCXX is provided.

pvicente commented 6 years ago

@DEGoodmanWilson ,

This command should work with your original conanfile.txt

conan install . && cmake -DCONAN_LIBCXX=libstdc++ . && cmake --build .

Tested with lasote/conanclang40 and lasote/conangcc6.

DEGoodmanWilson commented 6 years ago

Interesting. What is different about the GCC case that this issue doesn't arise there?

pvicente commented 6 years ago

Because with this definition conan is explicitly setting:

add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0)

which is choosing the right ABI for libstdc++ to build your test program.

If you want to use libstdc++11 you have to use:

conan install . -s compiler.libcxx=libstdc++11 && cmake -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DCONAN_LIBCXX=libstdc++11 . && cmake --build .

and you'll see -D_GLIBCXX_USE_CXX11_ABI=1 when building your test program.

DEGoodmanWilson commented 6 years ago

Forgive me, as now I am quite confused. This advice seems in conflict with what was said here: https://github.com/conan-io/conan/issues/2658#issuecomment-376153286

You can also force to use a concrete version of libstdcxx and it should work too. For clang: conan install . -s compiler.libcxx=libc++ && conan build . For gcc>=5: conan install . -s compiler.libcxx=libstdc++11 && conan build . Hope it helps.

(Indeed, I can confirm this advice has worked for me with GCC, that is, without having to specify -DCONAN_LIBCXX to cmake. Why is clang handled differently from GCC?)

pvicente commented 6 years ago

clang decided to implement its own version of stdc++ library with libc++.

But you can use clang and link your program against the gcc version libstdc++.

If you run in lasote/conanclang40:

conan install . && cmake -DCONAN_LIBCXX=libstdc++ . && cmake --build .

it should work.

and to make it work with libc++ and clang:

conan install . -s compiler.libcxx=libc++ && cmake -DCONAN_LIBCXX=libc++ -DCONAN_COMPILER=clang . && cmake --build .

DEGoodmanWilson commented 6 years ago

I think we're talking past each other here. What I'm trying to express is that I'm surprised that, with clang, -DCONAN_LIBCXX=libc++ is necessary. This step isn't necessary when specifying which standard library to use with GCC. And you hadn't mentioned it before. I'm trying to understand why it is necessary for clang + libc++—this feels like really terrible UX for using conan if usage looks like this:

gcc 5 clang 4
C++98 conan install . && cmake . conan install . && cmake .
C++11 conan install . -s compiler.libcxx=libstdc++11 && cmake . conan install . -s compiler.libcxx=libc++ && cmake -DCONAN_LIBCXX=libstdc++ .

It seems like conan should be able to add the necessary directive to cmake itself, rather than requiring they be added by the end user, in the "clang 4 + c++11" case.

DEGoodmanWilson commented 6 years ago

It occurs to me that part of the confusion is that I have silently shifted my question from “how do I make this work?” To “why do I have to do that to make it work?” My apologies.

pvicente commented 6 years ago

Sorry I've just closed the issue by mistake. I'm not sure if I'm following you but it looks like you don't like the idea of passing CONAN_XXXX variables for your project configuration when using CMake?

I'm going to try to summarise all the tests/configurations that I've done with your test project (target to C++14) to make it work with conan 1.1.1

Test has been done with lasote/conangcc5 and lasote/conanclang4 and the following remotes:

conan remote add -i 0 DEGoodmanWilson https://api.bintray.com/conan/degoodmanwilson/opensource
conan remote add bincrafters https://api.bintray.com/conan/bincrafters/public-conan
lasote/conangcc5 lasote/conanclang40
libstdc++(default in profile) conan install . && cmake . -DCONAN_LIBCXX=libstdc++ conan install . && cmake . -DCONAN_LIBCXX=libstdc++
libstdc++11 conan install . -s compiler.libcxx=libstdc++11 && cmake . -DCONAN_LIBCXX=libstdc++11 conan install . -s compiler.libcxx=libstdc++11 && cmake . -DCONAN_LIBCXX=libstdc++11
libc++ ---------------- conan install . -s compiler.libcxx=libc++ && cmake . -DCONAN_LIBCXX=libc++ -DCONAN_COMPILER=clang

I think that from here we can discuss why do we need to define these variables or how can we improve it from consumer point of view, if that's ok with you.

As a note to don't forget a conanfile.py has also been provided (some comments above) as a workaround to simplify the buid of your test project in case building with cmake and passing -DCONAN_XXXX is problematic.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from conans import ConanFile, CMake

class TestPackageConan(ConanFile):
    settings = "os", "compiler", "build_type", "arch"
    generators = "cmake"
    requires = "luna/[~=5.0]@DEGoodmanWilson/stable", "jsonformoderncpp/[~= 3.1]@vthiery/stable"
    build_requires = "cpr/[~=1.3]@DEGoodmanWilson/stable", "Catch/[~=1.9]@bincrafters/stable"

    def build(self):
        cmake = CMake(self)
        cmake.verbose = True
        cmake.configure()
        cmake.build()
DEGoodmanWilson commented 6 years ago

Yes, I think you've got the gist of what I'm saying. You might break it down, in fact, into two concerns:

  1. You don't have to define CONAN_LIBCXX in most contexts.
  2. You shouldn't have to define CONAN_LIBCXX in any context, because it's redundant with the -s compiler.libcxx=… parameter to conan.
pvicente commented 6 years ago

I think that we'll need the input from @memsharded / @lasote They'll know the real reasons about this behaviour or if there is something that we are missing.

memsharded commented 6 years ago

The rationale behind this is the following:

In short: CONAN_LIBCXX shouldn't be defined by the user, as long as the consumer project or CMakeLists.txt uses the same standard lib that was used to install the dependencies. But conan will not change the consumer build configuration proactively by using the generated conanbuildinfo.cmake. If the user wants conan to define build system configuration, it has to explicitly tell it.

As usual, it is a trade-off: a bit of duplication and some non fully automated tasks, yes, but it achieves very strong decoupling between the package manager and the build system and great flexibility while never blocking users because of being too smart. And if further automation is desired, it is as simple as providing a conanfile.py with a build() method in it. This layered and decoupled design is key to conan operation. We have been there in the past, trying to add too much intelligence to the build system, and it always leads to blocked users and more pain than it solves.

DEGoodmanWilson commented 6 years ago

@memsharded Thank you that clarifies quite a lot.

I had understood the right way to inform CMake of my language desires was with

set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

Correct? Which is what I'm doing. If that is the correct way, then I don't need to mess around with CONAN_LIBCXX at all, is that what you are saying?


So, back to my original question then, as it doesn't seem to be settled after all:

It looks like CMake is not adding --std=c++14 to the compiler flags, or -lstdc++ to the linker flags, which is quite surprising to me. I'm going to have to dig deeper to solve these linker errors.

DEGoodmanWilson commented 6 years ago

Ok the problem lies somewhere between me and CMake at this point so I’m going to call this closed. Thank you all so much!

memsharded commented 6 years ago

They are actually different settings:

There might be different ways to define the libcxx natively in CMake, to be honest, I am not sure which one is really recommended, I tend to use set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -stdlib=libc++")

Thanks to you for all the feedback!