Cmake generated `seastar.pc` missing most of seastar's dependencies in the link flags

denesb commented 5 years ago

What seems to be there:

boost (most of it)
fmt
cares
cryptocpp

What is missing:

-lrt
-ldl
-lnuma
-lhwloc
-lpthread
YAML-cpp
-lstdc++fs
boost::filesystem

denesb commented 5 years ago

\cc @hakuch

denesb commented 5 years ago

Also the linked executable won't start:

$ ./generate_test_file 
./generate_test_file: error while loading shared libraries: libfmt.so.5: cannot open shared object file: No such file or directory

$ ldd generate_test_file
        linux-vdso.so.1 (0x00007ffef48ce000)
        libboost_program_options.so.1.66.0 => /lib64/libboost_program_options.so.1.66.0 (0x00007f1cc7948000)
        libboost_system.so.1.66.0 => /lib64/libboost_system.so.1.66.0 (0x00007f1cc7743000)
        libboost_thread.so.1.66.0 => /lib64/libboost_thread.so.1.66.0 (0x00007f1cc7517000)
        libcares.so.2 => /lib64/libcares.so.2 (0x00007f1cc7305000)
        libcryptopp.so.6 => /lib64/libcryptopp.so.6 (0x00007f1cc6c7d000)
        libfmt.so.5 => not found
        liblz4.so.1 => /lib64/liblz4.so.1 (0x00007f1cc6a66000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f1cc685e000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f1cc665a000)
        libhwloc.so.5 => /lib64/libhwloc.so.5 (0x00007f1cc641c000)
        libnuma.so.1 => /lib64/libnuma.so.1 (0x00007f1cc6210000)
        libyaml-cpp.so.0.6 => /lib64/libyaml-cpp.so.0.6 (0x00007f1cc5f9a000)
        libboost_filesystem.so.1.66.0 => /lib64/libboost_filesystem.so.1.66.0 (0x00007f1cc5d80000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f1cc59ed000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f1cc5659000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f1cc5441000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1cc5222000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f1cc4e63000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f1cc7bc9000)
        libltdl.so.7 => /lib64/libltdl.so.7 (0x00007f1cc4c59000)

Notice how the dynamic linker doesn't know where to find libfmt.so.5

hakuch commented 5 years ago

This description actually confuses a few different things, and I believe it is incorrect.

First, the seastar.pc file actually does include the libraries that you are describing as missing. This is the contents of a generated file on my machine for the Debug build type:

prefix=/usr/local
exec_prefix=${prefix}
includedir=${prefix}/include
libdir=${prefix}/lib64

Name: Seastar
Url: http://seastar-project.org
Description: Advanced C++ framework for high-performance server applications on modern hardware.
Version: 1.0

# Platform dependencies.
dl_libs=-ldl
stdfilesystem_libs=-lstdc++fs
rt_libs=-lrt

# Dependencies of dependencies.
boost_system_libs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/libboost_system.so

# Dependencies.
boost_cflags=-I/home/jhaberku/src/seastar/build/_cooking/installed/include
boost_filesystem_libs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/libboost_filesystem.so
boost_program_options_libs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/libboost_program_options.so
boost_thread_libs=${boost_system_libs} /home/jhaberku/src/seastar/build/_cooking/installed/lib/libboost_thread.so
c_ares_cflags=-I/home/jhaberku/src/seastar/build/_cooking/installed/include
c_ares_libs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/libcares.so
cryptopp_cflags=-I/home/jhaberku/src/seastar/build/_cooking/installed/include
cryptopp_libs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/libcryptopp.so
dpdk_libs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_cfgfile.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_cmdline.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_eal.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_ethdev.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_hash.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_kvargs.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_mbuf.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_mempool.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_mempool_ring.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_bnxt.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_e1000.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_ena.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_enic.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_fm10k.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_qede.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_i40e.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_ixgbe.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_nfp.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_ring.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_sfc_efx.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_vmxnet3_uio.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_ring.a
fmt_cflags=-I/home/jhaberku/src/seastar/build/_cooking/installed/include
fmt_libs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/libfmtd.so.5.2.1
lksctp_tools_cflags=-I/home/jhaberku/src/seastar/build/_cooking/installed/include
lksctp_tools_libs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/libsctp.so
numactl_cflags=-I/home/jhaberku/src/seastar/build/_cooking/installed/include
numactl_libs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/libnuma.so
sanitizers_cflags=-fsanitize=address -fsanitize=undefined

# Us.
seastar_cflags=-I${includedir} -std=gnu++17 -fconcepts -DSEASTAR_HAVE_GCC6_CONCEPTS -DSEASTAR_DEBUG -DSEASTAR_DEBUG_SHARED_PTR -DSEASTAR_ASAN_ENABLED -DSEASTAR_THREAD_STACK_GUARDS -DFMT_SHARED
seastar_libs=${libdir}/libseastar.a

Requires: liblz4 >= 1.8.0
Requires.private: gnutls >= 3.5.18, protobuf >= 3.3.0, hwloc >= 1.11.5, yaml-cpp >= 0.5.3
Conflicts:
Cflags: ${boost_cflags} ${c_ares_cflags} ${cryptopp_cflags} ${fmt_cflags} ${lksctp_tools_cflags} ${numactl_cflags} ${sanitizers_cflags} ${seastar_cflags}
Libs: ${boost_program_options_libs} ${boost_thread_libs} ${c_ares_libs} ${cryptopp_libs} ${fmt_libs} ${seastar_libs}
Libs.private: ${dl_libs} ${rt_libs} ${boost_filesystem_libs} ${boost_thread_libs} ${dpdk_libs} ${lksctp_tools_libs} ${numactl_libs} ${stdfilesystem_libs}

I see all of the libraries you mentioned there.

Second, you are confusing (if I'm not mistaken) compile-time shared-library resolution with run-time shared-library resolution. The latter is controlled by something called the rpath.

The fmt library is resolved at compile-time by CMake and the library is linked successfully. However, CMake by default clears the rpath when installing the files, since the shared-library can be provided from any location on a system where Seastar is installed.

What this means is that the fmt library has to be installed in location accessible to the linker at run-time: either in a system-standard location like /usr/lib or perhaps in another location with the addition of an environmental variable like LD_LIBRARY_PATH.

I think this issue should be closed in its current form, but if you have any questions about this we can talk either on the mailing-list or in private.

denesb commented 5 years ago

On Fri, 2018-12-14 at 06:25 -0800, Jesse Haber-Kucharsky wrote:

This description actually confuses a few different things, and I believe it is incorrect.

First, the seastar.pc file actually does include the libraries that you are describing as missing. This is the contents of a generated file on my machine for the Release build type:

prefix=/usr/localexec_prefix=${prefix}includedir=${prefix}/includelib dir=${prefix}/lib64 Name: SeastarUrl: http://seastar-project.orgDescription: Advanced C++ framework for high-performance server applications on modern hardware.Version: 1.0

Platform dependencies.dl_libs=-ldlstdfilesystem_libs=-

lstdc++fsrt_libs=-lrt

Dependencies of

dependencies.boost_system_libs=/home/jhaberku/src/seastar/build/_cook ing/installed/lib/libboost_system.so

Dependencies.boost_cflags=-

I/home/jhaberku/src/seastar/build/_cooking/installed/includeboost_fil esystem_libs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/ libboost_filesystem.soboost_program_options_libs=/home/jhaberku/src/s eastar/build/_cooking/installed/lib/libboost_programoptions.soboost thread_libs=${boost_system_libs} /home/jhaberku/src/seastar/build/_cooking/installed/lib/libboost_thre ad.soc_ares_cflags=- I/home/jhaberku/src/seastar/build/_cooking/installed/includec_ares_li bs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/libcares.s ocryptopp_cflags=- I/home/jhaberku/src/seastar/build/cooking/installed/includecryptopp libs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/libcrypt opp.sodpdk_libs=/home/jhaberku/src/seastar/build/_cooking/installed/l ib/librte_cfgfile.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_cmdlin e.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_eal.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_ethdev .a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_hash.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_kvargs .a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_mbuf.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_mempoo l.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_mempoo l_ring.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_bn xt.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_e1 000.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_en a.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_en ic.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_fm 10k.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_qe de.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_i4 0e.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_ix gbe.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_nf p.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_ri ng.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_sf c_efx.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_vm xnet3_uio.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_ring.a fmt_cflags=- I/home/jhaberku/src/seastar/build/_cooking/installed/includefmt_libs= /home/jhaberku/src/seastar/build/_cooking/installed/lib/libfmtd.so.5. 2.1lksctp_tools_cflags=- I/home/jhaberku/src/seastar/build/_cooking/installed/includelksctp_to ols_libs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/libs ctp.sonumactl_cflags=- I/home/jhaberku/src/seastar/build/_cooking/installed/includenumactl_l ibs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/libnuma.s osanitizers_cflags=-fsanitize=address -fsanitize=undefined

Us.seastar_cflags=-I${includedir} -std=gnu++17 -fconcepts

-DSEASTAR_HAVE_GCC6_CONCEPTS -DSEASTAR_DEBUG -DSEASTAR_DEBUG_SHARED_PTR -DSEASTAR_ASAN_ENABLED -DSEASTAR_THREAD_STACK_GUARDS -DFMT_SHAREDseastar_libs=${libdir}/libseastar.a Requires: liblz4 >= 1.8.0Requires.private: gnutls >= 3.5.18, protobuf

= 3.3.0, hwloc >= 1.11.5, yaml-cpp >= 0.5.3Conflicts:Cflags: ${boost_cflags} ${c_ares_cflags} ${cryptopp_cflags} ${fmt_cflags} ${lksctp_tools_cflags} ${numactl_cflags} ${sanitizers_cflags} ${seastar_cflags}Libs: ${boost_program_options_libs} ${boost_thread_libs} ${c_ares_libs} ${cryptopp_libs} ${fmt_libs} ${seastar_libs}Libs.private: ${dl_libs} ${rt_libs} ${boost_filesystem_libs} ${boost_thread_libs} ${dpdk_libs} ${lksctp_tools_libs} ${numactl_libs} ${stdfilesystem_libs} I see all of the libraries you mentioned there.

Second, you are confusing (if I'm not mistaken) compile-time shared- library resolution with run-time shared-library resolution. The latter is controlled by something called the rpath.

The fmt library is resolved at compile-time by CMake and the library is linked successfully. However, CMake by default clears the rpath when installing the files, since the shared-library can be provided from any location on a system where Seastar is installed.

What this means is that the fmt library has to be installed in location accessible to the linker at run-time: either in a system- standard location like /usr/lib or perhaps in another location with the addition of an environmental variable like LD_LIBRARY_PATH.

I think this issue should be closed in its current form, but if you have any questions about this we can talk either on the mailing-list or in private.

I think it's best to continue the discussion here, for future reference.

After I submitted the issue I had a look at seastar.pc and indeed, most (I'd even say all) of the required dependencies are indeed there. The problem seems to be that pkg-config --libs seastar.pc doesn't list private dependencies, those are exactly the ones I listed as missing. On second though, pkg-config is correct in not listing those, that's why they are private dependencies. Could it be that libseastar.a was not linked against its private dependencies, and now this has to be done by however wants to use libseastar.a?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c5 5493e4bb","name":"GitHub"},"entity":{"external_key":"github/scylladb/ seastar","title":"scylladb/seastar","subtitle":"GitHub repository","main_image_url":" https://github.githubassets.com/images/email/message_cards/header.png","avatar_image_url":"https://github.githubassets.com/images/email/message_cards/avatar.png","action":{"name":"Open in GitHub","url":" https://github.com/scylladb/seastar"}},"updates":{"snippets":[{"icon":"PERSON","message":"@hakuch in #555: This description actually confuses a few different things, and I believe it is incorrect.\r\n\r\nFirst, the seastar.pc file actually does include the libraries that you are describing as missing. This is the contents of a generated file on my machine for the Release build type:\r\n\r\n\r\nprefix=/usr/local\r\nexec_prefix=${prefix}\r\ninc ludedir=${prefix}/include\r\nlibdir=${prefix}/lib64\r\n\r\nName: Seastar\r\nUrl: http://seastar-project.org\r\nDescription: Advanced C++ framework for high-performance server applications on modern hardware.\r\nVersion: 1.0\r\n\r\n# Platform dependencies.\r\ndl_libs=-ldl\r\nstdfilesystem_libs=- lstdc++fs\r\nrt_libs=-lrt\r\n\r\n# Dependencies of dependencies.\r\nboost_system_libs=/home/jhaberku/src/seastar/build/_ cooking/installed/lib/libboost_system.so\r\n\r\n# Dependencies.\r\nboost_cflags=- I/home/jhaberku/src/seastar/build/_cooking/installed/include\r\nboost _filesystem_libs=/home/jhaberku/src/seastar/build/_cooking/installed/ lib/libboost_filesystem.so\r\nboost_program_options_libs=/home/jhaber ku/src/seastar/build/_cooking/installed/lib/libboost_program_options. so\r\nboost_thread_libs=${boost_system_libs} /home/jhaberku/src/seastar/build/_cooking/installed/lib/libboost_thre ad.so\r\nc_ares_cflags=- I/home/jhaberku/src/seastar/build/_cooking/installed/include\r\nc_are s_libs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/libcar es.so\r\ncryptopp_cflags=- I/home/jhaberku/src/seastar/build/_cooking/installed/include\r\ncrypt opp_libs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/libc ryptopp.so\r\ndpdk_libs=/home/jhaberku/src/seastar/build/_cooking/ins talled/lib/librte_cfgfile.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_cmdlin e.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_eal.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_ethdev .a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_hash.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_kvargs .a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_mbuf.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_mempoo l.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_mempoo l_ring.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_bn xt.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_e1 000.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_en a.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_en ic.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_fm 10k.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_qe de.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_i4 0e.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_ix gbe.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_nf p.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_ri ng.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_sf c_efx.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_pmd_vm xnet3_uio.a /home/jhaberku/src/seastar/build/_cooking/installed/lib/librte_ring.a \r\nfmt_cflags=- I/home/jhaberku/src/seastar/build/_cooking/installed/include\r\nfmt_l ibs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/libfmtd.s o.5.2.1\r\nlksctp_tools_cflags=- I/home/jhaberku/src/seastar/build/_cooking/installed/include\r\nlksct p_tools_libs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/ libsctp.so\r\nnumactl_cflags=- I/home/jhaberku/src/seastar/build/_cooking/installed/include\r\nnumac tl_libs=/home/jhaberku/src/seastar/build/_cooking/installed/lib/libnu ma.so\r\nsanitizers_cflags=-fsanitize=address -fsanitize=undefined\r\n\r\n# Us.\r\nseastar_cflags=-I${includedir} -std=gnu++17 -fconcepts -DSEASTAR_HAVE_GCC6_CONCEPTS -DSEASTAR_DEBUG -DSEASTAR_DEBUG_SHARED_PTR -DSEASTAR_ASAN_ENABLED -DSEASTAR_THREAD_STACK_GUARDS -DFMT_SHARED\r\nseastar_libs=${libdir}/libseastar.a\r\n\r\nRequires: liblz4 \u003e= 1.8.0\r\nRequires.private: gnutls \u003e= 3.5.18, protobuf \u003e= 3.3.0, hwloc \u003e= 1.11.5, yaml-cpp \u003e= 0.5.3\r\nConflicts:\r\nCflags: ${boost_cflags} ${c_ares_cflags} ${cryptopp_cflags} ${fmt_cflags} ${lksctp_tools_cflags} ${numactl_cflags} ${sanitizers_cflags} ${seastar_cflags}\r\nLibs: ${boost_program_options_libs} ${boost_thread_libs} ${c_ares_libs} ${cryptopp_libs} ${fmt_libs} ${seastar_libs}\r\nLibs.private: ${dl_libs} ${rt_libs} ${boost_filesystem_libs} ${boost_thread_libs} ${dpdk_libs} ${lksctp_tools_libs} ${numactl_libs} ${stdfilesystem_libs}\r\n\r\n\r\nI see all of the libraries you mentioned there.\r\n\r\nSecond, you are confusing (if I'm not mistaken) compile-time shared-library resolution with run-time shared-library resolution. The latter is controlled by something called the rpath.\r\n\r\nThe fmt library is resolved at compile- time by CMake and the library is linked successfully. However, CMake by default clears the rpath when installing the files, since the shared-library can be provided from any location on a system where Seastar is installed.\r\n\r\nWhat this means is that the fmt library has to be installed in location accessible to the linker at run-time: either in a system-standard location like /usr/lib or perhaps in another location with the addition of an environmental variable like LD_LIBRARY_PATH.\r\n\r\nI think this issue should be closed in its current form, but if you have any questions about this we can talk either on the mailing-list or in private."}],"action":{"name":"View Issue","url":" https://github.com/scylladb/seastar/issues/555#issuecomment-447340529 "}}} [ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": " https://github.com/scylladb/seastar/issues/555#issuecomment-447340529 ", "url": " https://github.com/scylladb/seastar/issues/555#issuecomment-447340529 ", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

hakuch commented 5 years ago

You need to invoke pkg-config with the --static option, since Seastar is a static library. This will bring everything in as expected.

g++ $(pkg-config --libs --cflags --static /path/to/seastar.pc)` foo.cc -o foo

denesb commented 5 years ago

Ok so it was my ignorance at fault after all. :) I will send a patch to add this to HACKING.md.

scylladb / seastar

Cmake generated `seastar.pc` missing most of seastar's dependencies in the link flags #555

Platform dependencies.dl_libs=-ldlstdfilesystem_libs=-

Dependencies of

Dependencies.boost_cflags=-

Us.seastar_cflags=-I${includedir} -std=gnu++17 -fconcepts