"undefined reference to `archive_read_new"

[mickey298]

os

fresh new vm

uname -a
Linux archlinuxtest 6.7.5-arch1-1 #1 SMP PREEMPT_DYNAMIC Sat, 17 Feb 2024 14:02:33 +0000 x86_64 GNU/Linux

seastar commit

f0b2971a24ab7716933a720572207fa9a1df38bd

---

configure and build

./configure.py --mode=release --enable-dpdk --prefix=/usr

ninja -C build/release

error:

ninja: Entering directory `build/release'
[104/351] Linking CXX executable tests/perf/fstream_perf
FAILED: tests/perf/fstream_perf 
: && /usr/bin/g++ -O2 -g -DNDEBUG  tests/perf/CMakeFiles/test_perf_fstream.dir/fstream_perf.cc.o -o tests/perf/fstream_perf  libseastar.a  /usr/lib/libboost_program_options.so  /usr/lib/libboost_thread.so  /usr/lib/libboost_chrono.so  /usr/lib/libboost_atomic.so  /usr/lib/libcares.so  /usr/lib/libfmt.so.10.2.0  /usr/lib/liblz4.so  -ldl  /usr/lib/libgnutls.so  -latomic  /usr/lib/libsctp.so  /usr/lib/libprotobuf.so  /usr/lib/libyaml-cpp.so  /usr/lib/libhwloc.so  /usr/lib/liburing.so  /usr/lib/libnuma.so && :
/usr/bin/ld: libseastar.a(dpdk.o): in function `firmware_read':
eal_firmware.c:(.text+0x2c6f2): undefined reference to `archive_read_new'
/usr/bin/ld: eal_firmware.c:(.text+0x2c706): undefined reference to `archive_read_support_format_raw'
/usr/bin/ld: eal_firmware.c:(.text+0x2c716): undefined reference to `archive_read_support_filter_xz'
/usr/bin/ld: eal_firmware.c:(.text+0x2c72e): undefined reference to `archive_read_open_filename'
/usr/bin/ld: eal_firmware.c:(.text+0x2c741): undefined reference to `archive_read_next_header'
/usr/bin/ld: eal_firmware.c:(.text+0x2c795): undefined reference to `archive_read_data'
/usr/bin/ld: eal_firmware.c:(.text+0x2c7c2): undefined reference to `archive_read_free'
/usr/bin/ld: eal_firmware.c:(.text+0x2c7e8): undefined reference to `archive_read_free'
collect2: error: ld returned 1 exit status
[106/351] Building CXX object tests/perf/CMakeFiles/test_perf_shared_token_bucket.dir/shared_token_bucket.cc.o
ninja: build stopped: subcommand failed.

issue still exists after installing libarchive.

@tchaikov

[tchaikov]

if we enable dpdk when building seastar, dpdk pulls libarchive in if the latter is installed in the system. and we do not use libdpdk.pc for calculating the ldflags exposed by seastar library. because we want to statically link against libdpdk, we just pull the .a archives manually with a hardwired name list, but we missed the libraries linked by it.

[tchaikov]

@mickey298 please add the information on how you configure seastar, and how dpdk is involved in the build.

[mickey298]

@mickey298 please add the information on how you configure seastar, and how dpdk is involved in the build. modified.

[mickey298]

I solved the problem temporarily by add ” target_link_libraries(seastar_private INTERFACE -larchive)" in the CMakeLists.txt. and continue building. an new issues appeared.

[1/16] Linking CXX executable apps/httpd/httpd
FAILED: apps/httpd/httpd
: && /usr/bin/g++ -O2 -g -DNDEBUG  apps/httpd/CMakeFiles/app_httpd.dir/demo.json.cc.o apps/httpd/CMakeFiles/app_httpd.dir/main.cc.o -o apps/httpd/httpd  libseastar.a  /usr/lib/libboost_program_options.so  /usr/lib/libboost_thread.so  /usr/lib/libboost_chrono.so  /usr/lib/libboost_atomic.so  /usr/lib/libcares.so  /usr/lib/libfmt.so.10.2.0  /usr/lib/liblz4.so  -ldl  /usr/lib/libgnutls.so  -latomic  /usr/lib/libsctp.so  /usr/lib/libprotobuf.so  /usr/lib/libyaml-cpp.so  /usr/lib/libhwloc.so  /usr/lib/liburing.so  /usr/lib/libnuma.so  -larchive && :
/usr/bin/ld: libseastar.a(prometheus.cc.o): undefined reference to symbol '_ZN4absl12lts_2023080212log_internal17MakeCheckOpStringIllEEPNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEET_T0_PKc'
/usr/bin/ld: /usr/lib/libabsl_log_internal_check_op.so.2308.0.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
[3/16] Linking CXX executable apps/io_tester/io_tester
ninja: build stopped: subcommand failed.

confirm the file the symbol:

❯ ls /usr/lib/libabsl_log_internal_check_op.so -alh                                                       
lrwxrwxrwx 1 root root 41 Sep 19 04:50 /usr/lib/libabsl_log_internal_check_op.so -> libabsl_log_internal_check_op.so.2308.0.0

❯ nm -D /usr/lib/libabsl_log_internal_check_op.so |rg "_ZN4absl12lts_2023080212log_internal17MakeCheckOpStringIllEEPNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEET_T0_PKc"
0000000000003b50 T _ZN4absl12lts_2023080212log_internal17MakeCheckOpStringIllEEPNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEET_T0_PKc

it seems a protobuf version problem from these information. i didn't sovled this issue eventually .maybe i need to wait for the official update? if anyone solved this issue,pls let me know about it. FindProtobuf does not look for abseil dependency ONNX 1.15.0: missing symbols (absl) onnx_gtests #5740 Unable to link absl log internal

Thank you for your help. @tchaikov
Are you going to add this ( target_link_libraries(seastar_private INTERFACE -larchive)) into master branch? maybe a better way? I hope close this issue soon If there are no other questions.

[tchaikov]

so there are two issues:

1. libarchive linkage pulled in by dpdk
2. absl libraries linkage pulled in by protobuf >= 22.0

regarding the 2nd one, before CMake upstream fixes it, we need to find a workaround.

[mickey298]

@tchaikov Still had issues. After building with same error message,i delete QUIET keyword and got this warning:

CMake Warning at cmake/SeastarDependencies.cmake:162 (find_package):
  Could not find a configuration file for package "Protobuf" that is
  compatible with requested version "2.5.0".

  The following configuration files were considered but not accepted:

    /usr/lib/cmake/protobuf/protobuf-config.cmake, version: 25.3.0
    /lib/cmake/protobuf/protobuf-config.cmake, version: 25.3.0

Call Stack (most recent call first):
  CMakeLists.txt:395 (seastar_find_dependencies)

-- Found Protobuf: /usr/lib/libprotobuf.so (found suitable version "4.25.3", minimum required is "2.5.0") 

At first,i thougt it's version problems. So i delete the version constraint(2.5.0),and continue build with this command：

./configure.py --mode=release --enable-dpdk --prefix=/usr

the configure works fine. and i got this error after building:

$ ninja -C build/release -j3
ninja: Entering directory `build/release'
[1/346] Generating gen/src/proto/metrics2.pb.h, gen/src/proto/metrics2.pb.cc
FAILED: gen/src/proto/metrics2.pb.h gen/src/proto/metrics2.pb.cc /home/warren/seastar/build/release/gen/src/proto/metrics2.pb.h /home/warren/seastar/build/release/gen/src/proto/metrics2.pb.cc 
cd /home/warren/seastar/build/release && /usr/bin/cmake -E make_directory /home/warren/seastar/build/release/gen/src/proto && --cpp_out=/home/warren/seastar/build/release/gen/src/proto -I/home/warren/seastar/src/proto /home/warren/seastar/src/proto/metrics2.proto
/bin/sh: line 1: --cpp_out=/home/warren/seastar/build/release/gen/src/proto: No such file or directory
[3/346] Generating catest.key
ninja: build stopped: subcommand failed.

PS. provide my envrionment in case someone try to reproduce the issue:

$uname -a
Linux archlinuxtest 6.7.8-arch1-1 #1 SMP PREEMPT_DYNAMIC Sun, 03 Mar 2024 00:30:36 +0000 x86_64 GNU/Linux

$gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-pc-linux-gnu/13.2.1/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /build/gcc/src/gcc/configure --enable-languages=ada,c,c++,d,fortran,go,lto,m2,objc,obj-c++ --enable-bootstrap --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://bugs.archlinux.org/ --with-build-config=bootstrap-lto --with-linker-hash-style=gnu --with-system-zlib --enable-__cxa_atexit --enable-cet=auto --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-default-ssp --enable-gnu-indirect-function --enable-gnu-unique-object --enable-libstdcxx-backtrace --enable-link-serialization=1 --enable-linker-build-id --enable-lto --enable-multilib --enable-plugin --enable-shared --enable-threads=posix --disable-libssp --disable-libstdcxx-pch --disable-werror
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 13.2.1 20230801 (GCC)

seastar version: commit e6799055c8cacd2dc15c3037c7181831b2e48d79

[mickey298]

Warining @ddef7a2 -- Found yaml-cpp: /usr/lib/libyaml-cpp.so (Required is at least version "0.5.1") CMake Warning (dev) at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:447 (message): find_package() specify a version range but the module Protobuf does not support this capability. Only the lower endpoint of the range will be used. Call Stack (most recent call first): /usr/share/cmake/Modules/FindProtobuf.cmake:749 (FIND_PACKAGE_HANDLE_STANDARD_ARGS) cmake/SeastarDependencies.cmake:169 (find_package) CMakeLists.txt:395 (seastar_find_dependencies) This warning is for project developers. Use -Wno-dev to suppress it

[tchaikov]

@mickey298 while you were testing. i just sent a fix for it.

[mickey298]

@mickey298 while you were testing. i just sent a fix for it.

Ok. I was in a hurry and i will testing it again after a few minutes.

[tchaikov]

@mickey298 thank you! i don't have an environment for testing the protobuf cmake configuration file at this moment. neither of my linux boxes have protobuf packaged with the configuration file installed: debian sid and f38. would be great if you could test again after removing QUIET if the latest revision does not work for you.

[mickey298]

@mickey298 thank you! i don't have an environment for testing the protobuf cmake configuration file at this moment. neither of my linux boxes have protobuf packaged with the configuration file installed: debian sid and f38. would be great if you could test again after removing QUIET if the latest revision does not work for you.

My pleasure. Have you finished？

[tchaikov]

@mickey298 yup, i've pushed the latest revision to #2132.

[mickey298]

@mickey298 yup, i've pushed the latest revision to #2132.

I got this.

CMake Warning at cmake/SeastarDependencies.cmake:162 (find_package):
  Could not find a configuration file for package "Protobuf" that is
  compatible with requested version range "22.0...25.3".

  The following configuration files were considered but not accepted:

    /usr/lib/cmake/protobuf/protobuf-config.cmake, version: 25.3.0
    /lib/cmake/protobuf/protobuf-config.cmake, version: 25.3.0

Call Stack (most recent call first):
  CMakeLists.txt:395 (seastar_find_dependencies)

-- Found Protobuf: /usr/lib/libprotobuf.so (found suitable version "4.25.3", minimum required is "2.5.0")

So i modified to 22.0...25.3.0,and i got this:

CMake Warning at cmake/SeastarDependencies.cmake:162 (find_package):
  Could not find a configuration file for package "Protobuf" that is
  compatible with requested version range "22.0...25.3.0".

  The following configuration files were considered but not accepted:

    /usr/lib/cmake/protobuf/protobuf-config.cmake, version: 25.3.0
    /lib/cmake/protobuf/protobuf-config.cmake, version: 25.3.0

Call Stack (most recent call first):
  CMakeLists.txt:395 (seastar_find_dependencies)

-- Found Protobuf: /usr/lib/libprotobuf.so (found suitable version "4.25.3", minimum required is "2.5.0")

The same error with 22.0.0...25.3.0. But Only use 25.3.0, compile succeesed.

Do you feel this is too strict? What if the version of protobuf upgrade to 25.4.0? Can we have a better way? @tchaikov

[tchaikov]

@mickey298 thanks. now i understand the problem better. and yeah, it's too strict. so i revised the patch again. see #2132 . could you take another look?

[mickey298]

from 0d4499 No warning.No error. Perfect result. Thank you for your hard work. @tchaikov

[tchaikov]

@mickey298 thank you for testing and the great analysis! without your help, i would not be able to come up with this fix =)

[tchaikov]

2132 should address this.