Closed userdocs closed 4 years ago
@userdocs
undefined symbol: _ZNK10libtorrent5entry4dictEv
Probably the same/similar cause as https://github.com/arvidn/libtorrent/issues/5024. Building libtorrent with C++14 should fix it.
I am pretty sure it is, using this build command
$install_dir/bin/b2" -j$(nproc) fpic=on python="$python_short_version" dht=on encryption=on crypto=openssl i2p=on extensions=on variant=release threading=multi link=static boost-link=static runtime-link=static cxxstd=14 cxxflags="$CXXFLAGS" cflags="$CPPFLAGS" linkflags="$LDFLAGS" toolset=gcc install --prefix="$install_dir"
To expand, i can successfully build qbittorrent using the same build method with no issues and the build path for libtorrent has cxx14 in the path so i am almost certain that all parts of my build are using the 14 standard, ,including boost and libtorrent.
It could be related but i can't see how, with my limited understanding of the source code itself.
The only thing that fixes it is reverting back to 1.2.8.0 as i have not seen any obvious errors when building.
it looks like there's some incompatibility between the main libtorrent library and the python binding. Are you sure both are builds of 1.2.9?
Yes. I build libtorrent myself then create the binding from it. It worked fine until this release.
If you demangle that symbol, its: libtorrent::entry::dict() const
I wonder why this didn't fail in CI. Possibly because CI only uses C++11.
looking at this a bit more, it's not clear why that symbol wouldn't be in the main library. @userdocs Could you list the exported symbols in your libtorrent.dll
? not the python binding, but main libtorrent library.
readelf -Ws libtorrent.a
give me 14 matches to this.
_ZNK10libtorrent5entry4dictB5cxx11Ev
And 0 matches to this
_ZNK10libtorrent5entry4dictEv
This seems to tie into this comment https://github.com/arvidn/libtorrent/issues/5040#issuecomment-678644936 but i don't understand why as i appear to be using cxx14. Here are my settings:
Configure script has finished system check.
Config results:
-=-=-=-=-=-=-=-=-
Package:
name: libtorrent-rasterbar
version: 1.2.9
git revision: a80f458
Build environment:
build system: x86_64-pc-linux-gnu
host system: x86_64-pc-linux-gnu
target system: x86_64-pc-linux-gnu
Compiler and linker flags:
CPPFlags: -I/root/python-libtorrent-build/include -I/root/python-libtorrent-build/include
CFlags: -pthread -g -O2 -fvisibility=hidden
CXXFlags: -std=c++14 -ftemplate-depth=512 -Wno-format-zero-length -fvisibility=hidden -fvisibility-inlines-hidden
LDFlags: -L/root/python-libtorrent-build/lib -Wl,--no-as-needed -ldl -L/root/python-libtorrent-build/lib -lpthread -pthread -fvisibility=hidden -fvisibility-inlines-hidden
Libs: -lboost_system -lpthread
Defs: -DPACKAGE_NAME=\"libtorrent-rasterbar\" -DPACKAGE_TARNAME=\"libtorrent-rasterbar\" -DPACKAGE_VERSION=\"1.2.9\" -DPACKAGE_STRING=\"libtorrent-rasterbar\ 1.2.9\" -DPACKAGE_BUGREPORT=\"arvid@libtorrent.org\" -DPACKAGE_URL=\"http://www.libtorrent.org\" -DPACKAGE=\"libtorrent-rasterbar\" -DVERSION=\"1.2.9\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_PTHREAD_PRIO_INHERIT=1 -DHAVE_PTHREAD=1 -DHAVE_BOOST=/\*\*/ -DHAVE_CXX11=1 -DHAVE_BOOST_SYSTEM=/\*\*/ -DHAVE_CLOCK_GETTIME=1 -DNDEBUG=1 -DTORRENT_USE_INVARIANT_CHECKS=0 -DTORRENT_USE_OPENSSL=1 -DTORRENT_USE_LIBCRYPTO=1 -DHAVE_PYTHON=\"3.7\" -DHAVE_BOOST_PYTHON=/\*\*/ -DHAVE_ICONV=1 -DICONV_CONST= -DBOOST_ASIO_HAS_STD_CHRONO=1 -DBOOST_EXCEPTION_DISABLE=1 -DBOOST_ASIO_ENABLE_CANCELIO=1 -DTORRENT_BUILDING_SHARED=1
Build options:
deprecated functions: yes
debug build: no
invariant checks: no
logging support: yes
Features:
encryption support: yes
dht support: yes
Extra builds:
examples: no
tests: no
python bindings: yes
Pthread library:
CFlags: -pthread
Libs: -lpthread
Boost libraries:
version:
CPPFlags: -I/root/python-libtorrent-build/include
LDFlags: -L/root/python-libtorrent-build/lib
boost.system: -lboost_system
boost.python: -lboost_python37
Python environment:
-automake-
binary: /usr/bin/python3.7
version: 3.7
platform: linux
prefix: ${prefix}
exec_prefix: ${exec_prefix}
pythondir: ${prefix}/lib/python3.7/site-packages
pkgpythondir: ${pythondir}/libtorrent-rasterbar
pyexecdir: ${exec_prefix}/lib/python3.7/site-packages
pkgpyexecdir: ${pyexecdir}/libtorrent-rasterbar
-m4-
cppflags: -I/usr/include/python3.7m
ldflags:
extra libs: -lcrypt -lpthread -ldl -lutil -lm
External libraries:
system libiconv: no
OpenSSL library:
OpenSSL Libs: -lssl -lcrypto
OpenSSL LDFlags: -L/root/python-libtorrent-build/lib
OpenSSL Includes: -I/root/python-libtorrent-build/include
Type 'make' to compile libtorrent-rasterbar 1.2.9
or type 'make V=1' for verbose compiling
and then 'make install' to install it into /usr/local
CXXFLAGS = -std=c++14
LDFLAGS = -Wl,--no-as-needed -ldl -L/root/python-libtorrent-build/lib -lpthread -pthread
OS = LINUX
...patience...
...found 1551 targets...
...updating 414 targets...
could you include a few more lines from the b2
output, right after you cut off?
CXXFLAGS = -std=c++14
LDFLAGS = -Wl,--no-as-needed -ldl -L/root/python-libtorrent-build/lib -lpthread -pthread
OS = LINUX
...patience...
...found 1551 targets...
...updating 414 targets...
common.copy /root/python-libtorrent-build/include/../include/libtorrent/xml_parse.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/write_resume_data.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/web_peer_connection.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/web_connection_base.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/version.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/vector_utils.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/utp_stream.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/utp_socket_manager.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/upnp.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/utf8.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/units.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/union_endpoint.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/udp_tracker_connection.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/udp_socket.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/tracker_manager.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/torrent_status.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/torrent_peer_allocator.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/torrent_peer.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/torrent_info.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/torrent_handle.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/torrent_flags.hpp
common.copy /root/python-libtorrent-build/include/../include/libtorrent/torrent.hpp
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/ed25519/src/add_scalar.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/ed25519/src/fe.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/ed25519/src/ge.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/ed25519/src/key_exchange.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/announce_entry.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/ed25519/src/sc.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/ed25519/src/keypair.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/alert_manager.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/assert.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/ed25519/src/sign.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/bandwidth_limit.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/ed25519/src/verify.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/bandwidth_queue_entry.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/bandwidth_manager.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/bitfield.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/bdecode.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/bloom_filter.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/chained_buffer.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/close_reason.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/cpuid.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/crc32c.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/block_cache.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/alert.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/disk_buffer_holder.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/choker.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/disk_buffer_pool.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/disk_io_job.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/disk_job_fence.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/create_torrent.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/disk_io_thread_pool.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/disk_job_pool.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/error_code.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/entry.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/escape_string.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/string_util.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/disk_io_thread.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/file.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/fingerprint.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/gzip.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/file_storage.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/path.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/hex.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/hasher.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/http_parser.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/identify_client.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/ip_filter.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/http_stream.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/ip_notifier.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/merkle.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/ip_voter.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/platform_util.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/listen_socket_handle.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/web_connection_base.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/http_connection.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/bt_peer_connection.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/http_seed_connection.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/web_peer_connection.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/peer_connection_handle.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/i2p_stream.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/lazy_bdecode.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/instantiate_connection.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/packet_buffer.o
gcc.compile.c++ bin/gcc-8/release/crypto-openssl/cxxstd-14-iso/fpic-on/link-static/threading-multi/src/peer_connection.o
...on 100th target...
and so on.
Are you able to shed any light on this as i am a little lost as to whether the issue is with my build process or the code of this release?
As far as i can see i am using cxx14 in the build process for boost and libtorrent and cannot find any documented solution i am not using to build with cxx14.
I have CXXFLAGS="-std=c++14"
and cxxstd=14
set.
If this is a release issue i will just wait it out.
The mangled symbol you found in libtorrent.a
is:
libtorrent::entry::dict[abi:cxx11]() const
that abi:cxx11
is an ABI tag. My reading of the abi_tag
attribute is that this is attached to this function because the return value is tagged. The return value being std::map
. Here's a post describing abi tags.
So, fundamentally, it still seems the client and the library are built in some slightly ABI incompatible ways.
Btw, the error you get in python suggests that the libtorrent library is not linked statically into the python binding. If you want to do that you have to say libtorrent-link=static
(and possibly boost-link=static
too), on the b2
command line. This could also explain the issue, that perhaps the binding is pulling in an older shared library. ldd
on the python module should tell you.
Can you tell me the command you invoke to build the binding on debian? Perhaps I can reproduce it on ubuntu.
libtorrent-link=static
is undocumented and b2 also throws an error about it not being recognised.
Here is my ldd of both the libtorrent.a
library and the python library. It is about as static as it can be due to glibc not linking statically on these platforms.
root@localhost:~# ldd python-libtorrent-build/lib/libtorrent.a
not a dynamic executable
root@localhost:~# ldd python-libtorrent-build/completed/lib/python3.8/site-packages/libtorrent.cpython-38-x86_64-linux-gnu.so
linux-vdso.so.1 (0x00007ffdb39e9000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fcc4b679000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fcc4b656000)
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fcc4b475000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fcc4b326000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fcc4b30b000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fcc4b119000)
/lib64/ld-linux-x86-64.so.2 (0x00007fcc4c110000)
I use a build script that you can just use on an Ubuntu VM to quickly produce all the relevant libraries (slightly modified from the standard due to the addition of openmpi).
https://gist.github.com/userdocs/fc0c45cfe3200f49ed21e5888bef4367
Relevant lines.
Boost install https://gist.github.com/userdocs/fc0c45cfe3200f49ed21e5888bef4367#file-python-binding-sh-L377
Libtorrent and python
https://gist.github.com/userdocs/fc0c45cfe3200f49ed21e5888bef4367#file-python-binding-sh-L401#L403
Note: the python binding requires the use of ./configure
to successfully build against the libtorrent library. Perhaps the issue arises here somehow.
I think the problem is here
...updated 414 targets...
running install
running build
running build_ext
building 'libtorrent' extension
creating build
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/root
creating build/temp.linux-x86_64-3.8/root/python-libtorrent-build
creating build/temp.linux-x86_64-3.8/root/python-libtorrent-build/libtorrent
creating build/temp.linux-x86_64-3.8/root/python-libtorrent-build/libtorrent/bindings
creating build/temp.linux-x86_64-3.8/root/python-libtorrent-build/libtorrent/bindings/python
creating build/temp.linux-x86_64-3.8/root/python-libtorrent-build/libtorrent/bindings/python/src
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -I/root/python-libtorrent-build/include -fPIC -I../../include -I/root/python-libtorrent-build/include -I/root/python-libtorrent-build/include -I/root/python-libtorrent-build/include -I/root/python-libtorrent-build/include -I/usr/include/python3.8 -c /root/python-libtorrent-build/libtorrent/bindings/python/src/alert.cpp -o build/temp.linux-x86_64-3.8/root/python-libtorrent-build/libtorrent/bindings/python/src/alert.o -DTORRENT_USE_OPENSSL -DTORRENT_USE_LIBCRYPTO -DBOOST_ASIO_HAS_STD_CHRONO=1 -DBOOST_EXCEPTION_DISABLE -DBOOST_ASIO_ENABLE_CANCELIO -DTORRENT_LINKING_SHARED -std=c++1y -ftemplate-depth=512 -Wno-format-zero-length -fvisibility=hidden -fvisibility-inlines-hidden -Wno-deprecated-declarations -g0 -Os -std=c++11
Specifically this -std=c++11
when the python binding starts.
I introduced that in my testing, the normal command also has -std=c++14
.
I have created a working build by switching to b2
for the building the python module instead of python setup install
method.
This creates a working library with which solves the original issue but does not explain it.
"$install_dir/bin/b2" -j$(nproc) fpic=on python="$python_short_version" dht=on encryption=on crypto=openssl i2p=on extensions=on variant=release threading=multi libtorrent-link=static boost-link=static cxxstd=14 cxxflags="$CXXFLAGS" cflags="$CPPFLAGS" linkflags="$LDFLAGS" toolset=gcc stage_module
Please provide the following information
1.2.9
platform/architecture:
Debian 10 amd64
please describe what symptom you see, what you would expect to see instead and how to reproduce it.
Build the python binding against 1.29 and there is a runtime fault when trying to start deluge.
This issue does not occur when i use 1.2.8 instead of 1.2.9. It is new and unique to this release.