search

arvidn / libtorrent

an efficient feature complete C++ bittorrent implementation
http://libtorrent.org
Other
5.09k stars 983 forks source link

Getting crashes (SIGABRT) on NetBSD 10.0 + Python 3.7 #7694

Open hellium6 opened 3 days ago

hellium6 commented 3 days ago

libtorrent version (or branch): v2.0.10

platform/architecture: NetBSD 10.0 amd64

compiler and compiler version: gcc (nb3 20231008) 10.5.0

My process is described in detail below. When I run python3.7 -c 'import libtorrent' it shows Abort trap (core dumped)

Here is a gdb backtrace:

#0  0x00007d2cd0f7e74a in _lwp_kill () from /usr/lib/libc.so.12
#1  0x00007d2cd0e48fa4 in __libc_mutex_unlock_stub () from /usr/lib/libc.so.12
#2  0x00007d2ccfc16e93 in boost::asio::ssl::detail::openssl_init_base::instance() () from /home/user/.local/lib/python3.7/site-packages/libtorrent-0.0.0-py3.7-netbsd-10.0-amd64.egg/libtorrent/__init__.so
#3  0x00007d2ccfbfa5ce in _GLOBAL__sub_I_converters.cpp () from /home/user/.local/lib/python3.7/site-packages/libtorrent-0.0.0-py3.7-netbsd-10.0-amd64.egg/libtorrent/__init__.so
#4  0x00007f7f3ba06416 in _rtld_call_init_function () from /usr/libexec/ld.elf_so
#5  0x00007f7f3ba0675c in _rtld_call_init_functions () from /usr/libexec/ld.elf_so
#6  0x00007f7f3ba07698 in dlopen () from /usr/libexec/ld.elf_so
#7  0x00007d2cd2966370 in _PyImport_FindSharedFuncptr () from /usr/pkg/lib/libpython3.7.so.1.0
#8  0x00007d2cd293df27 in _PyImport_LoadDynamicModuleWithSpec () from /usr/pkg/lib/libpython3.7.so.1.0
#9  0x00007d2cd293c1d8 in _imp_create_dynamic () from /usr/pkg/lib/libpython3.7.so.1.0
#10 0x00007d2cd28826af in _PyMethodDef_RawFastCallDict () from /usr/pkg/lib/libpython3.7.so.1.0
#11 0x00007d2cd288277a in _PyCFunction_FastCallDict () from /usr/pkg/lib/libpython3.7.so.1.0
#12 0x00007d2cd29c08c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.7.so.1.0
#13 0x00007d2cd29262de in _PyEval_EvalCodeWithName () from /usr/pkg/lib/libpython3.7.so.1.0
#14 0x00007d2cd2881af9 in _PyFunction_FastCallKeywords () from /usr/pkg/lib/libpython3.7.so.1.0
#15 0x00007d2cd29b87d2 in call_function () from /usr/pkg/lib/libpython3.7.so.1.0
#16 0x00007d2cd29bf90a in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.7.so.1.0
#17 0x00007d2cd29b76e8 in function_code_fastcall () from /usr/pkg/lib/libpython3.7.so.1.0
#18 0x00007d2cd29b87d2 in call_function () from /usr/pkg/lib/libpython3.7.so.1.0
#19 0x00007d2cd29be0c1 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.7.so.1.0
#20 0x00007d2cd29b76e8 in function_code_fastcall () from /usr/pkg/lib/libpython3.7.so.1.0
#21 0x00007d2cd29b87d2 in call_function () from /usr/pkg/lib/libpython3.7.so.1.0
#22 0x00007d2cd29bd1c0 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.7.so.1.0
#23 0x00007d2cd29b76e8 in function_code_fastcall () from /usr/pkg/lib/libpython3.7.so.1.0
#24 0x00007d2cd29b87d2 in call_function () from /usr/pkg/lib/libpython3.7.so.1.0
#25 0x00007d2cd29bd1c0 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.7.so.1.0
#26 0x00007d2cd29b76e8 in function_code_fastcall () from /usr/pkg/lib/libpython3.7.so.1.0
#27 0x00007d2cd29b87d2 in call_function () from /usr/pkg/lib/libpython3.7.so.1.0
#28 0x00007d2cd29bd1c0 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.7.so.1.0
#29 0x00007d2cd29b76e8 in function_code_fastcall () from /usr/pkg/lib/libpython3.7.so.1.0
#30 0x00007d2cd2883487 in object_vacall () from /usr/pkg/lib/libpython3.7.so.1.0
#31 0x00007d2cd2883730 in _PyObject_CallMethodIdObjArgs () from /usr/pkg/lib/libpython3.7.so.1.0
#32 0x00007d2cd293d104 in PyImport_ImportModuleLevelObject () from /usr/pkg/lib/libpython3.7.so.1.0
#33 0x00007d2cd29c0a21 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.7.so.1.0
#34 0x00007d2cd29262de in _PyEval_EvalCodeWithName () from /usr/pkg/lib/libpython3.7.so.1.0
#35 0x00007d2cd2926576 in PyEval_EvalCodeEx () from /usr/pkg/lib/libpython3.7.so.1.0
#36 0x00007d2cd29265c0 in PyEval_EvalCode () from /usr/pkg/lib/libpython3.7.so.1.0
#37 0x00007d2cd294ef3a in run_mod () from /usr/pkg/lib/libpython3.7.so.1.0
#38 0x00007d2cd295076e in PyRun_StringFlags () from /usr/pkg/lib/libpython3.7.so.1.0
#39 0x00007d2cd29507cc in PyRun_SimpleStringFlags () from /usr/pkg/lib/libpython3.7.so.1.0
#40 0x00007d2cd296c4fd in pymain_main () from /usr/pkg/lib/libpython3.7.so.1.0
#41 0x00007d2cd296cd26 in _Py_UnixMain () from /usr/pkg/lib/libpython3.7.so.1.0
#42 0x0000000066400aad in _start ()

The same process followed for Python 3.11.8 from NetBSD repos did not crash.

I need v2.0.6 to work, but for reporting I'm going with the latest release, v2.0.10.

I installed python 3.7.17 from this commit of NetBSD/pkgsrc. Apart from adding some CFLAGS and LDFLAGS in Makefile there is not much difference. Something like:

git clone --depth 1 -b 01db152b21d3830339e882468c3c0451ffe918ca https://github.com/NetBSD/pkgsrc
cd pkgsrc/lang/python37
bmake package
doas bmake install || doas bmake replace
Details on build here ```sh $ uname -mrs NetBSD 10.0 amd64 cd ~ mkdir test cd test git clone --depth 1 -b OpenSSL_1_1_1w https://github.com/openssl/openssl cd openssl git switch -c clone #fix detached HEAD git submodule update --init --recursive --depth 1 ./config -fPIC -shared --prefix=/usr/local/custom gmake #GNU Make doas gmake all install_sw cd .. $ cc --version cc (nb3 20231008) 10.5.0 ... $ cat ~/user-config.jam using python : 3.7 ; using gcc : fail : /dev/null : ; # It will fail. To test if this is ever selected (wrongfully). # gnu++14 for NetBSD alloca() GNU extensions - https://github.com/tstack/lnav/issues/1158 using gcc : test : g++ : -std=gnu++14 -fPIC openssl -I/usr/local/custom/include -L/usr/local/custom/lib -Wl,-R/usr/local/custom/lib ; $ gcc --version gcc (nb3 20231008) 10.5.0 $ python3.7 --version Python 3.7.17 curl -LO https://github.com/boostorg/boost/releases/download/boost-1.84.0/boost-1.84.0.tar.xz tar -xvf boost-1.84.0.tar.xz cd boost-1.84.0 export BOOST_ROOT=$PWD export BOOST_BUILD_PATH=$BOOST_ROOT/tools/build cd $BOOST_BUILD_PATH ./bootstrap.sh doas ./b2 install cd $BOOST_ROOT b2 toolset=gcc-test pch=off variant=debug doas b2 toolset=gcc-test pch=off variant=debug install --prefix=/usr/local/custom curl -LO https://github.com/arvidn/libtorrent/releases/download/v2.0.10/libtorrent-rasterbar-2.0.10.tar.gz tar -xvf libtorrent-rasterbar-2.0.10.tar.gz cd libtorrent-rasterbar-2.0.10 # I) fix for pthread_np.h not found $ cat fix.patch This patch is a fix for "fatal error: pthread_np.h: No such file or directory" on NetBSD Based on: https://github.com/NetBSD/pkgsrc/commit/2d1e5065b5bdeec046025bf4bf25a37cd344cd64#diff-ef0645b95bcbcc7ceee1cf25ce8b42799ce0b36157a0ea20252e9170d0fa3a45 --- src/platform_util.cpp.orig 2024-06-25 12:02:05.623242181 +0000 +++ src/platform_util.cpp 2024-06-25 12:04:13.293242181 +0000 @@ -40,7 +40,7 @@ #if TORRENT_HAS_PTHREAD_SET_NAME #include -#ifdef TORRENT_BSD +#if defined(TORRENT_BSD) && !defined(__NetBSD__) #include #endif #endif @@ -105,7 +105,11 @@ TORRENT_UNUSED(name); #if TORRENT_HAS_PTHREAD_SET_NAME #ifdef TORRENT_BSD +#if defined(__NetBSD__) + pthread_setname_np(pthread_self(), name, nullptr); +#else pthread_set_name_np(pthread_self(), name); +#endif #else pthread_setname_np(pthread_self(), name); #endif $ patch < fix.patch # II)Without these, cxxstd=14 (c++14) is set and gnu++14 cannot be used. GCC GNU # Extensions are needed for NetBSD alloca() as described above. sed -i'.bak' -e '/local CXXVER/,+5d' -e 's/14//g' Jamfile sed -i'.bak' -e 's/14//g' bindings/python/Jamfile # III) Also comment library_dirs related lines to ensure that our "library-path" # is used for lssl and lcrypt. library-path would then add our # /usr/local/custom before other "-L"s. For the rest we'll add a -L/usr/pkg/lib # in the build_ext command at the end just in case. sed -i'.bak' -e '/for path in self/ s/./#&/' -e '/f"library-path/ s/./#&/' bindings/python/setup.py b2 toolset=gcc-test variant=debug link=shared crypto=openssl warnings=off address-model=64 doas b2 toolset=gcc-test variant=debug link=shared crypto=openssl warnings=off address-model=64 install --prefix=/usr/local/custom python3.7 setup.py build_ext --b2-args="toolset=gcc-test variant=debug crypto=openssl cxxflags=-I/usr/local/custom/include cxxflags=-L/usr/local/custom/lib cxxflags=-Wl,-R/usr/local/custom/lib cxxflags=-L/usr/pkg/lib library-path=/usr/local/custom/lib" install --prefix=~/.local ``` It ended with: ``` Extracting libtorrent-0.0.0-py3.7-netbsd-10.0-amd64.egg to /home/user/.local/lib/python3.7/site-packages Adding libtorrent 0.0.0 to easy-install.pth file Installed /home/user/.local/lib/python3.7/site-packages/libtorrent-0.0.0-py3.7-netbsd-10.0-amd64.egg Processing dependencies for libtorrent==0.0.0 Finished processing dependencies for libtorrent==0.0.0 ``` Details about installed files: ``` $ tree ~/.local/lib/python3.7/site-packages/libtorrent-0.0.0-py3.7-netbsd-10.0-amd64.egg/ /home/user/.local/lib/python3.7/site-packages/libtorrent-0.0.0-py3.7-netbsd-10.0-amd64.egg/ |-- EGG-INFO | |-- PKG-INFO | |-- SOURCES.txt | |-- dependency_links.txt | |-- native_libs.txt | |-- not-zip-safe | `-- top_level.txt `-- libtorrent |-- __init__.py |-- __init__.so `-- __pycache__ `-- __init__.cpython-37.pyc 3 directories, 9 files $ ldd ~/.local/lib/python3.7/site-packages/libtorrent-0.0.0-py3.7-netbsd-10.0-amd64.egg/libtorrent/__init__.so /home/user/.local/lib/python3.7/site-packages/libtorrent-0.0.0-py3.7-netbsd-10.0-amd64.egg/libtorrent/__init__.so: -lssl.1.1 => /usr/local/custom/lib/libssl.so.1.1 -lcrypto.1.1 => /usr/local/custom/lib/libcrypto.so.1.1 -lpthread.1 => /usr/lib/libpthread.so.1 -lc.12 => /usr/lib/libc.so.12 -lutil.7 => /usr/lib/libutil.so.7 -lstdc++.9 => /usr/lib/libstdc++.so.9 -lm.0 => /usr/lib/libm.so.0 -lgcc_s.1 => /usr/lib/libgcc_s.so.1 ``` Test: ```sh $ python3.7 -c 'import libtorrent' Abort trap (core dumped) # After repeating the build for python 3.11 $ python3.11 -c 'import libtorrent' $ # no crash ``` bt ``` $ gdb -ex "set pagination off" -ex "backtrace" -ex "quit" `which python3.7` python3.7.core GNU gdb (GDB) 11.0.50.20200914-git ... Reading symbols from /usr/pkg/bin/python3.7... (No debugging symbols found in /usr/pkg/bin/python3.7) [New process 21096] Core was generated by `python3.7'. Program terminated with signal SIGABRT, Aborted. #0 0x000073f26cb7e74a in _lwp_kill () from /usr/lib/libc.so.12 #0 0x000073f26cb7e74a in _lwp_kill () from /usr/lib/libc.so.12 #1 0x000073f26ca48fa4 in __libc_mutex_unlock_stub () from /usr/lib/libc.so.12 #2 0x000073f26ab78213 in __gthread_active_p () at /usr/include/g++/bits/gthr-posix.h:201 #3 0x000073f26b00be26 in __gnu_cxx::__atomic_add_dispatch (__val=1, __mem=0x73f26dffa908) at /usr/include/g++/ext/atomicity.h:94 #4 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_add_ref_copy (this=0x73f26dffa900) at /usr/include/g++/bits/shared_ptr_base.h:142 #5 0x000073f26b00b6d1 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count (this=0x73f26bef8c48 ::instance_+8>, __r=...) at /usr/include/g++/bits/shared_ptr_base.h:743 #6 0x000073f26b024c61 in std::__shared_ptr::__shared_ptr (this=0x73f26bef8c40 ::instance_>) at /usr/include/g++/bits/shared_ptr_base.h:1186 #7 0x000073f26b024c8b in std::shared_ptr::shared_ptr (this=0x73f26bef8c40 ::instance_>) at /usr/include/g++/bits/shared_ptr.h:149 #8 0x000073f26b024d3b in boost::asio::ssl::detail::openssl_init_base::instance () at /usr/local/custom/include/boost/asio/ssl/detail/impl/openssl_init.ipp:152 #9 0x000073f26b025ff5 in boost::asio::ssl::detail::openssl_init::openssl_init (this=0x73f26bef8c40 ::instance_>) at /usr/local/custom/include/boost/asio/ssl/detail/openssl_init.hpp:60 #10 0x000073f26ab798d9 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at /usr/local/custom/include/boost/asio/ssl/detail/openssl_init.hpp:90 #11 0x000073f26ab79c9c in _GLOBAL__sub_I_converters.cpp(void) () at src/converters.cpp:559 #12 0x00007f7ec4c06416 in _rtld_call_init_function () from /usr/libexec/ld.elf_so #13 0x00007f7ec4c0675c in _rtld_call_init_functions () from /usr/libexec/ld.elf_so #14 0x00007f7ec4c07698 in dlopen () from /usr/libexec/ld.elf_so #15 0x000073f26e566370 in _PyImport_FindSharedFuncptr () from /usr/pkg/lib/libpython3.7.so.1.0 #16 0x000073f26e53df27 in _PyImport_LoadDynamicModuleWithSpec () from /usr/pkg/lib/libpython3.7.so.1.0 #17 0x000073f26e53c1d8 in _imp_create_dynamic () from /usr/pkg/lib/libpython3.7.so.1.0 #18 0x000073f26e4826af in _PyMethodDef_RawFastCallDict () from /usr/pkg/lib/libpython3.7.so.1.0 #19 0x000073f26e48277a in _PyCFunction_FastCallDict () from /usr/pkg/lib/libpython3.7.so.1.0 #20 0x000073f26e5c08c8 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.7.so.1.0 #21 0x000073f26e5262de in _PyEval_EvalCodeWithName () from /usr/pkg/lib/libpython3.7.so.1.0 #22 0x000073f26e481af9 in _PyFunction_FastCallKeywords () from /usr/pkg/lib/libpython3.7.so.1.0 #23 0x000073f26e5b87d2 in call_function () from /usr/pkg/lib/libpython3.7.so.1.0 #24 0x000073f26e5bf90a in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.7.so.1.0 #25 0x000073f26e5b76e8 in function_code_fastcall () from /usr/pkg/lib/libpython3.7.so.1.0 #26 0x000073f26e5b87d2 in call_function () from /usr/pkg/lib/libpython3.7.so.1.0 #27 0x000073f26e5be0c1 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.7.so.1.0 #28 0x000073f26e5b76e8 in function_code_fastcall () from /usr/pkg/lib/libpython3.7.so.1.0 #29 0x000073f26e5b87d2 in call_function () from /usr/pkg/lib/libpython3.7.so.1.0 #30 0x000073f26e5bd1c0 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.7.so.1.0 #31 0x000073f26e5b76e8 in function_code_fastcall () from /usr/pkg/lib/libpython3.7.so.1.0 #32 0x000073f26e5b87d2 in call_function () from /usr/pkg/lib/libpython3.7.so.1.0 #33 0x000073f26e5bd1c0 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.7.so.1.0 #34 0x000073f26e5b76e8 in function_code_fastcall () from /usr/pkg/lib/libpython3.7.so.1.0 #35 0x000073f26e5b87d2 in call_function () from /usr/pkg/lib/libpython3.7.so.1.0 #36 0x000073f26e5bd1c0 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.7.so.1.0 #37 0x000073f26e5b76e8 in function_code_fastcall () from /usr/pkg/lib/libpython3.7.so.1.0 #38 0x000073f26e483487 in object_vacall () from /usr/pkg/lib/libpython3.7.so.1.0 #39 0x000073f26e483730 in _PyObject_CallMethodIdObjArgs () from /usr/pkg/lib/libpython3.7.so.1.0 #40 0x000073f26e53d104 in PyImport_ImportModuleLevelObject () from /usr/pkg/lib/libpython3.7.so.1.0 #41 0x000073f26e5c0a21 in _PyEval_EvalFrameDefault () from /usr/pkg/lib/libpython3.7.so.1.0 #42 0x000073f26e5262de in _PyEval_EvalCodeWithName () from /usr/pkg/lib/libpython3.7.so.1.0 #43 0x000073f26e526576 in PyEval_EvalCodeEx () from /usr/pkg/lib/libpython3.7.so.1.0 #44 0x000073f26e5265c0 in PyEval_EvalCode () from /usr/pkg/lib/libpython3.7.so.1.0 #45 0x000073f26e54ef3a in run_mod () from /usr/pkg/lib/libpython3.7.so.1.0 #46 0x000073f26e55076e in PyRun_StringFlags () from /usr/pkg/lib/libpython3.7.so.1.0 #47 0x000073f26e5507cc in PyRun_SimpleStringFlags () from /usr/pkg/lib/libpython3.7.so.1.0 #48 0x000073f26e56c4fd in pymain_main () from /usr/pkg/lib/libpython3.7.so.1.0 #49 0x000073f26e56cd26 in _Py_UnixMain () from /usr/pkg/lib/libpython3.7.so.1.0 #50 0x00000000f0000aad in _start () ```