Segmentation fault on Debian Stretch - Linux armv7

[i96751414]

libtorrent version (or branch): 1.2.6 platform/architecture: linux armv7 (debian stretch) compiler and compiler version: gcc 6.3.0

Hi @arvidn. I have been using libtorrent on some platforms and so far so good. However, recently when I ran it on my RPi3, it crashed (segmentation fault) right away after starting the application. This is not really an issue for me because if I statically build my application using musl based toolchains I don't have this problem. Nevertheless, I would appreciate your input on this, please :)

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
[New Thread 0x66984a80 (LWP 1595)]
[New Thread 0x66183a80 (LWP 1596)]
[New Thread 0x657fea80 (LWP 1597)]
[New Thread 0x64ffda80 (LWP 1598)]
[New Thread 0x645fea80 (LWP 1599)]
[New Thread 0x63dfda80 (LWP 1600)]
[New Thread 0x635dca80 (LWP 1601)]
[New Thread 0x62bfea80 (LWP 1602)]
[New Thread 0x621fea80 (LWP 1603)]

Thread 8 "app" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x635dca80 (LWP 1601)]
0x0084e7d0 in libtorrent::dht::ip_set::erase(boost::asio::ip::address const&)
    ()
(gdb) bt
#0  0x0084e7d0 in libtorrent::dht::ip_set::erase(boost::asio::ip::address const&) ()
#1  0x00850604 in libtorrent::dht::routing_table::add_node_impl(libtorrent::dht::node_entry) ()
#2  0x00850e0a in libtorrent::dht::routing_table::add_node(libtorrent::dht::node_entry const&) ()
#3  0x0085101a in libtorrent::dht::routing_table::heard_about(libtorrent::digest32<160> const&, boost::asio::ip::basic_endpoint<boost::asio::ip::udp> const&) ()
#4  0x008565be in libtorrent::dht::traversal_algorithm::traverse(libtorrent::digest32<160> const&, boost::asio::ip::basic_endpoint<boost::asio::ip::udp> const&) [clone .part.228] ()
#5  0x0085506c in libtorrent::dht::look_for_nodes(char const*, boost::asio::ip::udp const&, libtorrent::bdecode_node const&, std::function<void (libtorrent::dht::node_endpoint const&)>) ()
#6  0x008551bc in libtorrent::dht::traversal_observer::reply(libtorrent::dht::msg const&) ()
#7  0x008e9600 in libtorrent::dht::find_data_observer::reply(libtorrent::dht::msg const&) ()
#8  0x00858c36 in libtorrent::dht::get_peers_observer::reply(libtorrent::dht::msg const&) ()
#9  0x0085378a in libtorrent::dht::rpc_manager::incoming(libtorrent::dht::msg const&, libtorrent::digest32<160>*) ()
#10 0x00848a14 in libtorrent::dht::node::incoming(libtorrent::aux::listen_socket_handle const&, libtorrent::dht::msg const&) ()
#11 0x0083ff1e in libtorrent::dht::dht_tracker::incoming_packet(libtorrent::aux::listen_socket_handle const&, boost::asio::ip::basic_endpoint<boost::asio::ip::udp> const&, libtorrent::span<char const>) ()
#12 0x007a544e in libtorrent::aux::session_impl::on_udp_packet(std::weak_ptr<libtorrent::aux::session_udp_socket>, std::weak_ptr<libtorrent::aux::listen_socket_t>, libtorrent::aux::transport, boost::system::error_code const&) ()
#13 0x007b643e in void libtorrent::aux::allocating_handler<std::_Bind<std::_Mem_fn<void (libtorrent::aux::session_impl::*)(std::weak_ptr<libtorrent::aux::session_udp_socket>, std::weak_ptr<libtorrent::aux::listen_socket_t>, libtorrent::aux::transport, boost::system::error_code const&)> (libtorrent::aux::session_impl*, std::shared_ptr<libtorrent::aux::session_udp_socket>, std::shared_ptr<libtorrent::aux::listen_socket_t>, libtorrent::aux::transport, std::_Placeholder<1>)>, 342u>::operator()<boost::system::error_code const&, unsigned int const&>(boost::system::error_code const&, unsigned int const&) const ()
#14 0x007b6890 in boost::asio::detail::reactive_null_buffers_op<libtorrent::aux::allocating_handler<std::_Bind<std::_Mem_fn<void (libtorrent::aux::session_impl::*)(std::weak_ptr<libtorrent::aux::session_udp_socket>, std::weak_ptr<libtorrent::aux::listen_socket_t>, libtorrent::aux::transport, boost::system::error_code const&)> (libtorrent::aux::session_impl*, std::shared_ptr<libtorrent::aux::session_udp_socket>, std::shared_ptr<libtorrent::aux::listen_socket_t>, libtorrent::aux::transport, std::_Placeholder<1>)>, 342u>, boost::asio::detail::io_object_executor<boost::asio::executor> >::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned int) ()
#15 0x007ad1c4 in boost::asio::detail::epoll_reactor::descriptor_state::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned int) ()
#16 0x00758172 in boost::asio::detail::scheduler::run(boost::system::error_code&) ()
#17 0x00785f10 in std::thread::_State_impl<std::_Bind_simple<libtorrent::session::start(libtorrent::session_params&&, boost::asio::io_context*)::{lambda()#1} ()> >::_M_run() ()
#18 0x76e729dc in ?? () from /usr/lib/arm-linux-gnueabihf/libstdc++.so.6
#19 0x76d88fc4 in start_thread (arg=0x635dca80) at pthread_create.c:458
#20 0x76ce8038 in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:76 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

This seems to me like a platform issue, however I have no clue on whats happening... Thank you for your work!

[userdocs]

I am going to say these issues are one and the same.

https://github.com/qbittorrent/qBittorrent/issues/11882

My solution was to build a static on another platform that can be used on this Debian version.

https://github.com/userdocs/qbittorrent-nox-static

The script i use builds qbittorrent but it is modular so you can create the static libtorrent library using this. It will still build qbittorrent but won't delete the libtorrent library after.

Try it and confirm the build resolves the issue.

~/qbittorrent-nox-static-glibc.sh all -nodel

[FranciscoPombal]

@userdocs

https://github.com/userdocs/qbittorrent-nox-static

Thanks for this, I have added it to a wiki page in qBIttorrent: https://github.com/qbittorrent/qBittorrent/wiki/Compiling-fully-static-binaries-on-Linux-(glibc-or-musl)

[arvidn]

this only happens if you link dynamically against the runtime library then? And the system runtime library is also musl I take it, is that right?

It would be interesting to see a run with sanitizers enabled, I'm not sure Linux/ARMv7 is supported though.

[userdocs]

1: musl builds are not the problem. A fully static musl build on Alpine linux ports fine to platform in question.

2: Sanitize - We already did this https://github.com/qbittorrent/qBittorrent/issues/11882#issuecomment-575162097

3: I don't think it matters if it is static or dynamic. I build the libtorrent library statically at all times.

root@server:~# ldd qbittorrent-build/lib/libtorrent.a
        not a dynamic executable

Here is the result of my completed qBittorrent build to show it is not linked dynacmically.

root@server:~# ldd qbittorrent-build/bin/qbittorrent-nox
        linux-vdso.so.1 (0x00007fffaa5ab000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f48f18d2000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f48f16b5000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f48f1333000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f48f102f000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f48f0e18000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f48f0a79000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f48f4f6d000)

Here is the qBittorrent error revieved.

qBittorrent version: v4.2.5

Caught signal: SIGSEGV
Stack trace:
  qbittorrent-build/bin/qbittorrent-nox : libtorrent::dht::traversal_algorithm::add_entry(libtorrent::digest32<160l> const&, boost::asio::ip::basic_endpoint<boost::asio::ip::udp> const&, libtorrent::flags::bitfield_flag<unsigned char, libtorrent::dht::observer_flags_tag, void>)+0x8d2  [0x55708923f102]
  qbittorrent-build/bin/qbittorrent-nox : ()+0x834448  [0x55708923f448]
  qbittorrent-build/bin/qbittorrent-nox : libtorrent::dht::look_for_nodes(char const*, boost::asio::ip::udp const&, libtorrent::bdecode_node const&, std::function<void (libtorrent::dht::node_endpoint const&)>)+0x180  [0x55708923d4e0]
  qbittorrent-build/bin/qbittorrent-nox : libtorrent::dht::traversal_observer::reply(libtorrent::dht::msg const&)+0x142  [0x55708923d752]
  qbittorrent-build/bin/qbittorrent-nox : libtorrent::dht::find_data_observer::reply(libtorrent::dht::msg const&)+0x20b  [0x5570892ff5db]
  qbittorrent-build/bin/qbittorrent-nox : libtorrent::dht::get_peers_observer::reply(libtorrent::dht::msg const&)+0x1a9  [0x557089303519]
  qbittorrent-build/bin/qbittorrent-nox : libtorrent::dht::rpc_manager::incoming(libtorrent::dht::msg const&, libtorrent::digest32<160l>*)+0x7f0  [0x55708923a180]
  qbittorrent-build/bin/qbittorrent-nox : libtorrent::dht::node::incoming(libtorrent::aux::listen_socket_handle const&, libtorrent::dht::msg const&)+0x20f  [0x557089226d6f]
  qbittorrent-build/bin/qbittorrent-nox : libtorrent::dht::dht_tracker::incoming_packet(libtorrent::aux::listen_socket_handle const&, boost::asio::ip::basic_endpoint<boost::asio::ip::udp> const&, libtorrent::span<char const>)+0x32b  [0x55708921a39b]
  qbittorrent-build/bin/qbittorrent-nox : libtorrent::aux::session_impl::on_udp_packet(std::weak_ptr<libtorrent::aux::session_udp_socket>, std::weak_ptr<libtorrent::aux::listen_socket_t>, libtorrent::aux::transport, boost::system::error_code const&)+0x7b6  [0x5570890eff06]
  qbittorrent-build/bin/qbittorrent-nox : void libtorrent::aux::allocating_handler<std::_Bind<std::_Mem_fn<void (libtorrent::aux::session_impl::*)(std::weak_ptr<libtorrent::aux::session_udp_socket>, std::weak_ptr<libtorrent::aux::listen_socket_t>, libtorrent::aux::transport, boost::system::error_code const&)> (libtorrent::aux::session_impl*, std::shared_ptr<libtorrent::aux::session_udp_socket>, std::shared_ptr<libtorrent::aux::listen_socket_t>, libtorrent::aux::transport, std::_Placeholder<1>)>, 342ul>::operator()<boost::system::error_code const&, unsigned long const&>(boost::system::error_code const&, unsigned long const&) const+0x88  [0x557089116a58]
  qbittorrent-build/bin/qbittorrent-nox : boost::asio::detail::reactive_null_buffers_op<libtorrent::aux::allocating_handler<std::_Bind<std::_Mem_fn<void (libtorrent::aux::session_impl::*)(std::weak_ptr<libtorrent::aux::session_udp_socket>, std::weak_ptr<libtorrent::aux::listen_socket_t>, libtorrent::aux::transport, boost::system::error_code const&)> (libtorrent::aux::session_impl*, std::shared_ptr<libtorrent::aux::session_udp_socket>, std::shared_ptr<libtorrent::aux::listen_socket_t>, libtorrent::aux::transport, std::_Placeholder<1>)>, 342ul>, boost::asio::detail::io_object_executor<boost::asio::executor> >::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long)+0x1d5  [0x5570891170b5]
  qbittorrent-build/bin/qbittorrent-nox : boost::asio::detail::epoll_reactor::descriptor_state::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long)+0x179  [0x557089104d29]
  qbittorrent-build/bin/qbittorrent-nox : boost::asio::detail::scheduler::run(boost::system::error_code&)+0x4c8  [0x55708906f878]
  qbittorrent-build/bin/qbittorrent-nox : ()+0x6aca56  [0x5570890b7a56]
  /usr/lib/x86_64-linux-gnu/libstdc++.so.6 : ()+0xb9e6f  [0x7fc95ad09e6f]
  /lib/x86_64-linux-gnu/libpthread.so.0 : ()+0x74a4  [0x7fc95afd94a4]
  /lib/x86_64-linux-gnu/libc.so.6 : clone()+0x3f  [0x7fc95a47ed0f]
Segmentation fault

Here is a link to the build of libtorrent on Debian stretch with the seg fault.

https://github.com/userdocs/libtorrent/raw/master/libtorrent.a

[i96751414]

If I build my own toolchain (with crosstool-ng) I don't have the segmentation fault... I'm building it with https://github.com/i96751414/cross-compiler/blob/master/docker/linux-armv7.Dockerfile and building libtorrent with https://github.com/i96751414/libtorrent-go/blob/master/docker/linux-armv7.Dockerfile

I suspect this is a bug in gcc version (arm-linux-gnueabihf-gcc-6) used in stretch... The thing I find strange, and the reason for opening this issue, is that this didn't happen in libtorrent 1.1.x

[arvidn]

It could also be some (valid) difference that triggers a bug in libtorrent. Is there an easy way for me to reproduce it? is there a KVM docker image I can use or something like that?

[i96751414]

I believe there is a way of doing that.. I'll try to reproduce it with a docker image and then come back to you.

Edit: Can you try with this? https://hub.docker.com/r/arm32v7/debian It uses qemu, but I think its enough for testing.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.