paullouisageneau / libdatachannel

C/C++ WebRTC network library featuring Data Channels, Media Transport, and WebSockets
https://libdatachannel.org/
Mozilla Public License 2.0
1.8k stars 361 forks source link

Crash on OPENSSL_sk_pop_free() #1187

Open hudMin opened 5 months ago

hudMin commented 5 months ago

My program crashes on the function OPENSSL_sk_pop_free() with the message: juice poll (53): EXC_BAD_ACCESS (code=1, address=0x8000000000000000). This always seems to happen before the connection to the other peer is completed. Also this only seems to happen in macOS.

paullouisageneau commented 5 months ago

This function is not used directly, and the library doesn't use the OpenSSL stack API directly either. Could you please post a stack trace?

hudMin commented 5 months ago

This is the full call stack:

#0  0x000000011a1d2b4f in OPENSSL_sk_pop_free ()

#1  0x000000011a162335 in EVP_PKEY_free ()

#2  0x0000000119c11a7d in ssl_cert_free ()

#3  0x0000000119c1b17c in SSL_CTX_free ()

#4  0x0000000104e33f85 in rtc::impl::DtlsTransport::DtlsTransport(std::__1::shared_ptr<rtc::impl::IceTransport>, std::__1::shared_ptr<rtc::impl::Certificate>, std::__1::optional<unsigned long>, rtc::CertificateFingerprint::Algorithm, std::__1::function<bool (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)>, std::__1::function<void (rtc::impl::Transport::State)>) ()

#5  0x0000000104e2ec57 in rtc::impl::DtlsSrtpTransport::DtlsSrtpTransport(std::__1::shared_ptr<rtc::impl::IceTransport>, std::__1::shared_ptr<rtc::impl::Certificate>, std::__1::optional<unsigned long>, rtc::CertificateFingerprint::Algorithm, std::__1::function<bool (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)>, std::__1::function<void (std::__1::shared_ptr<rtc::Message>)>, std::__1::function<void (rtc::impl::Transport::State)>) ()

#6  0x0000000104e3f869 in rtc::impl::PeerConnection::initDtlsTransport() ()

#7  0x0000000104e51de2 in std::__1::__function::__func<rtc::impl::PeerConnection::initIceTransport()::$_6, std::__1::allocator<rtc::impl::PeerConnection::initIceTransport()::$_6>, void (rtc::impl::Transport::State)>::operator()(rtc::impl::Transport::State&&) ()

#8  0x0000000104e82baf in rtc::synchronized_callback<rtc::impl::Transport::State>::call(rtc::impl::Transport::State) const ()

#9  0x0000000104e82265 in rtc::impl::Transport::changeState(rtc::impl::Transport::State) ()

#10 0x0000000104e371e4 in rtc::impl::IceTransport::StateChangeCallback(juice_agent*, juice_state, void*) ()

#11 0x0000000104e86850 in agent_bookkeeping ()

#12 0x0000000104e85df9 in agent_conn_update ()

#13 0x0000000104e8b1e0 in conn_poll_process ()

#14 0x0000000104e8b2fb in conn_poll_run ()

#15 0x0000000104e8acfd in conn_thread_entry ()
paullouisageneau commented 5 months ago

What happens here is that an OpenSSL error triggers an exception in the DtlsTransport constructor, then OpenSSL crashes in the catch block when attempting to destroy the OpenSSL context, so you don't even see the error. Commenting out this line should allow you to see the error in the log.

Without more context, it looks like something is broken with the OpenSSL library. What version is it? How was it built? Do you link other libraries in your program?