Crash using Boost.Asio sockets after upgrading to latest code

[jdale88]

I'm not sure if this is related to WebSocket++ at all, but I've been looking at it all evening and figured I'd at least ask to see if anyone else has any thoughts, or has seen something similar.

Since upgrading from the legacy branch to the current master branch, I get a crash as soon as I try to use a Boost.Asio socket of any kind. The crash seems to be due to it trying to take a lock on a critical section which appears to contain invalid data.

This is the callstack (this callstack is when trying to create the WebSocket++ server):

boost::asio::detail::win_mutex::lock()  Line 50 + 0xc bytes C++
boost::asio::detail::scoped_lock<boost::asio::detail::win_mutex>::scoped_lock<boost::asio::detail::win_mutex>(boost::asio::detail::win_mutex & m={...})  Line 37    C++
boost::asio::detail::task_io_service::init_task()  Line 113 + 0xf bytes C++
boost::asio::detail::select_reactor::init_task()  Line 98   C++
boost::asio::detail::deadline_timer_service<boost::asio::time_traits<boost::posix_time::ptime> >::deadline_timer_service<boost::asio::time_traits<boost::posix_time::ptime> >(boost::asio::io_service & io_service={...})  Line 67  C++
boost::asio::deadline_timer_service<boost::posix_time::ptime,boost::asio::time_traits<boost::posix_time::ptime> >::deadline_timer_service<boost::posix_time::ptime,boost::asio::time_traits<boost::posix_time::ptime> >(boost::asio::io_service & io_service={...})  Line 73    C++
boost::asio::detail::service_registry::create<boost::asio::deadline_timer_service<boost::posix_time::ptime,boost::asio::time_traits<boost::posix_time::ptime> > >(boost::asio::io_service & owner={...})  Line 81 + 0x2f bytes  C++
boost::asio::detail::service_registry::do_use_service(const boost::asio::io_service::service::key & key={...}, boost::asio::io_service::service * (boost::asio::io_service &)* factory=0x00cd6a51)  Line 123 + 0xc bytes    C++
boost::asio::detail::service_registry::use_service<boost::asio::deadline_timer_service<boost::posix_time::ptime,boost::asio::time_traits<boost::posix_time::ptime> > >()  Line 49   C++
boost::asio::use_service<boost::asio::deadline_timer_service<boost::posix_time::ptime,boost::asio::time_traits<boost::posix_time::ptime> > >(boost::asio::io_service & ios={...})  Line 34  C++
boost::asio::basic_io_object<boost::asio::deadline_timer_service<boost::posix_time::ptime,boost::asio::time_traits<boost::posix_time::ptime> > >::basic_io_object<boost::asio::deadline_timer_service<boost::posix_time::ptime,boost::asio::time_traits<boost::posix_time::ptime> > >(boost::asio::io_service & io_service={...})  Line 91 + 0x4b bytes C++
boost::asio::basic_deadline_timer<boost::posix_time::ptime,boost::asio::time_traits<boost::posix_time::ptime>,boost::asio::deadline_timer_service<boost::posix_time::ptime,boost::asio::time_traits<boost::posix_time::ptime> > >::basic_deadline_timer<boost::posix_time::ptime,boost::asio::time_traits<boost::posix_time::ptime>,boost::asio::deadline_timer_service<boost::posix_time::ptime,boost::asio::time_traits<boost::posix_time::ptime> > >(boost::asio::io_service & io_service={...}, const boost::posix_time::time_duration & expiry_time={...})  Line 181 + 0x4e bytes  C++
websocketpp::role::server<websocketpp::endpoint<websocketpp::role::server,websocketpp::socket::plain,websocketpp::log::logger> >::server<websocketpp::endpoint<websocketpp::role::server,websocketpp::socket::plain,websocketpp::log::logger> >(boost::asio::io_service & m={...})  Line 196 + 0xb1 bytes   C++
websocketpp::endpoint<websocketpp::role::server,websocketpp::socket::plain,websocketpp::log::logger>::endpoint<websocketpp::role::server,websocketpp::socket::plain,websocketpp::log::logger>(boost::shared_ptr<websocketpp::endpoint_traits<websocketpp::endpoint<websocketpp::role::server,websocketpp::socket::plain,websocketpp::log::logger> >::handler> handler={...})  Line 148 + 0x64 bytes C++

I also get the same crash from a UDP socket I create to test if a port is available:

boost::asio::detail::win_mutex::lock()  Line 50 + 0xc bytes C++
boost::asio::detail::scoped_lock<boost::asio::detail::win_mutex>::scoped_lock<boost::asio::detail::win_mutex>(boost::asio::detail::win_mutex & m={...})  Line 37    C++
boost::asio::detail::task_io_service::init_task()  Line 113 + 0xf bytes C++
boost::asio::detail::select_reactor::init_task()  Line 98   C++
boost::asio::detail::reactive_socket_service_base::reactive_socket_service_base(boost::asio::io_service & io_service={...})  Line 35    C++
boost::asio::detail::reactive_socket_service<boost::asio::ip::udp>::reactive_socket_service<boost::asio::ip::udp>(boost::asio::io_service & io_service={...})  Line 79  C++
boost::asio::datagram_socket_service<boost::asio::ip::udp>::datagram_socket_service<boost::asio::ip::udp>(boost::asio::io_service & io_service={...})  Line 91  C++
boost::asio::detail::service_registry::create<boost::asio::datagram_socket_service<boost::asio::ip::udp> >(boost::asio::io_service & owner={...})  Line 81 + 0x2f bytes C++
boost::asio::detail::service_registry::do_use_service(const boost::asio::io_service::service::key & key={...}, boost::asio::io_service::service * (boost::asio::io_service &)* factory=0x01183883)  Line 123 + 0xc bytes    C++
boost::asio::detail::service_registry::use_service<boost::asio::datagram_socket_service<boost::asio::ip::udp> >()  Line 49  C++
boost::asio::use_service<boost::asio::datagram_socket_service<boost::asio::ip::udp> >(boost::asio::io_service & ios={...})  Line 34 C++
boost::asio::basic_io_object<boost::asio::datagram_socket_service<boost::asio::ip::udp> >::basic_io_object<boost::asio::datagram_socket_service<boost::asio::ip::udp> >(boost::asio::io_service & io_service={...})  Line 91 + 0x2c bytes   C++
boost::asio::basic_socket<boost::asio::ip::udp,boost::asio::datagram_socket_service<boost::asio::ip::udp> >::basic_socket<boost::asio::ip::udp,boost::asio::datagram_socket_service<boost::asio::ip::udp> >(boost::asio::io_service & io_service={...}, const boost::asio::ip::basic_endpoint<boost::asio::ip::udp> 
boost::asio::basic_datagram_socket<boost::asio::ip::udp,boost::asio::datagram_socket_service<boost::asio::ip::udp> >::basic_datagram_socket<boost::asio::ip::udp,boost::asio::datagram_socket_service<boost::asio::ip::udp> >(boost::asio::io_service & io_service={...}, const boost::asio::ip::basic_endpoint<boos

The only thing that's changed is that WebSocket++ has been upgraded to the master branch, and our corresponding code has been updated for the new API. Everything else is exactly as it was before.

[zaphoyd]

what version of boost are you using?

[jdale88]

Boost 1.48.0.

[jdale88]

I also tried upgrading to 1.49.0, but it didn't make any difference.

[zaphoyd]

A few more questions. What OS? All single threaded? Is the UDP socket using the WebSocket++ io_service? Are you using any timers with the io_service? Does this crash happen with any of the WS++ examples?

[jdale88]

What OS: Windows 7 x64 (the compiled application is 32-bit though).

All single threaded: I only explicitly create one background thread for the IO service we use elsewhere (I also used to give WS++ this same IO service but can't any more due to #90). Other libraries we use will create additional threads, but they won't have been initialised by the point at which the crash happens.

Is the UDP socket using the WebSocket++ io_service: No, it uses the IO service I mentioned above. WS++ hasn't been initialised by the time that UDP socket is created (it tests that the port I want to use for WS++ is available).

Are you using any timers with the io_service: I have no idea what that means, so I'll assume not :)

Does this crash happen with any of the WS++ examples: Not sure, I never could get them to link (see #104). I did create a simple test project of my own to try and isolate the cause, and that is so far working fine.

Basically it looks like something in our code is causing something bad to happen after applying the WS++ upgrade. I'm not sure what, but I'll keep stripping stuff out until it runs to hopefully try and isolate the problem (which as I said, may not even relate to WS++ at all).

One other thing I noticed, the WS++ server seems to be blocking now? You used to be able to create it with an IO service and it would do its work on that rather than blocking whatever thread created it, however it now seems that calling listen blocks whilst the server is running. Does this mean I need to create a thread to run the WS++ server on, or am I doing something wrong?

[zaphoyd]

It sounds like there are are two threads with their own io_services that are conflicting in some way. I can confirm that running multiple WS++ endpoints concurrently in separate threads don't conflict with each other. I'll try and see if I can get a non-WS++ asio example to conflict when run concurrently. There must be some sort of shared asio state that is not being accessed properly. In both cases the crash is in the constructor of an asio object.

Regarding the server blocking. Unlike the 0.1 branch, which required the end user to supply and manage their own io_service, WS++ 0.2 manages its io_service internally. server.listen calls the internal io_service.run. So yes, this would need to be its own thread unless you aren't doing anything else with the program. This was an often requested option for people who weren't previously using ASIO. Based on more recent feedback it seems like an entirely managed io_service swings too far in the other direction. version 0.3.0 will offer a more flexible alternative to monolithic listen that allows both use cases. This should address both #90 and #106 and the folks asking for thread per connection and lock free designs.

[jdale88]

Do you have any thoughts as to what this shared state might be so I can try and narrow down my search?