chriskohlhoff / asio

Asio C++ Library
http://think-async.com/Asio
4.85k stars 1.2k forks source link

BOOST_ASIO_CONCURRENCY_HINT_UNSAFE - perf shows --__GI___pthread_mutex_lock #1233

Open Ziemien opened 1 year ago

Ziemien commented 1 year ago

Hello

I have a use case where the io_context is only used within single thread. By recording some data samples in perf I can see the boost::asio::io_context::poll is causing call to phtread_mutex_lock

               |--2.24%--__GI___pthread_mutex_lock (inlined)
               |          |
               |          |--1.34%--asm_sysvec_apic_timer_interrupt
               |          |          irqentry_exit_to_user_mode
               |          |          exit_to_user_mode_prepare
               |          |          exit_to_user_mode_loop
               |          |          schedule
               |          |          __schedule
               |          |          __schedule

I followed the concurrency hints documentation on boost.org https://www.boost.org/doc/libs/1_81_0/doc/html/boost_asio/overview/core/concurrency_hint.html

I added two described compilation definition into my cmake file

BOOST_ASIO_CONCURRENCY_HINT_DEFAULT=1
BOOST_ASIO_CONCURRENCY_HINT_1=BOOST_ASIO_CONCURRENCY_HINT_UNSAFE

However, I can still see the lock to pthread_mutex_lock being invoked on the io_context::poll.

Given there is no need to lock/unlock the mutex as it's single threaded application I'd like to remove to overhead of doing so. I thought the concurrency flag would be one way to do it. I also tried the BOOST_ASIO_DISABLE_THREADS flag, however the application does not run with it enabled - it seems to stall on a deadlock/mutex lock. I am probably misunderstanding something.

Linux Kernel: 5.15.90-54.138.amzn2.x86_64 Compiler: Clang 14 Boost: 1.81

Best Bart

Ziemien commented 1 year ago

when using BOOST_ASIO_DISABLE_THREADS

(lldb) bt 10
* thread #1, name = 'xx.bin', stop reason = signal SIGSEGV
  * frame #0: 0x00000000004b4397 xx.bin`boost::asio::detail::epoll_reactor::perform_io_cleanup_on_block_exit::~perform_io_cleanup_on_block_exit() [inlined] boost::asio::detail::call_stack<boost::asio::detail::thread_context, boost::asio::detail::thread_info_base>::contains(k=0x00007fd320000b70) at call_stack.hpp:98:22
    frame #1: 0x00000000004b438f xx.bin`boost::asio::detail::epoll_reactor::perform_io_cleanup_on_block_exit::~perform_io_cleanup_on_block_exit() [inlined] boost::asio::detail::scheduler::compensating_work_started() at scheduler.ipp:330
    frame #2: 0x00000000004b438f xx.bin`boost::asio::detail::epoll_reactor::perform_io_cleanup_on_block_exit::~perform_io_cleanup_on_block_exit(this=0x00007fd33daaeba8) at epoll_reactor.ipp:742
    frame #3: 0x00000000004b42ee xx.bin`boost::asio::detail::epoll_reactor::descriptor_state::perform_io(this=<unavailable>, events=5) at epoll_reactor.ipp:794:1
    frame #4: 0x00000000004b40d9 xx.bin`boost::asio::detail::epoll_reactor::descriptor_state::do_complete(owner=0x00007fd320000b70, base=<unavailable>, ec=0x00007fd33daaeda0, bytes_transferred=<unavailable>) at epoll_reactor.ipp:804:42
    frame #5: 0x000000000049a394 xx.bin`boost::asio::detail::scheduler::do_poll_one(boost::asio::detail::conditionally_enabled_mutex::scoped_lock&, boost::asio::detail::scheduler_thread_info&, boost::system::error_code const&) [inlined] boost::asio::detail::scheduler_operation::complete(this=0x00007fd320009f30, owner=0x00007fd320000b70) at scheduler_operation.hpp:40:5
    frame #6: 0x000000000049a383 xx.bin`boost::asio::detail::scheduler::do_poll_one(this=0x00007fd320000b70, lock=0x00007fd33daaecb0, this_thread=0x00007fd33daaecc0, ec=0x00007fd33daaeda0) at scheduler.ipp:628
    frame #7: 0x000000000049a0d6 xx.bin`boost::asio::detail::scheduler::poll(this=0x00007fd320000b70, ec=0x00007fd33daaeda0) at scheduler.ipp:277:10
    frame #8: 0x0000000000499f53 xx.bin`boost::asio::io_context::poll(this=<unavailable>) at io_context.ipp:93:24
ShakesB33r commented 1 year ago

Assuming fully single-threaded lifetime and operations for io_context and objects, is the most optimal hint 1?