search

chriskohlhoff / asio

Asio C++ Library
http://think-async.com/Asio
4.89k stars 1.21k forks source link

co_spawn_cancellation_handler use after free crash #1332

Closed battlmonstr closed 4 months ago

battlmonstr commented 1 year ago

We're using asio awaitables, awaitable_operators and strands, and started to see a crash in cancellation_signal.emit() on this line: https://github.com/chriskohlhoff/asio/blob/d6b95c0188e0359a8cdbdb6571f0cbacf11a538c/asio/include/asio/cancellation_signal.hpp#L118 because the cancellation_signal->handler_ pointer is invalid.

That call comes from co_spawn_cancellation_handler here: https://github.com/chriskohlhoff/asio/blob/d6b95c0188e0359a8cdbdb6571f0cbacf11a538c/asio/include/asio/impl/co_spawn.hpp#L244

Debugging has shown that sig is destroyed before the dispatch lambda runs. The co_spawn_cancellation_handler object (this) itself is destroyed, and the co_spawn_cancellation_handler->signal_ (sig), becomes a dangling pointer. The destructor is called in a call stack originating from awaitable_handler calling clear_cancellation_slot.

Side note: similar code has been noticed in the parallel_group implementation which might have the same problem in some conditions:

https://github.com/chriskohlhoff/asio/blob/d6b95c0188e0359a8cdbdb6571f0cbacf11a538c/asio/include/asio/experimental/impl/parallel_group.hpp#L285 https://github.com/chriskohlhoff/asio/blob/d6b95c0188e0359a8cdbdb6571f0cbacf11a538c/asio/include/asio/experimental/impl/parallel_group.hpp#L646

Related issue: https://github.com/chriskohlhoff/asio/issues/1286

Demo

An example to reproduce this crash is here: https://github.com/chriskohlhoff/asio/pull/1331

It is a bit complex, but it is already a significantly dumbed down version of what happens in the real application. I couldn't cut it further, because it seems to be very time sensitive and stops reproducing.

On macOS with Xcode 14.3 it can be built it with:

./configure --without-boost CXX=clang 'CXXFLAGS=-std=c++20 -stdlib=libc++' 'LIBS=-lc++ -lc++abi'
cd src/examples/cpp20
make coroutines/chat_server

If ran without a debugger, it will crash.

Running with lldb:

lldb -o run coroutines/chat_server

prints the exception:

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x8)
    frame #0: 0x000000010001262b chat_server`asio::cancellation_signal::emit(asio::cancellation_type) + 43
chat_server`asio::cancellation_signal::emit:

The 2nd commit "debug patch" on the demo PR is meant for root cause debugging. If ran with lldb, it triggers the assert and prints a stack trace of the asio::dispatch call before the crash which has to do with parallel_group:

dispatch call stack trace ``` 0 chat_server 0x0000000100011f25 _ZN4asio6detail29co_spawn_cancellation_handlerINS0_17awaitable_handlerINS_15any_io_executorEJSt13exception_ptrEEES3_vEclENS_17cancellation_typeE + 37 1 chat_server 0x0000000100011e4f _ZN4asio6detail20cancellation_handlerINS0_29co_spawn_cancellation_handlerINS0_17awaitable_handlerINS_15any_io_executorEJSt13exception_ptrEEES4_vEEE4callENS_17cancellation_typeE + 31 2 chat_server 0x00000001000124e0 _ZN4asio19cancellation_signal4emitENS_17cancellation_typeE + 48 3 chat_server 0x0000000100012dd4 _ZN4asio18cancellation_state4implINS_19cancellation_filterILNS_17cancellation_typeE1EEES4_EclES3_ + 84 4 chat_server 0x0000000100012cdf _ZN4asio6detail20cancellation_handlerINS_18cancellation_state4implINS_19cancellation_filterILNS_17cancellation_typeE1EEES6_EEE4callES5_ + 31 5 chat_server 0x00000001000124e0 _ZN4asio19cancellation_signal4emitENS_17cancellation_typeE + 48 6 chat_server 0x000000010003528b _ZN4asio6detail29co_spawn_cancellation_handlerINS_12experimental6detail25parallel_group_op_handlerILm1ENS2_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSA_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSA_ENS0_17initiate_co_spawnISC_EEJNS0_21awaitable_as_functionIvSC_EEEEESK_EEESC_vEclENS_17cancellation_typeE + 27 7 chat_server 0x000000010003521f _ZN4asio6detail20cancellation_handlerINS0_29co_spawn_cancellation_handlerINS_12experimental6detail25parallel_group_op_handlerILm1ENS3_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSB_ENS0_17initiate_co_spawnISD_EEJNS0_21awaitable_as_functionIvSD_EEEEESL_EEESD_vEEE4callENS_17cancellation_typeE + 31 8 chat_server 0x00000001000124e0 _ZN4asio19cancellation_signal4emitENS_17cancellation_typeE + 48 9 chat_server 0x00000001000319ea _ZN4asio12experimental6detail25parallel_group_op_handlerILm0ENS0_18wait_for_one_errorENS_6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS9_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS9_ENS4_17initiate_co_spawnISB_EEJNS4_21awaitable_as_functionIvSB_EEEEESJ_EEclIJS9_EEEvDpT_ + 282 10 chat_server 0x0000000100031892 _ZZN4asio6detail20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS2_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSA_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSA_ENS0_17initiate_co_spawnISC_EEJNS0_21awaitable_as_functionIvSC_EEEEESK_EEESC_SJ_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSM_IvSO_EENS0_14co_spawn_stateIT_SO_T1_vEEENUlvE_clEv + 50 11 chat_server 0x0000000100031855 _ZN4asio6detail7binder0IZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS3_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSB_ENS0_17initiate_co_spawnISD_EEJNS0_21awaitable_as_functionIvSD_EEEEESL_EEESD_SK_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSN_IvSP_EENS0_14co_spawn_stateIT_SP_T1_vEEEUlvE_EclEv + 21 12 chat_server 0x0000000100033bb5 _ZN4asio19asio_handler_invokeINS_6detail7binder0IZNS1_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS4_18wait_for_one_errorENS1_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSC_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSC_ENS1_17initiate_co_spawnISE_EEJNS1_21awaitable_as_functionIvSE_EEEEESM_EEESE_SL_EENS_9awaitableINS1_28awaitable_thread_entry_pointET0_EEPNSO_IvSQ_EENS1_14co_spawn_stateIT_SQ_T1_vEEEUlvE_EEEEvRSV_z + 21 13 chat_server 0x0000000100033b8f _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail7binder0IZNS2_20co_spawn_entry_pointINS1_12experimental6detail25parallel_group_op_handlerILm0ENS5_18wait_for_one_errorENS2_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSD_ENS1_15any_io_executorEEEJNS1_24deferred_async_operationIFvSD_ENS2_17initiate_co_spawnISF_EEJNS2_21awaitable_as_functionIvSF_EEEEESN_EEESF_SM_EENS1_9awaitableINS2_28awaitable_thread_entry_pointET0_EEPNSP_IvSR_EENS2_14co_spawn_stateIT_SR_T1_vEEEUlvE_EESZ_EEvRSW_RSR_ + 31 14 chat_server 0x0000000100033b5d _ZN4asio6detail19asio_handler_invokeINS0_7binder0IZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS4_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSC_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSC_ENS0_17initiate_co_spawnISE_EEJNS0_21awaitable_as_functionIvSE_EEEEESM_EEESE_SL_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSO_IvSQ_EENS0_14co_spawn_stateIT_SQ_T1_vEEEUlvE_EESY_EEvRSV_PNS2_ISQ_EE + 29 15 chat_server 0x0000000100033abd _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail7binder0IZNS2_20co_spawn_entry_pointINS1_12experimental6detail25parallel_group_op_handlerILm0ENS5_18wait_for_one_errorENS2_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSD_ENS1_15any_io_executorEEEJNS1_24deferred_async_operationIFvSD_ENS2_17initiate_co_spawnISF_EEJNS2_21awaitable_as_functionIvSF_EEEEESN_EEESF_SM_EENS1_9awaitableINS2_28awaitable_thread_entry_pointET0_EEPNSP_IvSR_EENS2_14co_spawn_stateIT_SR_T1_vEEEUlvE_EES10_EEvRSW_RSR_ + 29 16 chat_server 0x00000001000339ae _ZN4asio6detail17executor_function8completeINS0_7binder0IZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS5_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSD_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSD_ENS0_17initiate_co_spawnISF_EEJNS0_21awaitable_as_functionIvSF_EEEEESN_EEESF_SM_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSP_IvSR_EENS0_14co_spawn_stateIT_SR_T1_vEEEUlvE_EENSA_9allocatorIvEEEEvPNS1_9impl_baseEb + 110 17 chat_server 0x0000000100016692 _ZN4asio6detail17executor_functionclEv + 66 18 chat_server 0x0000000100016645 _ZN4asio19asio_handler_invokeINS_6detail17executor_functionEEEvRT_z + 21 19 chat_server 0x00000001000160af _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail17executor_functionES3_EEvRT_RT0_ + 31 20 chat_server 0x0000000100026ef3 _ZNK4asio10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EE7executeINS_6detail17executor_functionEEEvOT_ + 115 21 chat_server 0x0000000100026e71 _ZNK25asio_execution_execute_fn4implclIRKN4asio10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EEENS2_6detail17executor_functionEEENS5_9enable_ifIXeqsr11call_traitsIS0_T_FvT0_EEE8overloadLNS_13overload_typeE0EENS_11call_traitsIS0_SE_SG_vvvvvE11result_typeEE4typeEOSE_OSF_ + 33 22 chat_server 0x0000000100026d31 _ZN4asio9execution6detail17any_executor_base10execute_exINS_10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EEEEEvRKS2_ONS_6detail17executor_functionE + 49 23 chat_server 0x00000001000316a5 _ZNK4asio9execution6detail17any_executor_base7executeINS_6detail7binder0IZNS4_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS7_18wait_for_one_errorENS4_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSF_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSF_ENS4_17initiate_co_spawnISH_EEJNS4_21awaitable_as_functionIvSH_EEEEESP_EEESH_SO_EENS_9awaitableINS4_28awaitable_thread_entry_pointET0_EEPNSR_IvST_EENS4_14co_spawn_stateIT_ST_T1_vEEEUlvE_EEEEvOSY_ + 197 24 chat_server 0x0000000100031571 _ZNK25asio_execution_execute_fn4implclIRKN4asio15any_io_executorENS2_6detail7binder0IZNS6_20co_spawn_entry_pointINS2_12experimental6detail25parallel_group_op_handlerILm0ENS9_18wait_for_one_errorENS6_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSH_ES3_EEJNS2_24deferred_async_operationIFvSH_ENS6_17initiate_co_spawnIS3_EEJNS6_21awaitable_as_functionIvS3_EEEEESQ_EEES3_SP_EENS2_9awaitableINS6_28awaitable_thread_entry_pointET0_EEPNSS_IvSU_EENS6_14co_spawn_stateIT_SU_T1_vEEEUlvE_EEEENSE_9enable_ifIXeqsr11call_traitsIS0_SZ_FvSU_EEE8overloadLNS_13overload_typeE0EENS_11call_traitsIS0_SZ_S15_vvvvvE11result_typeEE4typeEOSZ_OSU_ + 33 25 chat_server 0x00000001000314f9 _ZNK4asio6detail31initiate_dispatch_with_executorINS_15any_io_executorEEclIZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS6_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSE_ES2_EEJNS_24deferred_async_operationIFvSE_ENS0_17initiate_co_spawnIS2_EEJNS0_21awaitable_as_functionIvS2_EEEEESN_EEES2_SM_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSP_IvSR_EENS0_14co_spawn_stateIT_SR_T1_vEEEUlvE_EEvOSW_PNSB_9enable_ifIXsr9execution11is_executorINSB_11conditionalILb1ES2_SW_E4typeEEE5valueEvE4typeEPNS11_IXntsr6detail27is_work_dispatcher_requiredINSB_5decayISW_E4typeES2_EE5valueEvE4typeE + 121 26 chat_server 0x0000000100031474 _ZN4asio6detail31completion_handler_async_resultIZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS3_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSB_ENS0_17initiate_co_spawnISD_EEJNS0_21awaitable_as_functionIvSD_EEEEESL_EEESD_SK_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSN_IvSP_EENS0_14co_spawn_stateIT_SP_T1_vEEEUlvE_JFvvEEE8initiateINS0_31initiate_dispatch_with_executorISD_EESX_JEEEvOSU_OSP_DpOT1_ + 36 27 chat_server 0x000000010003143d _ZN4asio14async_initiateIZNS_6detail20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS3_18wait_for_one_errorENS1_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSB_ENS1_17initiate_co_spawnISD_EEJNS1_21awaitable_as_functionIvSD_EEEEESL_EEESD_SK_EENS_9awaitableINS1_28awaitable_thread_entry_pointET0_EEPNSN_IvSP_EENS1_14co_spawn_stateIT_SP_T1_vEEEUlvE_JFvvEENS1_31initiate_dispatch_with_executorISD_EEJEEENS_10constraintIXsr6detail31async_result_has_initiate_memfnISU_DpT0_EE5valueEDTclsr12async_resultINS8_5decayISU_E4typeES13_EE8initiatecl7declvalIOSV_EEcl7declvalIOSU_EEspcl7declvalIOT2_EEEEE4typeES17_RSU_DpS1A_ + 29 28 chat_server 0x000000010003131f _ZN4asio8dispatchINS_15any_io_executorEZNS_6detail20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS4_18wait_for_one_errorENS2_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSC_ES1_EEJNS_24deferred_async_operationIFvSC_ENS2_17initiate_co_spawnIS1_EEJNS2_21awaitable_as_functionIvS1_EEEEESL_EEES1_SK_EENS_9awaitableINS2_28awaitable_thread_entry_pointET0_EEPNSN_IvSP_EENS2_14co_spawn_stateIT_SP_T1_vEEEUlvE_EEDaRKSU_OSP_NS_10constraintIXoosr9execution11is_executorISU_EE5valuesr11is_executorISU_EE5valueEiE4typeE + 63 29 chat_server 0x000000010003edca _ZN4asio6detail20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS2_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSA_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSA_ENS0_17initiate_co_spawnISC_EEJNS0_21awaitable_as_functionIvSC_EEEEESK_EEESC_SJ_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSM_IvSO_EENS0_14co_spawn_stateIT_SO_T1_vEE.resume + 2378 30 chat_server 0x000000010000a308 _ZNKSt3__116coroutine_handleIvE6resumeB6v15006Ev + 24 31 chat_server 0x000000010000a1ee _ZN4asio6detail20awaitable_frame_baseINS_15any_io_executorEE6resumeEv + 46 32 chat_server 0x0000000100009f54 _ZN4asio6detail16awaitable_threadINS_15any_io_executorEE4pumpEv + 36 33 chat_server 0x00000001000327ba _ZN4asio6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS5_ENS_15any_io_executorEEclIJRS4_S5_S5_EEEvDpOT_ + 122 34 chat_server 0x000000010003273a _ZN4asio12experimental6detail33parallel_group_completion_handlerINS_6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS8_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS8_ENS3_17initiate_co_spawnISA_EEJNS3_21awaitable_as_functionIvSA_EEEEESI_EE6invokeIJS8_S8_EJLm0ELm1EEEEvONS5_5tupleIJDpT_EEENS5_16integer_sequenceImJXspT0_EEEE + 74 35 chat_server 0x000000010003263d _ZN4asio12experimental6detail33parallel_group_completion_handlerINS_6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS8_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS8_ENS3_17initiate_co_spawnISA_EEJNS3_21awaitable_as_functionIvSA_EEEEESI_EE6invokeIJS8_S8_EEEvONS5_5tupleIJDpT_EEE + 29 36 chat_server 0x00000001000325df _ZN4asio12experimental6detail33parallel_group_completion_handlerINS_6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS8_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS8_ENS3_17initiate_co_spawnISA_EEJNS3_21awaitable_as_functionIvSA_EEEEESI_EE6invokeIJLm0ELm1EEEEvNS5_16integer_sequenceImJXspT_EEEE + 95 37 chat_server 0x0000000100032575 _ZN4asio12experimental6detail33parallel_group_completion_handlerINS_6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS8_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS8_ENS3_17initiate_co_spawnISA_EEJNS3_21awaitable_as_functionIvSA_EEEEESI_EEclEv + 21 38 chat_server 0x0000000100032555 _ZN4asio6detail7binder0INS_12experimental6detail33parallel_group_completion_handlerINS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS9_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS9_ENS0_17initiate_co_spawnISB_EEJNS0_21awaitable_as_functionIvSB_EEEEESJ_EEEEclEv + 21 39 chat_server 0x0000000100033635 _ZN4asio19asio_handler_invokeINS_6detail7binder0INS_12experimental6detail33parallel_group_completion_handlerINS1_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSA_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSA_ENS1_17initiate_co_spawnISC_EEJNS1_21awaitable_as_functionIvSC_EEEEESK_EEEEEEEvRT_z + 21 40 chat_server 0x000000010003360f _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail7binder0INS1_12experimental6detail33parallel_group_completion_handlerINS2_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS1_15any_io_executorEEEJNS1_24deferred_async_operationIFvSB_ENS2_17initiate_co_spawnISD_EEJNS2_21awaitable_as_functionIvSD_EEEEESL_EEEEESM_EEvRT_RT0_ + 31 41 chat_server 0x00000001000335dd _ZN4asio6detail19asio_handler_invokeINS0_7binder0INS_12experimental6detail33parallel_group_completion_handlerINS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSA_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSA_ENS0_17initiate_co_spawnISC_EEJNS0_21awaitable_as_functionIvSC_EEEEESK_EEEEESL_EEvRT_PNS2_IT0_EE + 29 42 chat_server 0x000000010003352d _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail7binder0INS1_12experimental6detail33parallel_group_completion_handlerINS2_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS1_15any_io_executorEEEJNS1_24deferred_async_operationIFvSB_ENS2_17initiate_co_spawnISD_EEJNS2_21awaitable_as_functionIvSD_EEEEESL_EEEEESN_EEvRT_RT0_ + 29 43 chat_server 0x00000001000333e6 _ZN4asio6detail17executor_function8completeINS0_7binder0INS_12experimental6detail33parallel_group_completion_handlerINS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSB_ENS0_17initiate_co_spawnISD_EEJNS0_21awaitable_as_functionIvSD_EEEEESL_EEEEENS8_9allocatorIvEEEEvPNS1_9impl_baseEb + 166 44 chat_server 0x0000000100016692 _ZN4asio6detail17executor_functionclEv + 66 45 chat_server 0x0000000100016645 _ZN4asio19asio_handler_invokeINS_6detail17executor_functionEEEvRT_z + 21 46 chat_server 0x00000001000160af _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail17executor_functionES3_EEvRT_RT0_ + 31 47 chat_server 0x0000000100026ef3 _ZNK4asio10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EE7executeINS_6detail17executor_functionEEEvOT_ + 115 48 chat_server 0x0000000100026e71 _ZNK25asio_execution_execute_fn4implclIRKN4asio10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EEENS2_6detail17executor_functionEEENS5_9enable_ifIXeqsr11call_traitsIS0_T_FvT0_EEE8overloadLNS_13overload_typeE0EENS_11call_traitsIS0_SE_SG_vvvvvE11result_typeEE4typeEOSE_OSF_ + 33 49 chat_server 0x0000000100026d31 _ZN4asio9execution6detail17any_executor_base10execute_exINS_10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EEEEEvRKS2_ONS_6detail17executor_functionE + 49 50 chat_server 0x0000000100032368 _ZNK4asio9execution6detail17any_executor_base7executeINS_6detail7binder0INS_12experimental6detail33parallel_group_completion_handlerINS4_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSD_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSD_ENS4_17initiate_co_spawnISF_EEJNS4_21awaitable_as_functionIvSF_EEEEESN_EEEEEEEvOT_ + 280 51 chat_server 0x0000000100032161 _ZNK25asio_execution_execute_fn4implclIRN4asio15any_io_executorENS2_6detail7binder0INS2_12experimental6detail33parallel_group_completion_handlerINS5_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSE_ES3_EEJNS2_24deferred_async_operationIFvSE_ENS5_17initiate_co_spawnIS3_EEJNS5_21awaitable_as_functionIvS3_EEEEESN_EEEEEEENSB_9enable_ifIXeqsr11call_traitsIS0_T_FvT0_EEE8overloadLNS_13overload_typeE0EENS_11call_traitsIS0_SR_ST_vvvvvE11result_typeEE4typeEOSR_OSS_ + 33 52 chat_server 0x0000000100032062 _ZNK4asio6detail17initiate_dispatchclINS_12experimental6detail33parallel_group_completion_handlerINS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSA_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSA_ENS0_17initiate_co_spawnISC_EEJNS0_21awaitable_as_functionIvSC_EEEEESK_EEEEEvOT_PNS7_9enable_ifIXsr9execution11is_executorINS_19associated_executorINS7_5decayISM_E4typeENS_21basic_system_executorINS_9execution6detail8blocking10possibly_tILi0EEENSV_12relationship6fork_tILi0EEENS7_9allocatorIvEEEEE4typeEEE5valueEvE4typeE + 178 53 chat_server 0x0000000100031fa1 _ZN4asio6detail31completion_handler_async_resultINS_12experimental6detail33parallel_group_completion_handlerINS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS9_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS9_ENS0_17initiate_co_spawnISB_EEJNS0_21awaitable_as_functionIvSB_EEEEESJ_EEEJFvvEEE8initiateINS0_17initiate_dispatchESK_JEEEvOT_OT0_DpOT1_ + 33 54 chat_server 0x0000000100031f6d _ZN4asio14async_initiateINS_12experimental6detail33parallel_group_completion_handlerINS_6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS9_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS9_ENS4_17initiate_co_spawnISB_EEJNS4_21awaitable_as_functionIvSB_EEEEESJ_EEEJFvvEENS4_17initiate_dispatchEJEEENS_10constraintIXsr6detail31async_result_has_initiate_memfnIT_DpT0_EE5valueEDTclsr12async_resultINS6_5decayISO_E4typeESQ_EE8initiatecl7declvalIOT1_EEcl7declvalIOSO_EEspcl7declvalIOT2_EEEEE4typeESV_RSO_DpSY_ + 29 55 chat_server 0x0000000100031bb9 _ZN4asio8dispatchINS_12experimental6detail33parallel_group_completion_handlerINS_6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS9_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS9_ENS4_17initiate_co_spawnISB_EEJNS4_21awaitable_as_functionIvSB_EEEEESJ_EEEEEDaOT_ + 25 56 chat_server 0x0000000100031a3d _ZN4asio12experimental6detail25parallel_group_op_handlerILm0ENS0_18wait_for_one_errorENS_6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS9_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS9_ENS4_17initiate_co_spawnISB_EEJNS4_21awaitable_as_functionIvSB_EEEEESJ_EEclIJS9_EEEvDpT_ + 365 57 chat_server 0x0000000100031892 _ZZN4asio6detail20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS2_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSA_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSA_ENS0_17initiate_co_spawnISC_EEJNS0_21awaitable_as_functionIvSC_EEEEESK_EEESC_SJ_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSM_IvSO_EENS0_14co_spawn_stateIT_SO_T1_vEEENUlvE_clEv + 50 58 chat_server 0x0000000100031855 _ZN4asio6detail7binder0IZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS3_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSB_ENS0_17initiate_co_spawnISD_EEJNS0_21awaitable_as_functionIvSD_EEEEESL_EEESD_SK_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSN_IvSP_EENS0_14co_spawn_stateIT_SP_T1_vEEEUlvE_EclEv + 21 59 chat_server 0x0000000100033bb5 _ZN4asio19asio_handler_invokeINS_6detail7binder0IZNS1_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS4_18wait_for_one_errorENS1_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSC_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSC_ENS1_17initiate_co_spawnISE_EEJNS1_21awaitable_as_functionIvSE_EEEEESM_EEESE_SL_EENS_9awaitableINS1_28awaitable_thread_entry_pointET0_EEPNSO_IvSQ_EENS1_14co_spawn_stateIT_SQ_T1_vEEEUlvE_EEEEvRSV_z + 21 60 chat_server 0x0000000100033b8f _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail7binder0IZNS2_20co_spawn_entry_pointINS1_12experimental6detail25parallel_group_op_handlerILm0ENS5_18wait_for_one_errorENS2_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSD_ENS1_15any_io_executorEEEJNS1_24deferred_async_operationIFvSD_ENS2_17initiate_co_spawnISF_EEJNS2_21awaitable_as_functionIvSF_EEEEESN_EEESF_SM_EENS1_9awaitableINS2_28awaitable_thread_entry_pointET0_EEPNSP_IvSR_EENS2_14co_spawn_stateIT_SR_T1_vEEEUlvE_EESZ_EEvRSW_RSR_ + 31 61 chat_server 0x0000000100033b5d _ZN4asio6detail19asio_handler_invokeINS0_7binder0IZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS4_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSC_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSC_ENS0_17initiate_co_spawnISE_EEJNS0_21awaitable_as_functionIvSE_EEEEESM_EEESE_SL_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSO_IvSQ_EENS0_14co_spawn_stateIT_SQ_T1_vEEEUlvE_EESY_EEvRSV_PNS2_ISQ_EE + 29 62 chat_server 0x0000000100033abd _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail7binder0IZNS2_20co_spawn_entry_pointINS1_12experimental6detail25parallel_group_op_handlerILm0ENS5_18wait_for_one_errorENS2_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSD_ENS1_15any_io_executorEEEJNS1_24deferred_async_operationIFvSD_ENS2_17initiate_co_spawnISF_EEJNS2_21awaitable_as_functionIvSF_EEEEESN_EEESF_SM_EENS1_9awaitableINS2_28awaitable_thread_entry_pointET0_EEPNSP_IvSR_EENS2_14co_spawn_stateIT_SR_T1_vEEEUlvE_EES10_EEvRSW_RSR_ + 29 63 chat_server 0x00000001000339ae _ZN4asio6detail17executor_function8completeINS0_7binder0IZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS5_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSD_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSD_ENS0_17initiate_co_spawnISF_EEJNS0_21awaitable_as_functionIvSF_EEEEESN_EEESF_SM_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSP_IvSR_EENS0_14co_spawn_stateIT_SR_T1_vEEEUlvE_EENSA_9allocatorIvEEEEvPNS1_9impl_baseEb + 110 64 chat_server 0x0000000100016692 _ZN4asio6detail17executor_functionclEv + 66 65 chat_server 0x0000000100016645 _ZN4asio19asio_handler_invokeINS_6detail17executor_functionEEEvRT_z + 21 66 chat_server 0x00000001000160af _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail17executor_functionES3_EEvRT_RT0_ + 31 67 chat_server 0x0000000100026ef3 _ZNK4asio10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EE7executeINS_6detail17executor_functionEEEvOT_ + 115 68 chat_server 0x0000000100026e71 _ZNK25asio_execution_execute_fn4implclIRKN4asio10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EEENS2_6detail17executor_functionEEENS5_9enable_ifIXeqsr11call_traitsIS0_T_FvT0_EEE8overloadLNS_13overload_typeE0EENS_11call_traitsIS0_SE_SG_vvvvvE11result_typeEE4typeEOSE_OSF_ + 33 69 chat_server 0x0000000100026d31 _ZN4asio9execution6detail17any_executor_base10execute_exINS_10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EEEEEvRKS2_ONS_6detail17executor_functionE + 49 70 chat_server 0x00000001000316a5 _ZNK4asio9execution6detail17any_executor_base7executeINS_6detail7binder0IZNS4_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS7_18wait_for_one_errorENS4_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSF_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSF_ENS4_17initiate_co_spawnISH_EEJNS4_21awaitable_as_functionIvSH_EEEEESP_EEESH_SO_EENS_9awaitableINS4_28awaitable_thread_entry_pointET0_EEPNSR_IvST_EENS4_14co_spawn_stateIT_ST_T1_vEEEUlvE_EEEEvOSY_ + 197 71 chat_server 0x0000000100031571 _ZNK25asio_execution_execute_fn4implclIRKN4asio15any_io_executorENS2_6detail7binder0IZNS6_20co_spawn_entry_pointINS2_12experimental6detail25parallel_group_op_handlerILm0ENS9_18wait_for_one_errorENS6_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSH_ES3_EEJNS2_24deferred_async_operationIFvSH_ENS6_17initiate_co_spawnIS3_EEJNS6_21awaitable_as_functionIvS3_EEEEESQ_EEES3_SP_EENS2_9awaitableINS6_28awaitable_thread_entry_pointET0_EEPNSS_IvSU_EENS6_14co_spawn_stateIT_SU_T1_vEEEUlvE_EEEENSE_9enable_ifIXeqsr11call_traitsIS0_SZ_FvSU_EEE8overloadLNS_13overload_typeE0EENS_11call_traitsIS0_SZ_S15_vvvvvE11result_typeEE4typeEOSZ_OSU_ + 33 72 chat_server 0x00000001000314f9 _ZNK4asio6detail31initiate_dispatch_with_executorINS_15any_io_executorEEclIZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS6_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSE_ES2_EEJNS_24deferred_async_operationIFvSE_ENS0_17initiate_co_spawnIS2_EEJNS0_21awaitable_as_functionIvS2_EEEEESN_EEES2_SM_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSP_IvSR_EENS0_14co_spawn_stateIT_SR_T1_vEEEUlvE_EEvOSW_PNSB_9enable_ifIXsr9execution11is_executorINSB_11conditionalILb1ES2_SW_E4typeEEE5valueEvE4typeEPNS11_IXntsr6detail27is_work_dispatcher_requiredINSB_5decayISW_E4typeES2_EE5valueEvE4typeE + 121 73 chat_server 0x0000000100031474 _ZN4asio6detail31completion_handler_async_resultIZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS3_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSB_ENS0_17initiate_co_spawnISD_EEJNS0_21awaitable_as_functionIvSD_EEEEESL_EEESD_SK_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSN_IvSP_EENS0_14co_spawn_stateIT_SP_T1_vEEEUlvE_JFvvEEE8initiateINS0_31initiate_dispatch_with_executorISD_EESX_JEEEvOSU_OSP_DpOT1_ + 36 74 chat_server 0x000000010003143d _ZN4asio14async_initiateIZNS_6detail20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS3_18wait_for_one_errorENS1_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSB_ENS1_17initiate_co_spawnISD_EEJNS1_21awaitable_as_functionIvSD_EEEEESL_EEESD_SK_EENS_9awaitableINS1_28awaitable_thread_entry_pointET0_EEPNSN_IvSP_EENS1_14co_spawn_stateIT_SP_T1_vEEEUlvE_JFvvEENS1_31initiate_dispatch_with_executorISD_EEJEEENS_10constraintIXsr6detail31async_result_has_initiate_memfnISU_DpT0_EE5valueEDTclsr12async_resultINS8_5decayISU_E4typeES13_EE8initiatecl7declvalIOSV_EEcl7declvalIOSU_EEspcl7declvalIOT2_EEEEE4typeES17_RSU_DpS1A_ + 29 75 chat_server 0x000000010003131f _ZN4asio8dispatchINS_15any_io_executorEZNS_6detail20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS4_18wait_for_one_errorENS2_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSC_ES1_EEJNS_24deferred_async_operationIFvSC_ENS2_17initiate_co_spawnIS1_EEJNS2_21awaitable_as_functionIvS1_EEEEESL_EEES1_SK_EENS_9awaitableINS2_28awaitable_thread_entry_pointET0_EEPNSN_IvSP_EENS2_14co_spawn_stateIT_SP_T1_vEEEUlvE_EEDaRKSU_OSP_NS_10constraintIXoosr9execution11is_executorISU_EE5valuesr11is_executorISU_EE5valueEiE4typeE + 63 76 chat_server 0x000000010003edca _ZN4asio6detail20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS2_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSA_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSA_ENS0_17initiate_co_spawnISC_EEJNS0_21awaitable_as_functionIvSC_EEEEESK_EEESC_SJ_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSM_IvSO_EENS0_14co_spawn_stateIT_SO_T1_vEE.resume + 2378 77 chat_server 0x000000010000a308 _ZNKSt3__116coroutine_handleIvE6resumeB6v15006Ev + 24 78 chat_server 0x000000010000a1ee _ZN4asio6detail20awaitable_frame_baseINS_15any_io_executorEE6resumeEv + 46 79 chat_server 0x0000000100009f54 _ZN4asio6detail16awaitable_threadINS_15any_io_executorEE4pumpEv + 36 80 chat_server 0x0000000100009dc5 _ZN4asio6detail17awaitable_handlerINS_15any_io_executorEJNSt3__110error_codeEEEclERKS4_ + 149 81 chat_server 0x0000000100009d1c _ZN4asio6detail7binder1INS0_17awaitable_handlerINS_15any_io_executorEJNSt3__110error_codeEEEES5_EclEv + 28 82 chat_server 0x0000000100009cf5 _ZN4asio19asio_handler_invokeINS_6detail7binder1INS1_17awaitable_handlerINS_15any_io_executorEJNSt3__110error_codeEEEES6_EEEEvRT_z + 21 83 chat_server 0x0000000100009c9f _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail7binder1INS2_17awaitable_handlerINS1_15any_io_executorEJNSt3__110error_codeEEEES7_EES8_EEvRT_RT0_ + 31 84 chat_server 0x000000010000874b _ZN4asio6detail12handler_workINS0_17awaitable_handlerINS_15any_io_executorEJNSt3__110error_codeEEEES3_vE8completeINS0_7binder1IS6_S5_EEEEvRT_RS6_ + 75 85 chat_server 0x0000000100008475 _ZN4asio6detail12wait_handlerINS0_17awaitable_handlerINS_15any_io_executorEJNSt3__110error_codeEEEES3_E11do_completeEPvPNS0_19scheduler_operationERKS5_m + 245 86 chat_server 0x0000000100016dee _ZN4asio6detail19scheduler_operation8completeEPvRKNSt3__110error_codeEm + 46 87 chat_server 0x000000010001f54a _ZN4asio6detail9scheduler10do_run_oneERNS0_27conditionally_enabled_mutex11scoped_lockERNS0_21scheduler_thread_infoERKNSt3__110error_codeE + 490 88 chat_server 0x000000010001f14f _ZN4asio6detail9scheduler3runERNSt3__110error_codeE + 239 89 chat_server 0x00000001000060ae _ZN4asio10io_context3runEv + 46 90 chat_server 0x0000000100005f28 main + 88 91 dyld 0x0000000200172310 start + 2432 ```
battlmonstr commented 1 year ago

The related code has appeared here: https://github.com/chriskohlhoff/asio/commit/f0a1e1c7c0387ad16358c81eb52528f190df625c

mscottmueller commented 11 months ago

I have been tracking down this issue with a parallel_group running in wait_for_one_error() mode. I have two tasks running- one is a network sequence and the other is a wait on a steady_timer which gets signaled in that network sequence when that sequence completes with success or while it throws an exception. Do you know what a workaround for this issue might be? This code is unchanged in 1.83.

battlmonstr commented 11 months ago

@mscottmueller for now I've basically copy-pasted impl/co_spawn.hpp, renamed it to not conflict with asio::co_spawn, and reverted this there. That code was probably solving some other problem, but at least it stopped crashing for me. See workaround here.

mscottmueller commented 11 months ago

He's trying to make certain that the cancellation gets dispatched on the same executor- so we need to ensure that the signal is alive. Something like this should also work:

template <typename Handler, typename Executor, typename = void>
class co_spawn_cancellation_handler
{
public:
  co_spawn_cancellation_handler(const Handler& handler, const Executor& ex)
    : ptr_(std::make_shared<std::tuple<cancellation_signal, Executor>>(
        { {}, ex }))
  {
  } 

  cancellation_slot slot() { return std::get<0>(*ptr_).slot(); }

  void operator()(cancellation_type_t type)
  {
    boost::asio::dispatch(std::get<1>(*ptr_),
      [ptr_, type]() mutable
      {
        if (ptr_.use_count() == 1) // if we have the only reference here in this dispatch, we need to let go.
          return;
        cancellation_signal* sig = &std::get<0>(*ptr_);
        sig->emit(type);
      });
  }

private:
  std::shared_ptr<
    std::tuple<cancellation_signal, Executor>>
    ptr_;
};

With this code I don't see the crash and I believe the intent of the original change is preserved. EDIT: Nope, this doesn't work either. I get a crash from a different cancellation handler instead.

battlmonstr commented 4 months ago

Fixed by https://github.com/chriskohlhoff/asio/commit/22269a24385ab94879861957584c16cb6b0ff507 The co_spawn_cancellation_handler::signal_ is changed to be a shared_ptr. The fix is included in boost asio 1.83