search

chriskohlhoff / asio

Asio C++ Library
http://think-async.com/Asio
4.96k stars 1.22k forks source link

co_spawn_cancellation_handler use after free crash #1332

Closed battlmonstr closed 5 months ago

battlmonstr commented 1 year ago

We're using asio awaitables, awaitable_operators and strands, and started to see a crash in cancellation_signal.emit() on this line: https://github.com/chriskohlhoff/asio/blob/d6b95c0188e0359a8cdbdb6571f0cbacf11a538c/asio/include/asio/cancellation_signal.hpp#L118 because the cancellation_signal->handler_ pointer is invalid.

That call comes from co_spawn_cancellation_handler here: https://github.com/chriskohlhoff/asio/blob/d6b95c0188e0359a8cdbdb6571f0cbacf11a538c/asio/include/asio/impl/co_spawn.hpp#L244

Debugging has shown that sig is destroyed before the dispatch lambda runs. The co_spawn_cancellation_handler object (this) itself is destroyed, and the co_spawn_cancellation_handler->signal_ (sig), becomes a dangling pointer. The destructor is called in a call stack originating from awaitable_handler calling clear_cancellation_slot.

Side note: similar code has been noticed in the parallel_group implementation which might have the same problem in some conditions:

https://github.com/chriskohlhoff/asio/blob/d6b95c0188e0359a8cdbdb6571f0cbacf11a538c/asio/include/asio/experimental/impl/parallel_group.hpp#L285 https://github.com/chriskohlhoff/asio/blob/d6b95c0188e0359a8cdbdb6571f0cbacf11a538c/asio/include/asio/experimental/impl/parallel_group.hpp#L646

Related issue: https://github.com/chriskohlhoff/asio/issues/1286

Demo

An example to reproduce this crash is here: https://github.com/chriskohlhoff/asio/pull/1331

It is a bit complex, but it is already a significantly dumbed down version of what happens in the real application. I couldn't cut it further, because it seems to be very time sensitive and stops reproducing.

On macOS with Xcode 14.3 it can be built it with:

./configure --without-boost CXX=clang 'CXXFLAGS=-std=c++20 -stdlib=libc++' 'LIBS=-lc++ -lc++abi'
cd src/examples/cpp20
make coroutines/chat_server

If ran without a debugger, it will crash.

Running with lldb:

lldb -o run coroutines/chat_server

prints the exception:

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x8)
    frame #0: 0x000000010001262b chat_server`asio::cancellation_signal::emit(asio::cancellation_type) + 43
chat_server`asio::cancellation_signal::emit:

The 2nd commit "debug patch" on the demo PR is meant for root cause debugging. If ran with lldb, it triggers the assert and prints a stack trace of the asio::dispatch call before the crash which has to do with parallel_group:

dispatch call stack trace ``` 0 chat_server 0x0000000100011f25 _ZN4asio6detail29co_spawn_cancellation_handlerINS0_17awaitable_handlerINS_15any_io_executorEJSt13exception_ptrEEES3_vEclENS_17cancellation_typeE + 37 1 chat_server 0x0000000100011e4f _ZN4asio6detail20cancellation_handlerINS0_29co_spawn_cancellation_handlerINS0_17awaitable_handlerINS_15any_io_executorEJSt13exception_ptrEEES4_vEEE4callENS_17cancellation_typeE + 31 2 chat_server 0x00000001000124e0 _ZN4asio19cancellation_signal4emitENS_17cancellation_typeE + 48 3 chat_server 0x0000000100012dd4 _ZN4asio18cancellation_state4implINS_19cancellation_filterILNS_17cancellation_typeE1EEES4_EclES3_ + 84 4 chat_server 0x0000000100012cdf _ZN4asio6detail20cancellation_handlerINS_18cancellation_state4implINS_19cancellation_filterILNS_17cancellation_typeE1EEES6_EEE4callES5_ + 31 5 chat_server 0x00000001000124e0 _ZN4asio19cancellation_signal4emitENS_17cancellation_typeE + 48 6 chat_server 0x000000010003528b _ZN4asio6detail29co_spawn_cancellation_handlerINS_12experimental6detail25parallel_group_op_handlerILm1ENS2_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSA_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSA_ENS0_17initiate_co_spawnISC_EEJNS0_21awaitable_as_functionIvSC_EEEEESK_EEESC_vEclENS_17cancellation_typeE + 27 7 chat_server 0x000000010003521f _ZN4asio6detail20cancellation_handlerINS0_29co_spawn_cancellation_handlerINS_12experimental6detail25parallel_group_op_handlerILm1ENS3_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSB_ENS0_17initiate_co_spawnISD_EEJNS0_21awaitable_as_functionIvSD_EEEEESL_EEESD_vEEE4callENS_17cancellation_typeE + 31 8 chat_server 0x00000001000124e0 _ZN4asio19cancellation_signal4emitENS_17cancellation_typeE + 48 9 chat_server 0x00000001000319ea _ZN4asio12experimental6detail25parallel_group_op_handlerILm0ENS0_18wait_for_one_errorENS_6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS9_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS9_ENS4_17initiate_co_spawnISB_EEJNS4_21awaitable_as_functionIvSB_EEEEESJ_EEclIJS9_EEEvDpT_ + 282 10 chat_server 0x0000000100031892 _ZZN4asio6detail20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS2_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSA_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSA_ENS0_17initiate_co_spawnISC_EEJNS0_21awaitable_as_functionIvSC_EEEEESK_EEESC_SJ_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSM_IvSO_EENS0_14co_spawn_stateIT_SO_T1_vEEENUlvE_clEv + 50 11 chat_server 0x0000000100031855 _ZN4asio6detail7binder0IZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS3_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSB_ENS0_17initiate_co_spawnISD_EEJNS0_21awaitable_as_functionIvSD_EEEEESL_EEESD_SK_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSN_IvSP_EENS0_14co_spawn_stateIT_SP_T1_vEEEUlvE_EclEv + 21 12 chat_server 0x0000000100033bb5 _ZN4asio19asio_handler_invokeINS_6detail7binder0IZNS1_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS4_18wait_for_one_errorENS1_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSC_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSC_ENS1_17initiate_co_spawnISE_EEJNS1_21awaitable_as_functionIvSE_EEEEESM_EEESE_SL_EENS_9awaitableINS1_28awaitable_thread_entry_pointET0_EEPNSO_IvSQ_EENS1_14co_spawn_stateIT_SQ_T1_vEEEUlvE_EEEEvRSV_z + 21 13 chat_server 0x0000000100033b8f _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail7binder0IZNS2_20co_spawn_entry_pointINS1_12experimental6detail25parallel_group_op_handlerILm0ENS5_18wait_for_one_errorENS2_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSD_ENS1_15any_io_executorEEEJNS1_24deferred_async_operationIFvSD_ENS2_17initiate_co_spawnISF_EEJNS2_21awaitable_as_functionIvSF_EEEEESN_EEESF_SM_EENS1_9awaitableINS2_28awaitable_thread_entry_pointET0_EEPNSP_IvSR_EENS2_14co_spawn_stateIT_SR_T1_vEEEUlvE_EESZ_EEvRSW_RSR_ + 31 14 chat_server 0x0000000100033b5d _ZN4asio6detail19asio_handler_invokeINS0_7binder0IZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS4_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSC_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSC_ENS0_17initiate_co_spawnISE_EEJNS0_21awaitable_as_functionIvSE_EEEEESM_EEESE_SL_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSO_IvSQ_EENS0_14co_spawn_stateIT_SQ_T1_vEEEUlvE_EESY_EEvRSV_PNS2_ISQ_EE + 29 15 chat_server 0x0000000100033abd _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail7binder0IZNS2_20co_spawn_entry_pointINS1_12experimental6detail25parallel_group_op_handlerILm0ENS5_18wait_for_one_errorENS2_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSD_ENS1_15any_io_executorEEEJNS1_24deferred_async_operationIFvSD_ENS2_17initiate_co_spawnISF_EEJNS2_21awaitable_as_functionIvSF_EEEEESN_EEESF_SM_EENS1_9awaitableINS2_28awaitable_thread_entry_pointET0_EEPNSP_IvSR_EENS2_14co_spawn_stateIT_SR_T1_vEEEUlvE_EES10_EEvRSW_RSR_ + 29 16 chat_server 0x00000001000339ae _ZN4asio6detail17executor_function8completeINS0_7binder0IZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS5_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSD_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSD_ENS0_17initiate_co_spawnISF_EEJNS0_21awaitable_as_functionIvSF_EEEEESN_EEESF_SM_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSP_IvSR_EENS0_14co_spawn_stateIT_SR_T1_vEEEUlvE_EENSA_9allocatorIvEEEEvPNS1_9impl_baseEb + 110 17 chat_server 0x0000000100016692 _ZN4asio6detail17executor_functionclEv + 66 18 chat_server 0x0000000100016645 _ZN4asio19asio_handler_invokeINS_6detail17executor_functionEEEvRT_z + 21 19 chat_server 0x00000001000160af _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail17executor_functionES3_EEvRT_RT0_ + 31 20 chat_server 0x0000000100026ef3 _ZNK4asio10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EE7executeINS_6detail17executor_functionEEEvOT_ + 115 21 chat_server 0x0000000100026e71 _ZNK25asio_execution_execute_fn4implclIRKN4asio10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EEENS2_6detail17executor_functionEEENS5_9enable_ifIXeqsr11call_traitsIS0_T_FvT0_EEE8overloadLNS_13overload_typeE0EENS_11call_traitsIS0_SE_SG_vvvvvE11result_typeEE4typeEOSE_OSF_ + 33 22 chat_server 0x0000000100026d31 _ZN4asio9execution6detail17any_executor_base10execute_exINS_10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EEEEEvRKS2_ONS_6detail17executor_functionE + 49 23 chat_server 0x00000001000316a5 _ZNK4asio9execution6detail17any_executor_base7executeINS_6detail7binder0IZNS4_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS7_18wait_for_one_errorENS4_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSF_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSF_ENS4_17initiate_co_spawnISH_EEJNS4_21awaitable_as_functionIvSH_EEEEESP_EEESH_SO_EENS_9awaitableINS4_28awaitable_thread_entry_pointET0_EEPNSR_IvST_EENS4_14co_spawn_stateIT_ST_T1_vEEEUlvE_EEEEvOSY_ + 197 24 chat_server 0x0000000100031571 _ZNK25asio_execution_execute_fn4implclIRKN4asio15any_io_executorENS2_6detail7binder0IZNS6_20co_spawn_entry_pointINS2_12experimental6detail25parallel_group_op_handlerILm0ENS9_18wait_for_one_errorENS6_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSH_ES3_EEJNS2_24deferred_async_operationIFvSH_ENS6_17initiate_co_spawnIS3_EEJNS6_21awaitable_as_functionIvS3_EEEEESQ_EEES3_SP_EENS2_9awaitableINS6_28awaitable_thread_entry_pointET0_EEPNSS_IvSU_EENS6_14co_spawn_stateIT_SU_T1_vEEEUlvE_EEEENSE_9enable_ifIXeqsr11call_traitsIS0_SZ_FvSU_EEE8overloadLNS_13overload_typeE0EENS_11call_traitsIS0_SZ_S15_vvvvvE11result_typeEE4typeEOSZ_OSU_ + 33 25 chat_server 0x00000001000314f9 _ZNK4asio6detail31initiate_dispatch_with_executorINS_15any_io_executorEEclIZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS6_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSE_ES2_EEJNS_24deferred_async_operationIFvSE_ENS0_17initiate_co_spawnIS2_EEJNS0_21awaitable_as_functionIvS2_EEEEESN_EEES2_SM_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSP_IvSR_EENS0_14co_spawn_stateIT_SR_T1_vEEEUlvE_EEvOSW_PNSB_9enable_ifIXsr9execution11is_executorINSB_11conditionalILb1ES2_SW_E4typeEEE5valueEvE4typeEPNS11_IXntsr6detail27is_work_dispatcher_requiredINSB_5decayISW_E4typeES2_EE5valueEvE4typeE + 121 26 chat_server 0x0000000100031474 _ZN4asio6detail31completion_handler_async_resultIZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS3_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSB_ENS0_17initiate_co_spawnISD_EEJNS0_21awaitable_as_functionIvSD_EEEEESL_EEESD_SK_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSN_IvSP_EENS0_14co_spawn_stateIT_SP_T1_vEEEUlvE_JFvvEEE8initiateINS0_31initiate_dispatch_with_executorISD_EESX_JEEEvOSU_OSP_DpOT1_ + 36 27 chat_server 0x000000010003143d _ZN4asio14async_initiateIZNS_6detail20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS3_18wait_for_one_errorENS1_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSB_ENS1_17initiate_co_spawnISD_EEJNS1_21awaitable_as_functionIvSD_EEEEESL_EEESD_SK_EENS_9awaitableINS1_28awaitable_thread_entry_pointET0_EEPNSN_IvSP_EENS1_14co_spawn_stateIT_SP_T1_vEEEUlvE_JFvvEENS1_31initiate_dispatch_with_executorISD_EEJEEENS_10constraintIXsr6detail31async_result_has_initiate_memfnISU_DpT0_EE5valueEDTclsr12async_resultINS8_5decayISU_E4typeES13_EE8initiatecl7declvalIOSV_EEcl7declvalIOSU_EEspcl7declvalIOT2_EEEEE4typeES17_RSU_DpS1A_ + 29 28 chat_server 0x000000010003131f _ZN4asio8dispatchINS_15any_io_executorEZNS_6detail20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS4_18wait_for_one_errorENS2_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSC_ES1_EEJNS_24deferred_async_operationIFvSC_ENS2_17initiate_co_spawnIS1_EEJNS2_21awaitable_as_functionIvS1_EEEEESL_EEES1_SK_EENS_9awaitableINS2_28awaitable_thread_entry_pointET0_EEPNSN_IvSP_EENS2_14co_spawn_stateIT_SP_T1_vEEEUlvE_EEDaRKSU_OSP_NS_10constraintIXoosr9execution11is_executorISU_EE5valuesr11is_executorISU_EE5valueEiE4typeE + 63 29 chat_server 0x000000010003edca _ZN4asio6detail20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS2_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSA_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSA_ENS0_17initiate_co_spawnISC_EEJNS0_21awaitable_as_functionIvSC_EEEEESK_EEESC_SJ_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSM_IvSO_EENS0_14co_spawn_stateIT_SO_T1_vEE.resume + 2378 30 chat_server 0x000000010000a308 _ZNKSt3__116coroutine_handleIvE6resumeB6v15006Ev + 24 31 chat_server 0x000000010000a1ee _ZN4asio6detail20awaitable_frame_baseINS_15any_io_executorEE6resumeEv + 46 32 chat_server 0x0000000100009f54 _ZN4asio6detail16awaitable_threadINS_15any_io_executorEE4pumpEv + 36 33 chat_server 0x00000001000327ba _ZN4asio6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS5_ENS_15any_io_executorEEclIJRS4_S5_S5_EEEvDpOT_ + 122 34 chat_server 0x000000010003273a _ZN4asio12experimental6detail33parallel_group_completion_handlerINS_6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS8_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS8_ENS3_17initiate_co_spawnISA_EEJNS3_21awaitable_as_functionIvSA_EEEEESI_EE6invokeIJS8_S8_EJLm0ELm1EEEEvONS5_5tupleIJDpT_EEENS5_16integer_sequenceImJXspT0_EEEE + 74 35 chat_server 0x000000010003263d _ZN4asio12experimental6detail33parallel_group_completion_handlerINS_6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS8_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS8_ENS3_17initiate_co_spawnISA_EEJNS3_21awaitable_as_functionIvSA_EEEEESI_EE6invokeIJS8_S8_EEEvONS5_5tupleIJDpT_EEE + 29 36 chat_server 0x00000001000325df _ZN4asio12experimental6detail33parallel_group_completion_handlerINS_6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS8_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS8_ENS3_17initiate_co_spawnISA_EEJNS3_21awaitable_as_functionIvSA_EEEEESI_EE6invokeIJLm0ELm1EEEEvNS5_16integer_sequenceImJXspT_EEEE + 95 37 chat_server 0x0000000100032575 _ZN4asio12experimental6detail33parallel_group_completion_handlerINS_6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS8_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS8_ENS3_17initiate_co_spawnISA_EEJNS3_21awaitable_as_functionIvSA_EEEEESI_EEclEv + 21 38 chat_server 0x0000000100032555 _ZN4asio6detail7binder0INS_12experimental6detail33parallel_group_completion_handlerINS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS9_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS9_ENS0_17initiate_co_spawnISB_EEJNS0_21awaitable_as_functionIvSB_EEEEESJ_EEEEclEv + 21 39 chat_server 0x0000000100033635 _ZN4asio19asio_handler_invokeINS_6detail7binder0INS_12experimental6detail33parallel_group_completion_handlerINS1_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSA_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSA_ENS1_17initiate_co_spawnISC_EEJNS1_21awaitable_as_functionIvSC_EEEEESK_EEEEEEEvRT_z + 21 40 chat_server 0x000000010003360f _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail7binder0INS1_12experimental6detail33parallel_group_completion_handlerINS2_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS1_15any_io_executorEEEJNS1_24deferred_async_operationIFvSB_ENS2_17initiate_co_spawnISD_EEJNS2_21awaitable_as_functionIvSD_EEEEESL_EEEEESM_EEvRT_RT0_ + 31 41 chat_server 0x00000001000335dd _ZN4asio6detail19asio_handler_invokeINS0_7binder0INS_12experimental6detail33parallel_group_completion_handlerINS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSA_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSA_ENS0_17initiate_co_spawnISC_EEJNS0_21awaitable_as_functionIvSC_EEEEESK_EEEEESL_EEvRT_PNS2_IT0_EE + 29 42 chat_server 0x000000010003352d _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail7binder0INS1_12experimental6detail33parallel_group_completion_handlerINS2_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS1_15any_io_executorEEEJNS1_24deferred_async_operationIFvSB_ENS2_17initiate_co_spawnISD_EEJNS2_21awaitable_as_functionIvSD_EEEEESL_EEEEESN_EEvRT_RT0_ + 29 43 chat_server 0x00000001000333e6 _ZN4asio6detail17executor_function8completeINS0_7binder0INS_12experimental6detail33parallel_group_completion_handlerINS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSB_ENS0_17initiate_co_spawnISD_EEJNS0_21awaitable_as_functionIvSD_EEEEESL_EEEEENS8_9allocatorIvEEEEvPNS1_9impl_baseEb + 166 44 chat_server 0x0000000100016692 _ZN4asio6detail17executor_functionclEv + 66 45 chat_server 0x0000000100016645 _ZN4asio19asio_handler_invokeINS_6detail17executor_functionEEEvRT_z + 21 46 chat_server 0x00000001000160af _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail17executor_functionES3_EEvRT_RT0_ + 31 47 chat_server 0x0000000100026ef3 _ZNK4asio10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EE7executeINS_6detail17executor_functionEEEvOT_ + 115 48 chat_server 0x0000000100026e71 _ZNK25asio_execution_execute_fn4implclIRKN4asio10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EEENS2_6detail17executor_functionEEENS5_9enable_ifIXeqsr11call_traitsIS0_T_FvT0_EEE8overloadLNS_13overload_typeE0EENS_11call_traitsIS0_SE_SG_vvvvvE11result_typeEE4typeEOSE_OSF_ + 33 49 chat_server 0x0000000100026d31 _ZN4asio9execution6detail17any_executor_base10execute_exINS_10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EEEEEvRKS2_ONS_6detail17executor_functionE + 49 50 chat_server 0x0000000100032368 _ZNK4asio9execution6detail17any_executor_base7executeINS_6detail7binder0INS_12experimental6detail33parallel_group_completion_handlerINS4_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSD_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSD_ENS4_17initiate_co_spawnISF_EEJNS4_21awaitable_as_functionIvSF_EEEEESN_EEEEEEEvOT_ + 280 51 chat_server 0x0000000100032161 _ZNK25asio_execution_execute_fn4implclIRN4asio15any_io_executorENS2_6detail7binder0INS2_12experimental6detail33parallel_group_completion_handlerINS5_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSE_ES3_EEJNS2_24deferred_async_operationIFvSE_ENS5_17initiate_co_spawnIS3_EEJNS5_21awaitable_as_functionIvS3_EEEEESN_EEEEEEENSB_9enable_ifIXeqsr11call_traitsIS0_T_FvT0_EEE8overloadLNS_13overload_typeE0EENS_11call_traitsIS0_SR_ST_vvvvvE11result_typeEE4typeEOSR_OSS_ + 33 52 chat_server 0x0000000100032062 _ZNK4asio6detail17initiate_dispatchclINS_12experimental6detail33parallel_group_completion_handlerINS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSA_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSA_ENS0_17initiate_co_spawnISC_EEJNS0_21awaitable_as_functionIvSC_EEEEESK_EEEEEvOT_PNS7_9enable_ifIXsr9execution11is_executorINS_19associated_executorINS7_5decayISM_E4typeENS_21basic_system_executorINS_9execution6detail8blocking10possibly_tILi0EEENSV_12relationship6fork_tILi0EEENS7_9allocatorIvEEEEE4typeEEE5valueEvE4typeE + 178 53 chat_server 0x0000000100031fa1 _ZN4asio6detail31completion_handler_async_resultINS_12experimental6detail33parallel_group_completion_handlerINS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS9_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS9_ENS0_17initiate_co_spawnISB_EEJNS0_21awaitable_as_functionIvSB_EEEEESJ_EEEJFvvEEE8initiateINS0_17initiate_dispatchESK_JEEEvOT_OT0_DpOT1_ + 33 54 chat_server 0x0000000100031f6d _ZN4asio14async_initiateINS_12experimental6detail33parallel_group_completion_handlerINS_6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS9_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS9_ENS4_17initiate_co_spawnISB_EEJNS4_21awaitable_as_functionIvSB_EEEEESJ_EEEJFvvEENS4_17initiate_dispatchEJEEENS_10constraintIXsr6detail31async_result_has_initiate_memfnIT_DpT0_EE5valueEDTclsr12async_resultINS6_5decayISO_E4typeESQ_EE8initiatecl7declvalIOT1_EEcl7declvalIOSO_EEspcl7declvalIOT2_EEEEE4typeESV_RSO_DpSY_ + 29 55 chat_server 0x0000000100031bb9 _ZN4asio8dispatchINS_12experimental6detail33parallel_group_completion_handlerINS_6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS9_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS9_ENS4_17initiate_co_spawnISB_EEJNS4_21awaitable_as_functionIvSB_EEEEESJ_EEEEEDaOT_ + 25 56 chat_server 0x0000000100031a3d _ZN4asio12experimental6detail25parallel_group_op_handlerILm0ENS0_18wait_for_one_errorENS_6detail26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrS9_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvS9_ENS4_17initiate_co_spawnISB_EEJNS4_21awaitable_as_functionIvSB_EEEEESJ_EEclIJS9_EEEvDpT_ + 365 57 chat_server 0x0000000100031892 _ZZN4asio6detail20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS2_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSA_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSA_ENS0_17initiate_co_spawnISC_EEJNS0_21awaitable_as_functionIvSC_EEEEESK_EEESC_SJ_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSM_IvSO_EENS0_14co_spawn_stateIT_SO_T1_vEEENUlvE_clEv + 50 58 chat_server 0x0000000100031855 _ZN4asio6detail7binder0IZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS3_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSB_ENS0_17initiate_co_spawnISD_EEJNS0_21awaitable_as_functionIvSD_EEEEESL_EEESD_SK_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSN_IvSP_EENS0_14co_spawn_stateIT_SP_T1_vEEEUlvE_EclEv + 21 59 chat_server 0x0000000100033bb5 _ZN4asio19asio_handler_invokeINS_6detail7binder0IZNS1_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS4_18wait_for_one_errorENS1_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSC_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSC_ENS1_17initiate_co_spawnISE_EEJNS1_21awaitable_as_functionIvSE_EEEEESM_EEESE_SL_EENS_9awaitableINS1_28awaitable_thread_entry_pointET0_EEPNSO_IvSQ_EENS1_14co_spawn_stateIT_SQ_T1_vEEEUlvE_EEEEvRSV_z + 21 60 chat_server 0x0000000100033b8f _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail7binder0IZNS2_20co_spawn_entry_pointINS1_12experimental6detail25parallel_group_op_handlerILm0ENS5_18wait_for_one_errorENS2_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSD_ENS1_15any_io_executorEEEJNS1_24deferred_async_operationIFvSD_ENS2_17initiate_co_spawnISF_EEJNS2_21awaitable_as_functionIvSF_EEEEESN_EEESF_SM_EENS1_9awaitableINS2_28awaitable_thread_entry_pointET0_EEPNSP_IvSR_EENS2_14co_spawn_stateIT_SR_T1_vEEEUlvE_EESZ_EEvRSW_RSR_ + 31 61 chat_server 0x0000000100033b5d _ZN4asio6detail19asio_handler_invokeINS0_7binder0IZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS4_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSC_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSC_ENS0_17initiate_co_spawnISE_EEJNS0_21awaitable_as_functionIvSE_EEEEESM_EEESE_SL_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSO_IvSQ_EENS0_14co_spawn_stateIT_SQ_T1_vEEEUlvE_EESY_EEvRSV_PNS2_ISQ_EE + 29 62 chat_server 0x0000000100033abd _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail7binder0IZNS2_20co_spawn_entry_pointINS1_12experimental6detail25parallel_group_op_handlerILm0ENS5_18wait_for_one_errorENS2_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSD_ENS1_15any_io_executorEEEJNS1_24deferred_async_operationIFvSD_ENS2_17initiate_co_spawnISF_EEJNS2_21awaitable_as_functionIvSF_EEEEESN_EEESF_SM_EENS1_9awaitableINS2_28awaitable_thread_entry_pointET0_EEPNSP_IvSR_EENS2_14co_spawn_stateIT_SR_T1_vEEEUlvE_EES10_EEvRSW_RSR_ + 29 63 chat_server 0x00000001000339ae _ZN4asio6detail17executor_function8completeINS0_7binder0IZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS5_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSD_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSD_ENS0_17initiate_co_spawnISF_EEJNS0_21awaitable_as_functionIvSF_EEEEESN_EEESF_SM_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSP_IvSR_EENS0_14co_spawn_stateIT_SR_T1_vEEEUlvE_EENSA_9allocatorIvEEEEvPNS1_9impl_baseEb + 110 64 chat_server 0x0000000100016692 _ZN4asio6detail17executor_functionclEv + 66 65 chat_server 0x0000000100016645 _ZN4asio19asio_handler_invokeINS_6detail17executor_functionEEEvRT_z + 21 66 chat_server 0x00000001000160af _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail17executor_functionES3_EEvRT_RT0_ + 31 67 chat_server 0x0000000100026ef3 _ZNK4asio10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EE7executeINS_6detail17executor_functionEEEvOT_ + 115 68 chat_server 0x0000000100026e71 _ZNK25asio_execution_execute_fn4implclIRKN4asio10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EEENS2_6detail17executor_functionEEENS5_9enable_ifIXeqsr11call_traitsIS0_T_FvT0_EEE8overloadLNS_13overload_typeE0EENS_11call_traitsIS0_SE_SG_vvvvvE11result_typeEE4typeEOSE_OSF_ + 33 69 chat_server 0x0000000100026d31 _ZN4asio9execution6detail17any_executor_base10execute_exINS_10io_context19basic_executor_typeINSt3__19allocatorIvEELm4EEEEEvRKS2_ONS_6detail17executor_functionE + 49 70 chat_server 0x00000001000316a5 _ZNK4asio9execution6detail17any_executor_base7executeINS_6detail7binder0IZNS4_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS7_18wait_for_one_errorENS4_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSF_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSF_ENS4_17initiate_co_spawnISH_EEJNS4_21awaitable_as_functionIvSH_EEEEESP_EEESH_SO_EENS_9awaitableINS4_28awaitable_thread_entry_pointET0_EEPNSR_IvST_EENS4_14co_spawn_stateIT_ST_T1_vEEEUlvE_EEEEvOSY_ + 197 71 chat_server 0x0000000100031571 _ZNK25asio_execution_execute_fn4implclIRKN4asio15any_io_executorENS2_6detail7binder0IZNS6_20co_spawn_entry_pointINS2_12experimental6detail25parallel_group_op_handlerILm0ENS9_18wait_for_one_errorENS6_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSH_ES3_EEJNS2_24deferred_async_operationIFvSH_ENS6_17initiate_co_spawnIS3_EEJNS6_21awaitable_as_functionIvS3_EEEEESQ_EEES3_SP_EENS2_9awaitableINS6_28awaitable_thread_entry_pointET0_EEPNSS_IvSU_EENS6_14co_spawn_stateIT_SU_T1_vEEEUlvE_EEEENSE_9enable_ifIXeqsr11call_traitsIS0_SZ_FvSU_EEE8overloadLNS_13overload_typeE0EENS_11call_traitsIS0_SZ_S15_vvvvvE11result_typeEE4typeEOSZ_OSU_ + 33 72 chat_server 0x00000001000314f9 _ZNK4asio6detail31initiate_dispatch_with_executorINS_15any_io_executorEEclIZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS6_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSE_ES2_EEJNS_24deferred_async_operationIFvSE_ENS0_17initiate_co_spawnIS2_EEJNS0_21awaitable_as_functionIvS2_EEEEESN_EEES2_SM_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSP_IvSR_EENS0_14co_spawn_stateIT_SR_T1_vEEEUlvE_EEvOSW_PNSB_9enable_ifIXsr9execution11is_executorINSB_11conditionalILb1ES2_SW_E4typeEEE5valueEvE4typeEPNS11_IXntsr6detail27is_work_dispatcher_requiredINSB_5decayISW_E4typeES2_EE5valueEvE4typeE + 121 73 chat_server 0x0000000100031474 _ZN4asio6detail31completion_handler_async_resultIZNS0_20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS3_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSB_ENS0_17initiate_co_spawnISD_EEJNS0_21awaitable_as_functionIvSD_EEEEESL_EEESD_SK_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSN_IvSP_EENS0_14co_spawn_stateIT_SP_T1_vEEEUlvE_JFvvEEE8initiateINS0_31initiate_dispatch_with_executorISD_EESX_JEEEvOSU_OSP_DpOT1_ + 36 74 chat_server 0x000000010003143d _ZN4asio14async_initiateIZNS_6detail20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS3_18wait_for_one_errorENS1_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSB_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSB_ENS1_17initiate_co_spawnISD_EEJNS1_21awaitable_as_functionIvSD_EEEEESL_EEESD_SK_EENS_9awaitableINS1_28awaitable_thread_entry_pointET0_EEPNSN_IvSP_EENS1_14co_spawn_stateIT_SP_T1_vEEEUlvE_JFvvEENS1_31initiate_dispatch_with_executorISD_EEJEEENS_10constraintIXsr6detail31async_result_has_initiate_memfnISU_DpT0_EE5valueEDTclsr12async_resultINS8_5decayISU_E4typeES13_EE8initiatecl7declvalIOSV_EEcl7declvalIOSU_EEspcl7declvalIOT2_EEEEE4typeES17_RSU_DpS1A_ + 29 75 chat_server 0x000000010003131f _ZN4asio8dispatchINS_15any_io_executorEZNS_6detail20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS4_18wait_for_one_errorENS2_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSC_ES1_EEJNS_24deferred_async_operationIFvSC_ENS2_17initiate_co_spawnIS1_EEJNS2_21awaitable_as_functionIvS1_EEEEESL_EEES1_SK_EENS_9awaitableINS2_28awaitable_thread_entry_pointET0_EEPNSN_IvSP_EENS2_14co_spawn_stateIT_SP_T1_vEEEUlvE_EEDaRKSU_OSP_NS_10constraintIXoosr9execution11is_executorISU_EE5valuesr11is_executorISU_EE5valueEiE4typeE + 63 76 chat_server 0x000000010003edca _ZN4asio6detail20co_spawn_entry_pointINS_12experimental6detail25parallel_group_op_handlerILm0ENS2_18wait_for_one_errorENS0_26awaitable_async_op_handlerIFvNSt3__15arrayImLm2EEESt13exception_ptrSA_ENS_15any_io_executorEEEJNS_24deferred_async_operationIFvSA_ENS0_17initiate_co_spawnISC_EEJNS0_21awaitable_as_functionIvSC_EEEEESK_EEESC_SJ_EENS_9awaitableINS0_28awaitable_thread_entry_pointET0_EEPNSM_IvSO_EENS0_14co_spawn_stateIT_SO_T1_vEE.resume + 2378 77 chat_server 0x000000010000a308 _ZNKSt3__116coroutine_handleIvE6resumeB6v15006Ev + 24 78 chat_server 0x000000010000a1ee _ZN4asio6detail20awaitable_frame_baseINS_15any_io_executorEE6resumeEv + 46 79 chat_server 0x0000000100009f54 _ZN4asio6detail16awaitable_threadINS_15any_io_executorEE4pumpEv + 36 80 chat_server 0x0000000100009dc5 _ZN4asio6detail17awaitable_handlerINS_15any_io_executorEJNSt3__110error_codeEEEclERKS4_ + 149 81 chat_server 0x0000000100009d1c _ZN4asio6detail7binder1INS0_17awaitable_handlerINS_15any_io_executorEJNSt3__110error_codeEEEES5_EclEv + 28 82 chat_server 0x0000000100009cf5 _ZN4asio19asio_handler_invokeINS_6detail7binder1INS1_17awaitable_handlerINS_15any_io_executorEJNSt3__110error_codeEEEES6_EEEEvRT_z + 21 83 chat_server 0x0000000100009c9f _ZN27asio_handler_invoke_helpers6invokeIN4asio6detail7binder1INS2_17awaitable_handlerINS1_15any_io_executorEJNSt3__110error_codeEEEES7_EES8_EEvRT_RT0_ + 31 84 chat_server 0x000000010000874b _ZN4asio6detail12handler_workINS0_17awaitable_handlerINS_15any_io_executorEJNSt3__110error_codeEEEES3_vE8completeINS0_7binder1IS6_S5_EEEEvRT_RS6_ + 75 85 chat_server 0x0000000100008475 _ZN4asio6detail12wait_handlerINS0_17awaitable_handlerINS_15any_io_executorEJNSt3__110error_codeEEEES3_E11do_completeEPvPNS0_19scheduler_operationERKS5_m + 245 86 chat_server 0x0000000100016dee _ZN4asio6detail19scheduler_operation8completeEPvRKNSt3__110error_codeEm + 46 87 chat_server 0x000000010001f54a _ZN4asio6detail9scheduler10do_run_oneERNS0_27conditionally_enabled_mutex11scoped_lockERNS0_21scheduler_thread_infoERKNSt3__110error_codeE + 490 88 chat_server 0x000000010001f14f _ZN4asio6detail9scheduler3runERNSt3__110error_codeE + 239 89 chat_server 0x00000001000060ae _ZN4asio10io_context3runEv + 46 90 chat_server 0x0000000100005f28 main + 88 91 dyld 0x0000000200172310 start + 2432 ```
battlmonstr commented 1 year ago

The related code has appeared here: https://github.com/chriskohlhoff/asio/commit/f0a1e1c7c0387ad16358c81eb52528f190df625c

mscottmueller commented 1 year ago

I have been tracking down this issue with a parallel_group running in wait_for_one_error() mode. I have two tasks running- one is a network sequence and the other is a wait on a steady_timer which gets signaled in that network sequence when that sequence completes with success or while it throws an exception. Do you know what a workaround for this issue might be? This code is unchanged in 1.83.

battlmonstr commented 1 year ago

@mscottmueller for now I've basically copy-pasted impl/co_spawn.hpp, renamed it to not conflict with asio::co_spawn, and reverted this there. That code was probably solving some other problem, but at least it stopped crashing for me. See workaround here.

mscottmueller commented 1 year ago

He's trying to make certain that the cancellation gets dispatched on the same executor- so we need to ensure that the signal is alive. Something like this should also work:

template <typename Handler, typename Executor, typename = void>
class co_spawn_cancellation_handler
{
public:
  co_spawn_cancellation_handler(const Handler& handler, const Executor& ex)
    : ptr_(std::make_shared<std::tuple<cancellation_signal, Executor>>(
        { {}, ex }))
  {
  } 

  cancellation_slot slot() { return std::get<0>(*ptr_).slot(); }

  void operator()(cancellation_type_t type)
  {
    boost::asio::dispatch(std::get<1>(*ptr_),
      [ptr_, type]() mutable
      {
        if (ptr_.use_count() == 1) // if we have the only reference here in this dispatch, we need to let go.
          return;
        cancellation_signal* sig = &std::get<0>(*ptr_);
        sig->emit(type);
      });
  }

private:
  std::shared_ptr<
    std::tuple<cancellation_signal, Executor>>
    ptr_;
};

With this code I don't see the crash and I believe the intent of the original change is preserved. EDIT: Nope, this doesn't work either. I get a crash from a different cancellation handler instead.

battlmonstr commented 5 months ago

Fixed by https://github.com/chriskohlhoff/asio/commit/22269a24385ab94879861957584c16cb6b0ff507 The co_spawn_cancellation_handler::signal_ is changed to be a shared_ptr. The fix is included in boost asio 1.83