search

vert-x3 / vertx-lang-kotlin

Vert.x for Kotlin
Apache License 2.0
296 stars 68 forks source link

StackOverflowError caused by two yield calls #233

Closed chemist777 closed 1 year ago

chemist777 commented 1 year ago

Version

Which version(s) did you encounter this bug ?

vertx-lang-kotlin-coroutines 4.4.2 kotlinx-coroutines-core: 1.6.1

Context

I encountered an exception while writing two consecutive yield calls in a CoroutineVerticle.

Do you have a reproducer?

A reproducer:

@Test
fun bug(): Unit = runBlocking {
    // it's a workaround to show correct exceptions instead of NoClassDefFoundError
    Class.forName("kotlinx.coroutines.CoroutineExceptionHandlerImplKt")
    Class.forName("kotlin.internal.PlatformImplementationsKt")
    // actual reproducer
    val vertx = Vertx.vertx()
    val dispatcher = vertx.orCreateContext.dispatcher()
    val scope = CoroutineScope(dispatcher + SupervisorJob())
    scope.async {
        (0 until 1000).forEach {
            yield()
            yield()
        }
    }.await()
}

Steps to reproduce

  1. Run the snippet.
  2. Observe a hang and the following StackOverflowError: 
    Exception in thread "vert.x-eventloop-thread-0 @coroutine#2" java.lang.RuntimeException: Exception while trying to handle coroutine exception
at kotlinx.coroutines.CoroutineExceptionHandlerKt.handlerException(CoroutineExceptionHandler.kt:38)
at kotlinx.coroutines.CoroutineExceptionHandlerKt.handleCoroutineException(CoroutineExceptionHandler.kt:29)
at kotlinx.coroutines.DispatchedTask.handleFatalException(DispatchedTask.kt:146)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:115)
at io.vertx.kotlin.coroutines.VertxCoroutineExecutor.execute(VertxCoroutine.kt:218)
at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:128)
at kotlinx.coroutines.CoroutineDispatcher.dispatchYield(CoroutineDispatcher.kt:150)
at kotlinx.coroutines.internal.DispatchedContinuation.dispatchYield$kotlinx_coroutines_core(DispatchedContinuation.kt:264)
at kotlinx.coroutines.YieldKt.yield(Yield.kt:37)
at io.coinmetrics.httpserver.ProcessingPoolTest$bug$1$1.invokeSuspend(ProcessingPoolTest.kt:115)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at io.vertx.kotlin.coroutines.VertxCoroutineExecutor.execute(VertxCoroutine.kt:218)
at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:128)
at kotlinx.coroutines.CoroutineDispatcher.dispatchYield(CoroutineDispatcher.kt:150)
at kotlinx.coroutines.internal.DispatchedContinuation.dispatchYield$kotlinx_coroutines_core(DispatchedContinuation.kt:264)
at kotlinx.coroutines.YieldKt.yield(Yield.kt:37)
at io.coinmetrics.httpserver.ProcessingPoolTest$bug$1$1.invokeSuspend(ProcessingPoolTest.kt:114)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at io.vertx.kotlin.coroutines.VertxCoroutineExecutor.execute(VertxCoroutine.kt:218)
at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:128)
at kotlinx.coroutines.CoroutineDispatcher.dispatchYield(CoroutineDispatcher.kt:150)
at kotlinx.coroutines.internal.DispatchedContinuation.dispatchYield$kotlinx_coroutines_core(DispatchedContinuation.kt:264)
at kotlinx.coroutines.YieldKt.yield(Yield.kt:37)
at io.coinmetrics.httpserver.ProcessingPoolTest$bug$1$1.invokeSuspend(ProcessingPoolTest.kt:115)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at io.vertx.kotlin.coroutines.VertxCoroutineExecutor.execute(VertxCoroutine.kt:218)
at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:128)
at kotlinx.coroutines.CoroutineDispatcher.dispatchYield(CoroutineDispatcher.kt:150)
at kotlinx.coroutines.internal.DispatchedContinuation.dispatchYield$kotlinx_coroutines_core(DispatchedContinuation.kt:264)
at kotlinx.coroutines.YieldKt.yield(Yield.kt:37)
at io.coinmetrics.httpserver.ProcessingPoolTest$bug$1$1.invokeSuspend(ProcessingPoolTest.kt:114)
    ... and so on (a recursion)

Most likely, the error is caused by the following fragment in VertxCoroutine.kt:

  override fun execute(command: Runnable) {
    if (Vertx.currentContext() != vertxContext) {
      vertxContext.runOnContext { command.run() }
    } else {
      command.run()
    }
  }

Extra

Java 11

vietj commented 1 year ago

what does happen with a regular kotlin coroutine ?

chemist777 commented 1 year ago

what does happen with a regular kotlin coroutine ?

The following code works well with regular thread pools. The problem is related to the thread pool managed by Vert.x/vertx-lang-kotlin.

@Test
fun `it works well with standard dispatchers`(): Unit = runBlocking {
    val dispatcher = Executors.newFixedThreadPool(4).asCoroutineDispatcher()
    val scope = CoroutineScope(dispatcher + SupervisorJob())
    scope.async {
        (0 until 1000).forEach {
            yield()
            yield()
        }
    }.await()
}
chemist777 commented 1 year ago

Any updates on this? I face this problem regularly when trying to use yield() in Verticles even with a single yield() call in some cases.

A stack trace from a real-world app when the latest coroutines and vert.x libraries are used:

Exception in thread "vert.x-eventloop-thread-7 @coroutine#14" kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[io.vertx.kotlin.coroutines.VertxCoroutineExecutor@566364d, Continuation at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:370)@58efcec1]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
    at kotlinx.coroutines.DispatchedTask.handleFatalException$kotlinx_coroutines_core(DispatchedTask.kt:146)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:117)
    at io.vertx.kotlin.coroutines.VertxCoroutineExecutor.execute(VertxCoroutine.kt:218)
    at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:135)
    at kotlinx.coroutines.CoroutineDispatcher.dispatchYield(CoroutineDispatcher.kt:150)
    at kotlinx.coroutines.internal.DispatchedContinuation.dispatchYield$kotlinx_coroutines_core(DispatchedContinuation.kt:264)
    at kotlinx.coroutines.YieldKt.yield(Yield.kt:36)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:370)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:358)
    at kotlinx.coroutines.flow.CancellableFlowImpl$collect$2.emit(Context.kt:275)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.emitAllImpl$FlowKt__ChannelsKt(Channels.kt:37)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.access$emitAllImpl$FlowKt__ChannelsKt(Channels.kt:1)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt$emitAllImpl$1.invokeSuspend(Channels.kt)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
    at io.vertx.kotlin.coroutines.VertxCoroutineExecutor.execute(VertxCoroutine.kt:218)
    at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:135)
    at kotlinx.coroutines.CoroutineDispatcher.dispatchYield(CoroutineDispatcher.kt:150)
    at kotlinx.coroutines.internal.DispatchedContinuation.dispatchYield$kotlinx_coroutines_core(DispatchedContinuation.kt:264)
    at kotlinx.coroutines.YieldKt.yield(Yield.kt:36)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:370)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:358)
    at kotlinx.coroutines.flow.CancellableFlowImpl$collect$2.emit(Context.kt:275)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.emitAllImpl$FlowKt__ChannelsKt(Channels.kt:37)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.access$emitAllImpl$FlowKt__ChannelsKt(Channels.kt:1)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt$emitAllImpl$1.invokeSuspend(Channels.kt)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
    at io.vertx.kotlin.coroutines.VertxCoroutineExecutor.execute(VertxCoroutine.kt:218)
    at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:135)
    at kotlinx.coroutines.CoroutineDispatcher.dispatchYield(CoroutineDispatcher.kt:150)
    at kotlinx.coroutines.internal.DispatchedContinuation.dispatchYield$kotlinx_coroutines_core(DispatchedContinuation.kt:264)
    at kotlinx.coroutines.YieldKt.yield(Yield.kt:36)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:370)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:358)
    at kotlinx.coroutines.flow.CancellableFlowImpl$collect$2.emit(Context.kt:275)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.emitAllImpl$FlowKt__ChannelsKt(Channels.kt:37)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.access$emitAllImpl$FlowKt__ChannelsKt(Channels.kt:1)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt$emitAllImpl$1.invokeSuspend(Channels.kt)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
    at io.vertx.kotlin.coroutines.VertxCoroutineExecutor.execute(VertxCoroutine.kt:218)
    at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:135)
    at kotlinx.coroutines.CoroutineDispatcher.dispatchYield(CoroutineDispatcher.kt:150)
    at kotlinx.coroutines.internal.DispatchedContinuation.dispatchYield$kotlinx_coroutines_core(DispatchedContinuation.kt:264)
    at kotlinx.coroutines.YieldKt.yield(Yield.kt:36)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:370)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:358)
    at kotlinx.coroutines.flow.CancellableFlowImpl$collect$2.emit(Context.kt:275)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.emitAllImpl$FlowKt__ChannelsKt(Channels.kt:37)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.access$emitAllImpl$FlowKt__ChannelsKt(Channels.kt:1)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt$emitAllImpl$1.invokeSuspend(Channels.kt)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
    at io.vertx.kotlin.coroutines.VertxCoroutineExecutor.execute(VertxCoroutine.kt:218)
    at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:135)
    at kotlinx.coroutines.CoroutineDispatcher.dispatchYield(CoroutineDispatcher.kt:150)
    at kotlinx.coroutines.internal.DispatchedContinuation.dispatchYield$kotlinx_coroutines_core(DispatchedContinuation.kt:264)
    at kotlinx.coroutines.YieldKt.yield(Yield.kt:36)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:370)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:358)
    at kotlinx.coroutines.flow.CancellableFlowImpl$collect$2.emit(Context.kt:275)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.emitAllImpl$FlowKt__ChannelsKt(Channels.kt:37)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.access$emitAllImpl$FlowKt__ChannelsKt(Channels.kt:1)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt$emitAllImpl$1.invokeSuspend(Channels.kt)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
    at io.vertx.kotlin.coroutines.VertxCoroutineExecutor.execute(VertxCoroutine.kt:218)
    at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:135)
    at kotlinx.coroutines.CoroutineDispatcher.dispatchYield(CoroutineDispatcher.kt:150)
    at kotlinx.coroutines.internal.DispatchedContinuation.dispatchYield$kotlinx_coroutines_core(DispatchedContinuation.kt:264)
    at kotlinx.coroutines.YieldKt.yield(Yield.kt:36)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:370)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:358)
    at kotlinx.coroutines.flow.CancellableFlowImpl$collect$2.emit(Context.kt:275)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.emitAllImpl$FlowKt__ChannelsKt(Channels.kt:37)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.access$emitAllImpl$FlowKt__ChannelsKt(Channels.kt:1)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt$emitAllImpl$1.invokeSuspend(Channels.kt)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
    at io.vertx.kotlin.coroutines.VertxCoroutineExecutor.execute(VertxCoroutine.kt:218)
    at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:135)
    at kotlinx.coroutines.CoroutineDispatcher.dispatchYield(CoroutineDispatcher.kt:150)
    at kotlinx.coroutines.internal.DispatchedContinuation.dispatchYield$kotlinx_coroutines_core(DispatchedContinuation.kt:264)
    at kotlinx.coroutines.YieldKt.yield(Yield.kt:36)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:370)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:358)
    at kotlinx.coroutines.flow.CancellableFlowImpl$collect$2.emit(Context.kt:275)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.emitAllImpl$FlowKt__ChannelsKt(Channels.kt:37)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.access$emitAllImpl$FlowKt__ChannelsKt(Channels.kt:1)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt$emitAllImpl$1.invokeSuspend(Channels.kt)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
    at io.vertx.kotlin.coroutines.VertxCoroutineExecutor.execute(VertxCoroutine.kt:218)
    at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:135)
    at kotlinx.coroutines.CoroutineDispatcher.dispatchYield(CoroutineDispatcher.kt:150)
    at kotlinx.coroutines.internal.DispatchedContinuation.dispatchYield$kotlinx_coroutines_core(DispatchedContinuation.kt:264)
    at kotlinx.coroutines.YieldKt.yield(Yield.kt:36)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:370)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:358)
    at kotlinx.coroutines.flow.CancellableFlowImpl$collect$2.emit(Context.kt:275)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.emitAllImpl$FlowKt__ChannelsKt(Channels.kt:37)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.access$emitAllImpl$FlowKt__ChannelsKt(Channels.kt:1)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt$emitAllImpl$1.invokeSuspend(Channels.kt)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
    at io.vertx.kotlin.coroutines.VertxCoroutineExecutor.execute(VertxCoroutine.kt:218)
    at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:135)
    at kotlinx.coroutines.CoroutineDispatcher.dispatchYield(CoroutineDispatcher.kt:150)
    at kotlinx.coroutines.internal.DispatchedContinuation.dispatchYield$kotlinx_coroutines_core(DispatchedContinuation.kt:264)
    at kotlinx.coroutines.YieldKt.yield(Yield.kt:36)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:370)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:358)
    at kotlinx.coroutines.flow.CancellableFlowImpl$collect$2.emit(Context.kt:275)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.emitAllImpl$FlowKt__ChannelsKt(Channels.kt:37)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.access$emitAllImpl$FlowKt__ChannelsKt(Channels.kt:1)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt$emitAllImpl$1.invokeSuspend(Channels.kt)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
    at io.vertx.kotlin.coroutines.VertxCoroutineExecutor.execute(VertxCoroutine.kt:218)
    at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:135)
    at kotlinx.coroutines.CoroutineDispatcher.dispatchYield(CoroutineDispatcher.kt:150)
    at kotlinx.coroutines.internal.DispatchedContinuation.dispatchYield$kotlinx_coroutines_core(DispatchedContinuation.kt:264)
    at kotlinx.coroutines.YieldKt.yield(Yield.kt:36)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:370)
    at io.coinmetrics.httpserver.impl.VertxHttpServerImpl$Verticle$sendResponse$3$1.emit(VertxHttpServerImpl.kt:358)
    at kotlinx.coroutines.flow.CancellableFlowImpl$collect$2.emit(Context.kt:275)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.emitAllImpl$FlowKt__ChannelsKt(Channels.kt:37)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt.access$emitAllImpl$FlowKt__ChannelsKt(Channels.kt:1)
    at kotlinx.coroutines.flow.FlowKt__ChannelsKt$emitAllImpl$1.invokeSuspend(Channels.kt)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
    at io.vertx.kotlin.coroutines.VertxCoroutineExecutor.execute(VertxCoroutine.kt:218)
    at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:135)
    at kotlinx.coroutines.CoroutineDispatcher.dispatchYield(CoroutineDispatcher.kt:150)
    at kotlinx.coroutines.internal.DispatchedContinuation.dispatchYield$kotlinx_coroutines_core(DispatchedContinuation.kt:264)
    at kotlinx.coroutines.YieldKt.yield(Yield.kt:36)
... stack is repeated in a loop ....
tsegismont commented 1 year ago

Fixed by #242