runBlocking with Android's Main.immediate scheduler locks with other nested contexts

[dlew]

I've been trying to figure out why this code freezes on Android (when started on the main thread):

runBlocking {
  Log.i("test", "start runBlocking")
  withContext(Dispatchers.Main.immediate) {
    Log.i("test", "start Main.immediate context")
    withContext(Dispatchers.IO) {
      Log.i("test", "running IO context") // Prints
    }
    Log.i("test", "end Main.immediate context") // Never prints
  }
  Log.i("test", "end runBlocking") // Never prints
}

The inner context has to be something like Dispatchers.Default or Dispatchers.IO.

I've been able to track down where the code freezes: BlockingCoroutine.joinBlocking() eventually calls parkNanos(this, Long.MAX) and thus the code just stops. I don't understand coroutines well enough to understand how it got into this state, though.

This came up because it's easy to accidentally run into this situation when calling suspending functions that are using their own withContext() calls. E.g., you call runBlocking { someSuspendingFunction() }, not knowing that you're going to end up inside Main.immediate and then IO (resulting in a freeze).

[elizarov]

It freezes because you use runBlocking to block the main thread. It's called runBlocking because that is what it does -- block the thread. If you block the main thread, it freezes your UI.

Does it help?

[dlew]

I understand that part - I expect it to freeze the UI temporarily. However, the above code freezes the UI permanently, even though everything inside of it finishes execution.

In particular, somewhere within coroutines, it's choosing to wait park itself for 292 years (i.e. the parkNanos(this, Long.MAX) call) .

[elizarov]

Aha! Indeed, that's a bug. It even reproduces with a Swing main dispatcher with a pretty small self-contained reproducer:

import kotlinx.coroutines.*
import javax.swing.*

fun main() {
    SwingUtilities.invokeAndWait {
        runBlocking {
            println(1)
            withContext(Dispatchers.Main.immediate) {
                println(2)
                withContext(Dispatchers.IO) {
                    println(3)
                }
                println(4) // NOT REACHED
            }
            println(5)
        }
    }
}

[elizarov]

Here's what happens:

1) The main thread is blocked by runBlocking { ... }

2) The coroutines execution context is switched into Dispatchers.Main.immediate, which does not do anything since the execution is already in the main thread.

Note here, that if you try withContext(Dispatchers.Main) (non immediate) from inside of runBlocking, then it hangs, because the main thread is blocked and cannot execute a scheduled asynchronous task.

3) The execution context is switched into Dispatchers.IO

4) There is an attempt to switch from Dispatchers.IO to Dispatcher.Main but it deadlocks, because the main thread is blocked with runBlocking.

TL;DR: Don't block the main thread with runBlocking. It produces all sorts of weird stuff.

Can we somehow fix this particular case? It is not really clear. It is also not clear if we even should fix this particular case, since the fact that it does even hang at step 2 is already quite a miracle (a happy side effect of using an immediate variant of Dispatchers.Main).

[slavonnet]

This as i try catch in https://github.com/Kotlin/kotlinx.coroutines/pull/2374

[larssn]

So I ran into this in a Flutter plugin, which are always using the main thread by default. So I use runBlocking to offload some work into non-ui threads which has worked great. However I ran into a scenario where one of these threads needs to report back to the main thread, which caused the hang.

Is there any workaround, or ideas?

[qwwdfsad]

The best solution would be not to use runBlocking, but rather regular coroutines (e.g. launch(Dispatcher.Main) {})

[slavonnet]

1. Why can't you define that the main thread is already blocked and just add a child? A link to the main coroutine can be stored in a variable local to the thread.

2. Why, if a thread is blocked for a long time, it is impossible to release it every X seconds to draw the interface?

3. Why do I need to block the main thread? Why can't an exception be made for the main android thread and work through a looper? Instead of blocking and waiting, you can perform the next task from the looper at this point.

Why do you rely on the looper as an android specific, but refuse the main dispatcher to fully adapt to the android specificity?

[zach-klippenstein]

You're basically just describing all the benefits of using coroutines without runBlocking as @qwwdfsad suggested. Coroutines are designed to work that way when used correctly on platforms like Android that have their own loopers. runBlocking is only intended for use when such infrastructure does not exist.

[slavonnet]

Hm. Tell how. executedao request to room suspend in onViewCreated without runBlocking. run all dao calls before is stupid becouse logic tree call only 5% dao call. Room changes m,ay reactive flow changes from dao and as resu;t nexr dao sub runblocking call

Second mass case is runBlocking -> suspend fun -> nonsuspend fuc > runBlocking - > suspend fun

Thred is fragment restor (subfragmen)t create)e by fragment manager and recall fun with runBlocking second time

why you can detect that thread is blocked and dontr call blocking corrutent second time? We can't simple add job as cxhild? Transform runBlocking to structured corrutine!

[slavonnet]

i think need replace runBlockign to Main.immediate llaunch but without launch (to garant no context swithing between calls) Thread block replace to looper.post(tryResumeTask)

like

onViewCreated() {
corutineContext{ // add future to result and post to looper check result hangler call
suspend call

}
}

And.... will be great do this in SMART without any additional dsl. If i call suispend from non suspend it will convert block to corrutine and starttt it imeditly / Kotlin smart way .... no?

I dont need to block thread. I need result for now with suspending and yeld another main thread operatrions. If i init 5 fragments anmd on every i have 10 suspend calls - its can switchj betweed creating wthen first fragment wait dao results. Mass MassMass case is backgroud inflating many view parralel.. You add views to XML and don't have common executor. On every view you must add logic and don't bnlock twice thread

Think about reject run runBlocking on main thread if looper is used . Don't block looper in any variants and add switching to next task on suspend. This Scheme remove all latancy issues and will be very easy to understanding for newbirs. . ...... and if convert all measures fun to suspend and run itparralel in IO scope .... you will fix all frame blocking/freese issues. Next level is move m,measure to openmCL or shaders

[joffrey-bion]

The best solution would be not to use runBlocking, but rather regular coroutines (e.g. launch(Dispatcher.Main) {})

@qwwdfsad that's not always possible though. We just had a discussion on Slack about this.

Imagine the following situation: a library that you don't control provides some sort of interface that you can implement. The library is calling you back via a regular synchronous function that expects a result from you via a return value (the function you implement is not suspend and provides no async way like a Future or Deferred or callback to return that result). For example:

interface ThirdParty {
    fun doSomething(): SomeResult
}

In that case, the contract of that function is to block the calling thread until the result is ready, at least from the perpective of the caller. There is no way around that, you can dispatch what you want on any thread inside, you still have to wait for the result before giving back control to the caller on the calling thread - so it's essentially blocked (from the caller's point of view). It's how that API was designed. In that sense, that's what runBlocking is for, and the nice thing about runBlocking is that even though the thread is blocked from the caller's perspective, you can still use it to run your coroutines from within runBlocking's lambda (thanks to the special event loop dispatcher tied to the calling thread).

Now the only problem is that the magic of reusing the currently blocked thread doesn't go multiple levels deep. So in a situation where the library in question calls doSomething() on the main thread, you can make use of the main thread when directly inside runBlocking, but not when using runBlocking { withContext(Main) { ... }}. Maybe it should be ok?

suspend fun doStuffOnMain() = withContext(Main) { ... }

class MyImpl : ThridParty {
    // called on Main thread by the library
    override fun doSomething(): SomeResult {
        runBlocking {
            // we can run stuff on the calling thread here, so on the main thread
            paintStuffOnUI()

            // however, explicitly switching hangs if not `.immediate`
            doStuffOnMain()
        }
    }
}