Support multi-threaded coroutines on Kotlin/Native

[elizarov]

You can have multiple threads in Kotlin/Native. ~Each thread can have its own event loop with runBlocking and have number of coroutines running there~. Currently communication between those threads via coroutine primitives (like channels) is not supported. This issue it to track enhancement of Kotlin/Native in kotlinx.coroutines library so that all the following becomes possible:

• Launching coroutines from one thread with a dispatcher on another thread
• Await/join coroutine running on another thread
• Send/Receive elements to/from coroutines on other threads

UPDATE: Currently, coroutines are supported only on the main thread. You cannot have coroutines off the main thread due to the way the library is currently structured.

UPDATE 2: the separate library version that supports Kotlin/Native multithreading is released on a regular basis. For the details and limitations, please follow kotlin-native-sharing.md document. The latest version: 1.5.2-native-mt

[brettwillis]

Do you have a ballpark time frame for implementing this (days, weeks, months, ...)? This will help me plan how to implement the first revision of our project. Thanks!

[mohit-gurumukhani]

Second that. Can we please get a rough estimate?

[elizarov]

We're in the design phase now. I'll update you on the status in couple of weeks.

[Alex009]

Have any progress?

[elizarov]

We have a work-in-progress branch in this repo with some of the code that is implemented, but it is way too complex a change, so the work there was stopped. It is hard to get it done in the current state. We've refocused our efforts on delivering high-quality single-threaded coroutines which work really well for sharing logic between Android and iOS UI apps (I highly recommend to checkout the code of KotlinConf app here https://github.com/JetBrains/kotlinconf-app). With respect to multithreading, we'll be back to drawing board to see how this story can be made easier to code with. Don't expect results soon, though.

[LouisCAD]

Does this mean it can already work without runBlocking, i.e. with launch? Because that's really something I'd like to advantage of to make multiplatform UI contracts.

On Thu, Oct 11, 2018, 11:03 PM Roman Elizarov notifications@github.com wrote:

We have a work-in-progress branch in this repo with some of the code that is implemented, but it is way too complex a change, so the work there was stopped. It is hard to get it done in the current state. We've refocused our efforts on delivering high-quality single-threaded coroutines which work really well for sharing logic between Android and iOS UI apps (I highly recommend to checkout the code of KotlinConf app here https://github.com/JetBrains/kotlinconf-app). With respect to multithreading, we'll be back to drawing board to see how this story can be made easier to code with. Don't expect results soon, though.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Kotlin/kotlinx.coroutines/issues/462#issuecomment-429117616, or mute the thread https://github.com/notifications/unsubscribe-auth/AGpvBbJNoE1CfiTsRyODDVV-FqOf-diVks5uj7ITgaJpZM4VjGFc .

[elizarov]

Yes, it works without runBlocking. The only extra effort you have to make, is you have to write a trivial UI CoroutineDispatcher for iOS. We don't include it in the library yet (that's issue #470), but you can copy-and-paste code from KotlinConf app (swift version here https://github.com/JetBrains/kotlinconf-app/blob/master/konfios/konfswift/ui/UI.swift) of from discussion in #470 (Kotlin version here https://github.com/Kotlin/kotlinx.coroutines/issues/470#issuecomment-414635811)

[luca992]

@elizarov Trying to convert Waiting for a job example to work without run blocking using (https://github.com/Kotlin/kotlinx.coroutines/issues/470#issuecomment-414635811) in a native macOs program. But I am still getting There is no event loop. Use runBlocking { ... } to start one. I think it probably is because my native program isn't starting NSRunLoop's mainloop. I can't quite figure it out.

I've tried starting the loop like:

fun main(args: Array<String>) {
    val job = GlobalScope.launch(MainLoopDispatcher) { // launch new coroutine and keep a reference to its Job
        delay(1000L)
        println("World!")
    }
    println("Hello,")
    NSRunLoop.currentRunLoop().runUntilDate(NSDate().dateByAddingTimeInterval(3.toDouble()))
}

But I don't think I'm doing that correctly, any ideas?

[qwwdfsad]

@luca992 please use runBlocking:

fun main(args: Array<String>) = runBlocking<Unit> { // <- This line
    val job = GlobalScope.launch(MainLoopDispatcher) {
        delay(1000L)
        println("World!")
    }
    println("Hello,")
}

[luca992]

@qwwdfsad I know that it works with run blocking... Are you saying it is only possible to run without runBlocking on iOS for some reason?

Edit: I'm using Qt for the UI in my native kotlin desktop app. Didn't figure out NSRunLoop. But, I figured out that if I run QGuiApplication.exec() inside runBlocking, I can call coroutines with Dispatchers.Unconfined. (And not have to wrap each one in runBlocking) .... Which is great beacuse now I can share presenters between the android app and the native desktop apps 👍

[LanderlYoung]

Is there any solution to support MultiThreaded coroutines yet? As far as I know, under current Kotlin/Native threading model, if an object is going to be shared between workers/threads, it must be either frozen or use a DetachedObjectGraph, while none of which works with CouroutineDispatcher, because we have a Continuation to pass through. Sadly the Continuation captures coroutine context (maybe more othre objects), which makes it impossible to froze or detach a Continuation.

IMHO, It's nearly impossible to implement a multi-threading coroutine dispatcher under current Kotlin/Native threading model. Should we redesign the threading model?

Maybe it is good for writing rebust code that, Kotlin/Native implement Lock/Thread/ThreadPool, and use those tools to implement coroutine. For those just want to offload jobs to different thread, it is good enough to use coroutine. And for those who cares very much about performace, give them the ability to use raw thread/thread pool. For example, to write a high performance low latency audio app, one usually create threads with the hieghest scheduling proiorty, and event bind those threads to a certain cpu core to eliminate context switch.

[LanderlYoung]

Current my solution is to totally move the threading part into native ios code. like this.

private val requestingHashMap = hashMapOf<String, IosHttpGetAgent.Callback>()

fun notifyHttpGetResponse(url: String, result: String?, error: String) {
    requestingHashMap.remove(url)?.onGetResult(url, result, error)
}

@Throws(IOException::class)
actual suspend fun httpGet(url: String): String {
    return suspendCoroutine { continuation ->
        val cb = object : IosHttpGetAgent.Callback {
            override fun onGetResult(url: String, result: String?, error: String) {
                if (result != null) {
                    continuation.resume(result)
                } else {
                    continuation.resumeWith(Result.failure(IOException(error)))
                }
            }
        }
        requestingHashMap[url] = cb
        iosHttpGetAgent.value!!.httpGet(url)
    }
}

While on the swift code.

    func httpGet(url: String) {

        let task = URLSession.shared.dataTask(with: URL(string: url)!) { (data, response, error) in
            if let resultData = data {
                DispatchQueue.main.async {
                    ActualKt.notifyHttpGetResponse(
                        url:url,
                        result: String(data: resultData, encoding: .utf8)!,
                        error: "success")
                }
            } else {
                DispatchQueue.main.async {
                    ActualKt.notifyHttpGetResponse(
                        url:url,
                        result: nil,
                        error: "success")
                }
            }
        }
        task.resume()

    }

So kotlin/native code runs totally on the main thread.

[elizarov]

Running totally on the main is the only solution for now. You can track #829 which will slightly expand your options and you'll be able to run coroutines separately on each threads (no easy way to communicate, though).

[LanderlYoung]

Hi, any progress on this issue? Or any possible solution for this issue? @elizarov I'll be glad to know about this. thanks.

[horita-yuya]

I'm also very concerned about this.

[SeekDaSky]

Well, the current state of multithreading in K/N is not really suitable for coroutines, the simple fact of giving a Continuation to a worker freeze the continuation, thus freezing the captured state and making it immutable (and pretty much unsuable).

For me multithreded coroutines is simply impossible with the current model.

[sellmair]

@SeekDaSky I am not very experienced with K/N's concurrency model nor with the way, coroutines work under the hood, but I would also want to see support for multi-threaded coroutines in K/N. Isn't there a way to make the continuation an immutable object that passes a new copy with the new state to the next K/N worker?

[SeekDaSky]

If I understand correctly you could detach the continuation and keep it mutable, but the state would not be accessible from another worker, so we still can't share values between threads.

We could heavily use channels and the actor paradigm to avoid any variable being shared, but this could lead to some performance degradation.

And this is just the developper side, developing a dispatcher with the limitations of the current concurrency model probably is daunting.

[sellmair]

No, this might be pretty naive, but wouldn't it be possible to capture the whole state of the coroutine in some structure (let's say a data class) and just pass a new copy to the next worker?

[SeekDaSky]

To achieve this I think you just have to detach the continuation and re-attach it inside the next worker but you still can't have two threads accessing the data at the same time. And this could lead to some weird side effect if you share a value between two continuations by mistake

[sellmair]

And this could lead to some weird side effect if you share a value between two continuations by mistake

Would you mind explaining this a little? I would be super interested ☺️

[SeekDaSky]

Well if you have a value that get caught in two distinct continuation, for example if you have two suspensions in two different methods that use the same property of an object, when the first continuation is detached, the second one will fail (because the shared value is now detached) and that would be a real pain to debug. And I don't know what would happen if a value is detached, re-attach in a worker and then detached from another one, there is so much corner cases that could lead to unexpected behaviour.

[sellmair]

Ah yeah, I got that, thanks! Obviously, the whole situation is way more complex than 'just expressing the state machine as an immutable structure', since you are able to access stuff outside of the scope of the suspending function itself. So while I really appreciate the work and ideas that went into K/N's concurrency model, I would rather prefer some concepts that coroutines bring to the table. For me personally (and most likely for our team), it would be much easier to work with concepts like Actors in our common source than handling the differences between the JVM and K/N. The new concurrency model (as nice as it might be) seems counter-productive towards multiplatform to me 😞

[qwwdfsad]

Your concerns about K/N memory model are valid (and pretty precise). In the current memory model, it is almost impossible to introduce multithreaded coroutines (at least implement already introduced common part to work with shared memory) that share the same semantics as JS and JVM primitives.

Yes it is possible to implement a separate library with detach/freeze semantics, actors and channels, but its API surface will be far from perfect and its impossible to write a common code with such primitives without constant fear of accidental freezings, detaches and in a way that some parts of the K/N-specific API do not leak into common and JVM code, so this is not something we are going to do.

But there is another way to address this problem: changes in K/N memory model. I am not going to discuss these changes (yet), but this is what we are aiming to. After these changes, we will implement a proper multithreaded K/N part of kotlinx.coroutines

[bwalter]

Thanks for the update @qwwdfsad, it seems to be a very great news! While the immutability check makes sense for a language like Rust where transfer rules are in the core of the language, it is very confusing in the case of K/N.

Looking forward to hearing about the memory model change!

[ildarsharafutdinov]

@qwwdfsad ,

changes in K/N memory model ... we are aiming to ... multithreaded K/N part of kotlinx.coroutines

Is there an ETA?

[qwwdfsad]

@ildarsharafutdinov no, there is no public ETA beyond "we will see a prototype in this year"

[SeekDaSky]

well that's a kind of ETA, is there a discussion/KEEP/forum post/whatever about the upcoming changes in K/N memory model ? I saw some movement in the Worker API but nothing that would improve the current situation.

[amelin]

Running totally on the main is the only solution for now. You can track #829 which will slightly expand your options and you'll be able to run coroutines separately on each threads (no easy way to communicate, though).

I'm confused. Are we still limited to running coroutines only on the main thread? Issue #829, mentioned earlier, was closed, but I don't understand how it relates to running "coroutines separately on each thread".

[elizarov]

I mean, that in the current version you can run coroutines separately in each thread.

[NilsLattek]

I mean, that in the current version you can run coroutines separately in each thread.

Should that also work on iOS? Because a dispatcher like

object MainLoopDispatcher: CoroutineDispatcher() {
    override fun dispatch(context: CoroutineContext, block: Runnable) {
        NSRunLoop.mainRunLoop().performBlock {
            block.run()
        }
    }
}

mentioned here: https://github.com/Kotlin/kotlinx.coroutines/issues/470#issuecomment-414635811 and here https://github.com/JetBrains/kotlinconf-app/blob/master/konfios/konfswift/ui/UI.swift will always just run on main thread. That means if I do something blocking in my code (call a non optimized framework) my UI will still freeze. So I am still looking for a way to run a coroutine on a non-main thread so that we can do heavy computations in it like here: https://github.com/Kotlin/kotlinx.coroutines/blob/master/ui/coroutines-guide-ui.md#blocking-operations-1

[elizarov]

While you can run different coroutines in different threads on Kotlin/Native now, there is no easy way to transfer data between threads in Kotlin/Native yet and withContext idiom to offload computation onto background threads would not work on Kotlin/Native as of now, because it entails accessing objects from different threads.

[NilsLattek]

Okay got it, thanks for clarification.

[amelin]

there is no easy way to transfer data between threads in Kotlin/Native yet and withContext idiom to offload computation onto background threads would not work

Actually, simple test shows that theses are not the only limitations. At least kotlinx.coroutines.delay suspend function is broken when called from non-main thread.

error output and sample code

version kotlin version 1.3.20, coroutines version 1.1.1. error output

zero delay works fine
non-zero delay fails with exception
kotlin.native.IncorrectDereferenceException: Trying to access top level value not marked as @ThreadLocal or @SharedImmutable from non-main thread
        at 0   main                                0x000000010a72fba6 ThrowIncorrectDereferenceException + 54
        at 1   main                                0x000000010a82a83e kfun:kotlinx.coroutines.delay(kotlin.Long) + 1870
        at 2   main                                0x000000010a58b782 kfun:example.BrokenDelayExample.$testDelay$lambda-1$COROUTINE$8.invokeSuspend(kotlin.Result)kotlin.Any? + 1346
        at 3   main                                0x000000010a7f03cc kfun:kotlinx.coroutines.DispatchedTask.run() + 3004
        at 4   main                                0x000000010a58c859 kfun:example.BrokenDelayExample.CurrentLoopDispatcher.$$FUNCTION_REFERENCE$11.invoke#internal + 89
        at 5   main                                0x000000010a937aa9 __platform_Foundation_kniBridge8343_block_invoke + 25
        at 6   CoreFoundation                      0x000000010d2b062c __CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
        at 7   CoreFoundation                      0x000000010d2afde0 __CFRunLoopDoBlocks + 336
        at 8   CoreFoundation                      0x000000010d2aa62e __CFRunLoopRun + 1246
        at 9   CoreFoundation                      0x000000010d2a9e11 CFRunLoopRunSpecific + 625
        at 10  Foundation                          0x000000010b731322 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 277
        at 11  main                                0x000000010a937855 platform_Foundation_kniBridge8340 + 85
        at 12  main                                0x000000010a58a045 kfun:example.BrokenDelayExample.$$FUNCTION_REFERENCE$10.invoke#internal + 1765

sample code ```kotlin package example import kotlinx.coroutines.* import platform.Foundation.* import kotlin.coroutines.CoroutineContext import kotlin.native.concurrent.freeze class BrokenDelayExample { fun run() { run(::testDelay) } private fun run(block: () -> Unit) { val blockWithRunLoop = { block() while (NSRunLoop.currentRunLoop.runMode(NSDefaultRunLoopMode, NSDate.distantFuture)) { // loop } NSThread.exit() } NSThread(blockWithRunLoop.freeze()) .start() } private fun testDelay() { CoroutineScope(CurrentLoopDispatcher() + Job()) .launch { delay(0) println("zero delay works fine") try { delay(50) } catch (ex: RuntimeException) { println("non-zero delay fails with exception") ex.printStackTrace() } } } class CurrentLoopDispatcher : CoroutineDispatcher() { override fun dispatch(context: CoroutineContext, block: Runnable) { NSRunLoop.currentRunLoop().performBlock { block.run() } } } } ```

[elizarov]

Unfortunately, that is true.

[bwalter]

Regarding memory ownership and concurrency issues, I really wish that Kotlin would handle "fearless concurrency" in a better way and more generally provide mechanisms like "exclusive class" like Rust. Even Swift is slightly moving into that direction.

[elizarov]

There are some non-trivial compromises with respect to other langauge features:

• Swift requires manual annotation of weak references, providing only limited form of automated garbage collection (only for non-cyclic garbage). Another compromise is that Swift programs corrupt memory (crash/coredump) on data race. We consider the later to be acceptable compromise, but not the former. We do belive in fully automatic memory management.

• Rust requires additional ownership annotations from developer. It forces developers to explicitly think about memory ownership and its transfer, so that you can have explicit control on memory allocations, but in a safe way. This is a great feature for an systems language, but we do belive that it is an extraneous concern for application developers that Kotlin targets.

[bwalter]

Thanks for the clarifications. The clear language design and its differentiation from Swift and Rust perfectly make sense.

Regarding compromises and runtime exceptions (vs manual annotations), the way optionals have been introduced into the language may be the perfect example to follow. While Java's null pointer exceptions did not bring much more (besides security aspects) than C/C++'s segmentation faults, the introduction of (semi-) manual annotations for optionals ('?') did definitly succeed to eliminate the issue without any significant drawback regarding ease of use.

Of course racing conditions is a much more complex problem to solve and the analogy with null pointers may not be completly fair but I still hope that the issue could be eliminated in an elegant way. (Optionally) marking instances and (their references) as mutable vs read-only might be part of the solution but dealing with nested structures makes it hard to achieve that goal. The introduction of mutability concepts into the language is generally a very complicated challenge.

The runtime restrictions introduced in Kotlin/Native (together with the possibility to freeze objects) is probably what confused the most as it adds strong restrictions (i.e. this issue) without solving the real problem. I would be personally happy enough if the new memory model could relax those restrictions, allow (unsafe) concurrent access to instances and keep the language consistent with its JVM variant.

[kylejbrock]

I concur, the forced immutability in order to share data between threads is single handedly the most frustrating aspect of K/N (I recognize there are ways around it, but none of them are desirable). Solving race conditions isn't a problem I want the language to solve for me, personally. I believe platforms have sufficient mechanisms for this already. It massively breaks what I want from the multiplatform paradigm I was hoping for from Kotlin. Specifically between iOS & Android. I otherwise think it's great and will continue to use it. I'll just grumble more than I would otherwise like.

[kpgalligan]

I'll add the dissenting point here. I don't like all aspects of the native concurrency, but I like the general direction, and I think simply reintroducing unsafe memory access to conform to the JVM would be a lost opportunity. I do think not having MT coroutines is very frustrating, but just dumping the concurrency constraints isn't the answer.

"adds strong restrictions (i.e. this issue) without solving the real problem"

I think it solves the problem of runtime verification of safe concurrent access, which is the point. How all these things resolve, we'll see, but if we're expressing desires, I vote to not simply abandon the concurrency efforts.

[benasher44]

UPDATE: Currently, coroutines are supported only on the main thread. You cannot have coroutines off the main thread due to the way the library is currently structured.

Is this still accurate? Is it not possible to use runBlocking on a background thread to execute some coroutine code using an unconfined dispatcher? I'm looking for a short term workaround to be able to write our code using coroutines, but still run it on a background thread (don't mind doing the work to ensure it all runs on one thread). Something along the lines of:

worker.execute({}) {
    runBlocking(Dispatchers.Unconfined) {
        launch {
            // do work
        }
    }
}

[benasher44]

Okay so I tried that ^. You can get it to work sometimes, but it's extremely fragile. Eventually, you run into coroutines internals that aren't thread safe, and then it blows up. FWIW, this exercise made me more excited/happy about the native concurrency model. It caught a bunch of problems quickly, without which I would have likely built complex concurrency stuff on top of fragile code. Anyway, can't wait for multi-threaded coroutines 🎉

[qwwdfsad]

@benasher44 could you please elaborate what machinery does not work in the worker?

I would be grateful if you could create an issue for that as this problem is expected to be fixed in #826

[benasher44]

Will do. I’ll spend today and tomorrow recreating this and trying to file issues. Thanks!

[benasher44]

Actually it works fine. I think yesterday I put myself in some weird scenario using multiple threads and Dispatchers.Unconfined. I started fresh today, and something like this works in a test:

// helper entrypoint into a worker context coroutine
fun Worker.launch(block: suspend CoroutineScope.() -> Unit) {
    block.freeze()
    execute(TransferMode.SAFE, { block }) {
        println("executing")
        runBlocking(block = it)
    }
}

class NativeCoroutineTest {

    @Test fun `bg coroutine test`() {
        val worker = Worker.start()
        worker.launch {
            println("started")
            launch {
                println("launch 1")
            }
            println("test delay")
            delay(10)
            println("test delay done")
        }
        runBlocking {
            // hang the test to give worker room to run
            delay(10000000000)
        }
    }
}

[benasher44]

The main issue we run into on native in this scenario is if we go out the the client app to have it make a network request and suspend the worker (or any) coroutine in the process, there doesn't seem to be a way to resume with the result of that network request back into the worker thread. I think the ability to do that falls into the feature "Support multi-threaded coroutines on Kotlin/Native," which brings me back here 😅. Do I understand that correctly? If there were a way to do exactly that in the short term (~i think this means making the event loop thread safe~), then that would be super helpful. The answer here in JVM I think would be to use withContext(otherThreadsContext) to then resume in the original context, but there doesn't appear to be a way to reference another thread's context from another thread.

[elizarov]

The problem is not withContext(otherThread) { ... } per se. The problem is that in this way you start sharing data between this and other thread and the only solution K/N offers now is to freeze the data you share. It works for some use-cases but does not cover the cases where you need to transfer data that is mutable.

[benasher44]

Right the mutable data part is understandably tricky, and I look forward to seeing that hashed out. In my particular scenario, I’m making a network request and suspending a coroutine running on a worker thread. The network request returns to the call site on a different thread. Getting from the network thread back to the coroutine thread turned out to be really difficult because a continuation cannot be frozen (gets mutated on resume) nor detached (referenced somewhere by internal coroutine machinery). I managed to get to a solution that uses a @ThreadLocal variable to hold onto the continuation, which I can then resume after freezing the network data and transferring it back to the coroutine thread. I’ll post the somewhat generalized solution today or Monday. This could be made a lot smoother if continuations were atomic (I.e. allow passing them around threads, even if it’s only legal to resume them on the original thread). Is that something worth ticketing, which would be done before the full solution for this issue?

[elizarov]

Making continuations atomic is hard for the same reason, as you resume continuation with a value and that value has to be somehow transferred to another thread.

[benasher44]

I agree, but in that case it's the responsibility of the client code to manage getting that value across threads, for which K/N already provides facilities (freeze, DetachedObjectGraph). The issue here is that I can manage safely transfering the object/value that I own outside of kotlinx.coroutines, but I have no recourse for passing the continuation across threads with the intent to eventually bring it back to its original thread to then resume it (i.e. basically want to be able to bring it across threads and promise that i'll do the right thing and only resume it on the original thread where it was suspended) because they (continutations) can't be frozen or detached.