Support multi-threaded coroutines on Kotlin/Native

[elizarov]

You can have multiple threads in Kotlin/Native. ~Each thread can have its own event loop with runBlocking and have number of coroutines running there~. Currently communication between those threads via coroutine primitives (like channels) is not supported. This issue it to track enhancement of Kotlin/Native in kotlinx.coroutines library so that all the following becomes possible:

• Launching coroutines from one thread with a dispatcher on another thread
• Await/join coroutine running on another thread
• Send/Receive elements to/from coroutines on other threads

UPDATE: Currently, coroutines are supported only on the main thread. You cannot have coroutines off the main thread due to the way the library is currently structured.

UPDATE 2: the separate library version that supports Kotlin/Native multithreading is released on a regular basis. For the details and limitations, please follow kotlin-native-sharing.md document. The latest version: 1.5.2-native-mt

[elizarov]

But the problem is that if Continuation is atomic, then you can only resume it with frozen objects even if your coroutines are in single-thread only. K/N memory model only allows you to store frozen objects in atomic references.

There is no way to make continuation "conditionally atomic". Alternatively, we can do massive copy-paste and have two parallel hierarchies of all coroutine classes -- atomic for cross-thread and regular for single-thread usage. This is a non-starter either.

[benasher44]

But the problem is that if Continuation is atomic, then you can only resume it with frozen objects even if your coroutines are in single-thread only. K/N memory model only allows you to store frozen objects in atomic references.

There are some common (imo) cases (the particular ones i'm dealing with 🙂) where this concession is okay:

• Going to the network, getting immutable data, and then transferring that back onto your coroutine worker thread to process/transform
• Transferring data that you're done mutating to a worker thread to write to a database

Alternatively, we can do massive copy-paste and have two parallel hierarchies of all coroutine classes -- atomic for cross-thread and regular for single-thread usage. This is a non-starter either.

I agree with you there. That sounds like a nightmare (wish I had more to offer/say around how this complexity could be managed, but I haven't spent enough time digging through continuation code to be able).

[vsct-jburet]

I don't understand the implications of the current state of kotlin-native. Is it doable to implement the following use case?

1) In the UI thread, pass a request object to a background thread

2) In the background thread, start a long polling task (like http call) and get a response object

3) Pass the response to the UI thread

In this scenario, it's perfectly ok to freeze the request and response objects.

Thank you

[evertsmits]

What is the status of this right now?

[IgorKey]

Guys, are there any updates?

[qwwdfsad]

No updates yet. We will post here as soon as there will be any.

[sellmair]

I think this commit might be extremely interesting/promising! ☺️

olonho: Very preliminary relaxed mode draft. https://github.com/JetBrains/kotlin-native/commit/81eb6b2be689591b4bd697464c910afdc0f2a456 (MemoryModel.RELAXED)

[SeekDaSky]

I have my doubts about this, K/N devs are really reluctant about this mode (and they are right to be), it certainly will need some kind of background GC to collect cyclic references and this will affect K/N performances quite a bit.

Plus, if you give devs an easy and known way (no control from the language about what you do with your mutable states), they WILL choose it over the standard "strict" memory model.

Unfortunately, if they decided to go this way, it's probably because they have no other options to have a proper kotlinx.coroutines implementation that is unified for all platforms.

also : https://discuss.kotlinlang.org/t/kotlin-native-1-3-50-relaxed-mode/13586/5

[sellmair]

I absolutely get your point and I also think the ideas that went into the new (strict) memory model are great. However, I think the multiplatform aspects of Kotlin just outweigh the benefits of the strict memory model! I think the solution with the flag is pretty elegant: K/N only project can still keep the strict mode, while multiplatform projects can use the relaxed mode (which just makes sense for this use case, considering the platforms you compile to: iOS/Swift, jvm/Java, ...)

[SeekDaSky]

I agree you can't have multiplatform without a relaxed memory model, I simply hope that pure K/N project will keep using the strict mode and not use relaxed. It is still very experimental at this point so wait and see.

[LanderlYoung]

@sellmair Agreed! Maybe like what java have done, with the strict memory model, we can still provide an unsafe mechanism, which is "unsafe" obviously but provides better performance and more flexibility. Programer is encouraged to use strict memory model, but still have the freedom to do unsafe stuff and of course, take their own responsibility.

This may be applied to Kotlin/Native too.

K/N provides Saner Concurrency model alongside traditional unsafe one. Common programmers use the Saner one, while advanced programmers may use the unsafe one to meet their performance requirement(s). As we have seen from other languages, the Saner model can be built on top up the unsafe one inside the standard library. And also, as for coroutines can be, of course, built on top of the unsafe APIs.

[SeekDaSky]

@LanderlYoung I don't think the unsafe model memory will be more performant, in the contrary it needs a fully functional GC able to detect cylic references in a multi-threaded program, which is not an easy task (the JVM have been struggling with their GC's for years). The strict model provides a way to manage memory without this because references explicitly can not be in two threads at a time, making it easy to collect (cf. Rust that doesn't even have a GC and still manage memory automatically, kinda).

[sellmair]

I think performance is not even a big point here. It's more about the overall big picture (multiplatform). Also, obviously, there are scenarios where you could find each memory model to outperform the other (GC might be easier/more performant on strict mode, multi-threaded matrix multiplications might be easier/more performant on relaxed mode). But let's keep in mind, that Kotlin identifies as an application language. I am looking forward to the next steps. For our team, multiplatform seems extremely promising!

[kpgalligan]

"Common programmers use the Saner one, while advanced programmers may use the unsafe one" I think you'd find the opposite would happen in practice, as the advanced programmers are generally the ones who would know why you want to use the "Saner" one.

"I think the solution with the flag is pretty elegant: K/N only project can still keep the strict mode" We'll see how it goes, but I think if there's a "relaxed mode" you'll see "strict mode" go away. Pretty much all apps, K/N-only or multiplatform, will need libraries, and relaxed and strict libraries won't really work together (easily). We'll (probably) need a way to fail builds if you're trying to use a relaxed library in strict mode. That kind of thing. Formally adding relaxed mode will be a big shift. It wouldn't be the first time I was wrong in life, but that's my guess. It just seems like maintaining both modes, across many platforms, while also optimizing the builds and runtime is going to be a lot of overhead.

Relaxed mode will definitely impact performance for the foreseeable future. "How much" is a big question, of course, but it will. Memory management is a big part of the performance in KN apps, and there are thread confined assumptions you can no longer make.

Not a vote for or against relaxed mode (and there isn't an election anyway). Just saying I'd bet on strict mode going away if you need to compile with relaxed mode for kotlinx.coroutines (and other libraries).

[LouisCAD]

I've just listened to Andrey Breslav talk about multi-threaded coroutines and Kotlin/Native memory safety in Episode 119 of Android Developers Backstage.

It seems the biggest struggle is regarding iOS/macOS background queues that give no guarantee to which thread they will run on, while threads are kind of deprecated here.

So, here's a path to a solution I found while listening:

Allowing to wrap a dispatch queue as a CoroutineDispatcher (much like done for Android's Handler), which does the following:

• When the queue runs the callback, note the thread it's running on, before running the actual code, so access can be checked to only happen on this thread.
• When the actual code finished to run, remove the dispatch queue <-> thread association.

⮑Any access to involved mutable data in a timeframe where there's no thread association would lead to an Error or an Exception to be thrown. ⮑Any access to involved mutable data on a thread different from the one currently being run by the dispatch queue would also lead to an Error or an Exception to be thrown.

Each time the CoroutineInterceptor (usually a CoroutineDispatcher) is changed, transfer ownership of the involved mutable data in the same way you would for a native platform that doesn't have kind of deprecated threads.

Mutable data thread ownership could be controlled by new experimental/internal/restricted APIs in Kotlin/Native, which would still check you remove ownership before adding a new ownership, or swap it atomically. That would prevent people from turning these facilities into shared mutable data, and we would keep the mind relaxing "Shared XOR Mutable" paradigm.

JetBrains folks have probably thought about this, and I'm certain it's not as easy as it might seem, but it might interest people following this issue.

[elizarov]

@LouisCAD In fact, we are going to experiment with multiple solutions for iOS dispatcher. On one side, we'll provide an implementation of newSingleThreadContext that creates "deprecated threads" and just confines all the code there. On the other hand, we'll provide ability to convert an iOS sync queue to a coroutine dispatcher using "detached object graph", which allows to do what you've described -- to transfer a graph of objects to another thread.

[LouisCAD]

@elizarov I assume that when you say iOS, you mean iOS (+iPadOS) and macOS? macOS compatibility is very helpful to run tests without spawning a simulator.

About threads, I stumbled upon Apple documentation and found they are not deprecated (that word is not used anywhere). It's only that their usage is discouraged (by words) in favor of Grand Central Dispatch (aka. GCD) for four reasons that I'm quoting:

1. It reduces the memory penalty your application pays for storing thread stacks in the application’s memory space.
2. It eliminates the code needed to create and configure your threads.
3. It eliminates the code needed to manage and schedule work on threads.
4. It simplifies the code you have to write.

The first point certainly applies in coroutines case as I assume GCD keeps threads in the OS and only creates them as apps block them but more are needed (includes NSOperation when it's sitting on top of GCD). However, all the three remaining points don't apply for kotlinx.coroutines users.

Here are all the relevant docs I found for interested people:

• Threading Programming Guide
• Concurrency Programming Guide: Migrating Away from Threads
• NSThread

[benasher44]

The problem with using GCD and coroutines together is that GCD queues don't guarantee the threads they run on. They only guarantee synchronization with respect to the queues you run your blocks on, which doesn't mesh with the current thread-ownership model of K/N. IMO, NSOperation should be avoid in the context of coroutines. It's a higher level system for writing application-level worker tasks.

Furthermore, the thread efficiency gains really come from carefully constructing GCD queue hierarchies and keeping a fixed number of them in your application. See this WWDC talk from 2017.

When comparing systems, coroutines and GCD are roughly the same level of abstraction (with respect to threads), and I'd expect thread management to be more successful if that weren't delegated from coroutines to GCD and instead handled by coroutines itself.

[chris-hatton]

@elizarov Can I politely enquire yours/JetBrain's current position on this issue?

The interest it's received - both above and on kotlinlang Slack - shows its importance for many multi-platform users, while this thread has not been updated in 2 months and is now marked as postponed, with no assignee or any mention in any 1.4 road-map etc. that I can find.

Would be good to have some idea whether JetBrains still consider it a relevant issue as it may affect our adoption of K/MP for mobile.

Pro-active question: If I were to go ahead and implement a Dispatcher that rested on NSThread or underlying pthread API's (and not GCD), with the understanding that my application-level design would need to respect object freezing across threads; is there any other obvious blocker to doing so? i.e. Is it the coroutine internals themselves that would breach object freezing rules and cause this 'naive' approach to immediately fail?

[elizarov]

@chris-hatton It is still relevant and is still in our roadmap. The challenge is exactly in making all the coroutine internals compatible with freezing, that is making sure that objects like Job can be freely shared across threads.

[sellmair]

@elizarov So does that mean, that we can assume that the relaxed memory model is off the table now? I am still convinced, that the current native memory model is harming multiplatform massively.

[elizarov]

@sellmair Relaxed memory model is also work in progress.

[elizarov]

To give you some more detail on what's holding us back. kotlinx.coroutines is a multiplatform library. Currently all the "multithreading-capable" data structures are part of JVM compilation only while Native and JS contains simple stubs. In order to support multi-threading on Native we need to share this code between JVM and Native, while leaving stubs for JS. Unfortunately, our current MPP model in Kotlin (which is still experimental) does not support this. We are working on its next version code-named "Hierarchical MPP" (HMPP) which enable this kind of fine-grained sharing.

[ZakTaccardi]

Currently, coroutines are supported only on the main thread. You cannot have coroutines off the main thread due to the way the library is currently structured.

Is this true - I just thought all coroutines need to run on the same thread? If so, could we have all coroutines run on a separate, single background thread?

[Soup-Flies]

@ZakTaccardi

That is not the case, and has not been since the beginning of this library. The idea has been on the radar since inception but as @elizarov reminds us, the problem is to make multi-threaded work possible outside of the JVM where we are capable of having coroutines on multiple threads already. There are many capabilites for doing structured synchronous work through the current API, but there are limitations. Such as if you want to resolve values and elevate them out of an async scope, you have very few options for the Non-JVM api's such as callbacks. My team has chosen this as our current solution, and because suspend functions are erased for non-jvm compilation targets, we just overload the signatures with the same name expecting callback functions for all other targets. When we reach feature parity between the platforms it's very little client work to update our clients from callbacks to the structured synchronicity as well as the consistent api exposure.

I'd rather JetBrains take a cautious approach to solving this issue and give us strong architecture with consistent api's than release volatile tools right now.

[ScottPierce]

@elizarov Couldn't you guys just copy it over the data structures for now to Kotlin/native to unblock yourselves :P. Sounds pretty trivial to clean up later once HMPP comes out.

[elizarov]

@ScottPierce We've been through that experience when we first introduced MPP into kotlinx.coroutines at at time when there was no real MPP support in Kotlin. It becomes a nightmare to maintain.

[NinoScript]

Is the Hierarchical MPP expected to be released anytime soon? (this year? sometime in 2020?)

Also, for my use-case, I don't really need multithreading, I just need everything to not be on the main thread. Is it currently possible to use just one background thread for everything? Or do I need to wait for the HMPP? (or is this not at all related?)

[ScottPierce]

@NinoScript There is a PR multi-threaded coroutines for Kotlin/Native out already based on 1.3.60. I assume that update had what was needed. You can take a look at the draft PR here.

I was able to build it locally, release it to maven local, and try it out. With the automatic freezing of objects, it feels pretty good.

[elizarov]

I’ve published the first development preview version of multi-threaded kotlinx.coroutines version 1.3.2-native-mt-1 for Kotlin/Native (version 1.3.60) . Make sure to study the docs before using it: https://github.com/Kotlin/kotlinx.coroutines/blob/native-mt/kotlin-native-sharing.md

[thoutbeckers]

Great to see official support on the way.

Others in the thread might be interested in https://github.com/Autodesk/coroutineworker until that arrives, it's working pretty well for us (I'm not affiliated with Autodesk).

[Piasy]

Is it possible to use coroutine or worker to implement the message queue pattern?

"message queue pattern" means: an object, let's call it Counter, provides some operations, e.g. increase and decrease, these operations could be called from main thread or other worker threads, but Counter will schedule these calls into its own single threaded message queue, then change its state, e.g. the count, it may also output its state through some callback, and the callback should be invoked on main thread.

On Android, we could using Handler or Executor to implement this pattern easily, I'm thinking if it's possible to do it on Kotlin/Native. I want to develop a Kotlin multiplatform app and want to use this pattern in it.

Sorry if this question is off-topic, but I really can't find a better place to ask it.

[LouisCAD]

@Piasy You can ask on https://slack.kotl.in or here, but in a new issue.

[elizarov]

I've published version 1.3.3-native-mt to Bintray with the updated version of the branch for native concurrency on top of kotlinx.coroutines 1.3.3 and Kotlin 1.3.61.

[qiaoyuang]

@elizarov Hi, can I ask a question? When I used Mutex with coroutines(1.3.3-native-mt), I found the Mutex would make a coroutine suspend and not resume, but the similar code will be work that run with Kotlin/JVM. Is this a bug? The code is shown in following:

fun main() = runBlocking {
    val testData = TestData()
    val bareTestData = DetachedObjectGraph(TransferMode.UNSAFE) { testData }
    val mutex = Mutex()
    val job = launch(Dispatchers.Default) {
        val outTestData = bareTestData.attach()
        repeat(20000) {
            mutex.withLock { outTestData.index++ }
        }
    }
    repeat(20000) {
        mutex.withLock { testData.index++ }
    }
    job.join()
    println(testData.index)
}

data class TestData(var index: Int = 0)

The similar code run with Kotlin/JVM:

fun main() = runBlocking {
    val testData = TestData()
    val mutex = Mutex()
    val job = launch(Dispatchers.Default) {
        repeat(20000) {
            mutex.withLock { testData.index++ }
        }
    }
    repeat(20000) {
        mutex.withLock { testData.index++ }
    }
    job.join()
    println(testData.index)
}

data class TestData(var index: Int = 0)

[NorbertSandor]

Is this idea ever considered to be included in Kotlin? https://itnext.io/designing-a-kotlin-memory-safe-mode-c76c06317c3e It is a very good idea IMHO...

[SeekDaSky]

Is this idea ever considered to be included in Kotlin? https://itnext.io/designing-a-kotlin-memory-safe-mode-c76c06317c3e It is a very good idea IMHO...

I read that article a while ago and it feels like the way to go, I guess the author could post a KEEP and see what the Kotlin devs think about it

[kpgalligan]

You'd have to finish the design first. All types would need to be const or not const, and you couldn't cast between them, so that would mean ditching the entire standard lib and starting over, and probably giving up on JVM interop.

"The next question is about polymorphism. Can we extend const classes? Are const interfaces allowed? If so, are all of their implementations required to be const? These are very relevant questions, but irrelevant in the scope of this article."

I'd say they're critical to any proposal to change the language.

I've seen this post come up a few times recently, but it's unrealistic and doesn't address some of the serious issues that would need to be addressed. How well would Kotlin fare in the JVM world when a huge portion of the JVM runtime would be rendered incompatible? We could say JVM runtime is given special dispensation to break the rules, bu then what's the point?

That, of course, has nothing to do with MT coroutines. Discussing core language changes is probably best done elsewhere.

[ScottPierce]

@elizarov Is there a version of 1.3.3-native-mt that's compatible with 1.3.70?

[elizarov]

Is there a version of 1.3.3-native-mt that's compatible with 1.3.70?

Not yet. I'll post here when it becomes available.

[ScottPierce]

@elizarov I noticed this is marked postponed. It's been close to 4 months now. I'm surprised we haven't seen this released yet. Any update you can share about what's going on?

[LouisCAD]

@ScottPierce There's work-in-progress in #1648, you can read the documentation in this (currently draft) PR. There's also some publications done as possible (not done for 1.3.70 yet because of a new memory leak that has a workaround until it gets fixed for real in Kotlin/Native).

I think it's postponed until Kotlin/Native is fully ready to have it work well, without memory leaks, but it's already iterating.

[IgorKey]

Any updates(1.3.4-mt publication)? It has been blocking my 1.3.70 migration @LouisCAD , do u have link for Kotlin native issue? Saw some commits about memory leaks detector etc

[LouisCAD]

@IgorKey Here you go: https://youtrack.jetbrains.com/issue/KT-37232 I don't think you can expect another native-mt release before the related PR is merged and makes it to a new Kotlin release (probably 1.3.71).

[wongk]

Unfortunately because our IDEA plugin version and Gradle plugin version must be equivalent, if we use this library in any one of our projects, then none of them can be upgraded to 1.3.70.

[IgorKey]

k/N 1.3.71 has just come) https://github.com/JetBrains/kotlin-native/releases/tag/v1.3.71 Leak problem solved

[elizarov]

📣 kotlinx.coroutines version 1.3.5-native-mt is available for Kotlin 1.3.71.

[kar]

@elizarov could you point me to where we can find the new version? I can't find it on maven.

[LouisCAD]

@kar It is on jcenter (https://bintray.com/package/info/kotlin/kotlinx/kotlinx.coroutines)

[elizarov]

Thanks. I've just uploaded it to Maven Central, too.