Multi-threading on Native

[armanbilge]

Scala Native 0.5 will support multithreading 🎉 in fact it has already been implemented in a series of PRs starting from https://github.com/scala-native/scala-native/pull/3114 and there are reports of early success in https://github.com/scala-native/scala-native/pull/3286#issuecomment-1561442170.

This is a super-issue to track and discuss efforts to support multi-threading on Cats Effect Native. The primary goal is to move the WorkStealingThreadPool from jvm-only to jvm-native shared sources. If we get this right, I believe the rest of the Typelevel ecosystem should Just Work:tm: on multi-threaded native without requiring any source changes.

Questions

1. Should we continue to have dedicated support for single-threading? By "dedicated support" I mean retaining the event-loop execution context and using link-time switches to fallback to efficient, non-thread safe data structures.

   On the one hand, a true single-threaded runtime is a myth. Blocking threads are always going to be needed for file I/O and typically DNS as well.

   On the other hand, APIs such as libuv (and to a lesser extent io_uring) can take responsibility for the blocking thread pool, so that userland can be single-threaded. It's quite possible that this will have higher performance than managing our own blocking threads, but considering our own optimizations e.g. to avoid thread-shifting blocking tasks, I'm not sure if this is so cleancut.

   So at this stage I'm not sure if it is worth the added complexity. Without a libuv polling system there's no viable single-threaded Cats Effect application without a blocking pool.

   Afterthought: a single-threaded runtime may be required to support WASM 🤔

Tasks

• [ ] Implement JDK concurrent data structures used in Cats Effect. @mox692 is working on this 🙏 https://github.com/scala-native/scala-native/issues/3165

• [ ] Move WSTP into shared jvm-native sources. Some portions of it are implemented in Java so these will have to be ported. For example some padding tricks are used in the LocalQueue to avoid "false-sharing". We need to figure out how to achieve the same on Native.

• [ ] Upgrade various APIs to use concurrency primitives. For example Ref, which is currently using an ordinary mutable field. We also need to fixup the implementations of IO.blocking 😅

• [ ] Add support for multi-threading to the polling systems. Notably, interruption and thread-safe cancelation. https://github.com/typelevel/cats-effect/pull/3332

• [ ] Audit test suite for disabled or otherwise special-cased tests that now should be run on Native.

• [ ] Make sure that we've deduplicated as much as possible/relevant between JVM and Native.

[armanbilge]

As of https://github.com/scala-native/scala-native/pull/3388 it looks like Scala Native will be publishing nightly snapshots of the 0.5.x dev branch with multi-threading support. This makes it feasible to start publishing some bootleg builds of Cats etc. so that we can start experimenting in Cats Effect.

Even while efforts to port concurrent JDKs are in-progress, we could try running IO on the fork-join pool already available in Scala Native. This would still exercise a lot of machinery (e.g. the fiber runloop) until we can get the WSTP cross-compiled.

[durban]

This makes it feasible to start publishing some bootleg builds of Cats etc.

To have Cats on 0.5.0-SNAPSHOT, we'd first need a release of munit on 0.5.0-SNAPSHOT, right? Or is there a way around that?

[armanbilge]

By "bootleg" I meant unofficial releases. That include Cats and all of its dependencies.

(In the initial Native effort, I published lots of bootlegs under my own group id, see here.)

[imjulioc]

It's now official 🙏🏽