Roadmap: Threading Support

[kateinoigakukun]

Recently a lot of work on thread support has been done in WASI: https://github.com/WebAssembly/wasi-threads After our toolchain support threading, it will unlock offloading computationally heavy operations to threads.

Here is a list of tasks to support threads in SwiftWasm:

• [x] 1. Enable wasm-ld's --gc-sections to unlock wasi-libc upgrade https://github.com/swiftwasm/swift/issues/5128
• [x] 2. https://github.com/swiftwasm/swift/issues/5547
• [x] 3. Update wasi-libc to the version, which includes pthread implementation based on wasi-threads
• [x] 4. Support wasm32-unknown-wasi-threads target triple
• [x] 5. Build stdlib with atomics enabled
• [x] 6. Add compiler options to enable atomics feature in LLVM code generator
• [x] 7. Setup Swift SDK distribution for the threaded SDK
• [x] 8. Fix unaligned or too large atomic load/store in HeapObject.cpp
  • https://github.com/apple/llvm-project/blob/d7f1b250afdb37a89d02731acb26912c1fff5625/clang/lib/CodeGen/CGAtomic.cpp#L837
  • This libcall requires libclang_rt.builtin.a to include atomic.c with atomics and bulk-memory features enabled, but it makes it difficult to share the archive between non-threaded target, so we should fix Swift runtime side.
  • https://github.com/swiftwasm/swiftwasm-build/pull/288 will fix this
• [x] 9. Add required options to toolset.json in Swift SDK
  • https://github.com/apple/swift-sdk-generator/pull/104
• [ ] 10. Implement a transformation pass for i32.atomic.wait -> spin lock to work around browser limitation on main thread
  • See https://github.com/WebAssembly/threads/issues/106 about details of this limitation
• [x] 11. JS polyfill implementation of wasi-threads based on Web Worker
  • https://github.com/kateinoigakukun/swiftwasm-threading-example/tree/main/Sources/JavaScript

Feedback and any help are welcome :)

[Kyle-Ye]

One more task IMO:

• [ ] Add generic_wasi_base.h and update the include logic here to support WASI https://github.com/apple/swift-corelibs-libdispatch/blob/5529d351e304eb2312b7b3bfb073f7d8285473e0/os/object.h#L24-L33

[kateinoigakukun]

Started building stdlib for wasm32-unknown-wasip1-threads target on main branch: https://github.com/apple/swift/pull/72650

[kkebo]

I have a question. Are there still blockers for compiling Swift for the wasm32-unknown-wasip1-threads target triple?

$ which swiftc
/home/kebo/downloads/swift-wasm-DEVELOPMENT-SNAPSHOT-2024-04-03-a/usr/bin/swiftc
$ swiftc --version
Swift version 6.0-dev (LLVM 3dd3bce1adb5b65, Swift 1f09be97a890c3e)
Target: aarch64-unknown-linux-gnu
$ cat hello.swift
print("hello")
$ swiftc -target wasm32-wasip1-threads -c hello.swift
<unknown>:0: warning: libc not found for 'wasm32-unknown-wasip1-threads'; C stdlib may be unavailable
<unknown>:0: error: could not find module '_Concurrency' for target 'wasm32-unknown-wasip1-threads'; found: wasm32-unknown-wasi, at: /home/kebo/downloads/swift-wasm-DEVELOPMENT-SNAPSHOT-2024-04-03-a/usr/lib/swift/wasi/_Concurrency.swiftmodule
$ uname -a
Linux Brown-rhinoceros-beetle 6.6.3-414.asahi.fc39.aarch64+16k #1 SMP PREEMPT_DYNAMIC Sun Mar 24 19:44:17 UTC 2024 aarch64 GNU/Linux

error: could not find module '_Concurrency' for target 'wasm32-unknown-wasip1-threads' also happens when using swift build with the Swift SDK.

[kateinoigakukun]

I still need some build script engineering 🙃

[kkebo]

It's OK for now, thank you.

[kateinoigakukun]

Tried to fix the issue around atomic.c in libclang_rt.builtin.a, but concluded that it's difficult to fix this with the current toolchain layout of clang, which requires the single builitin library for each arch.

```patch From e3ab0445dce6d0aabdf9719742240eaafc5731a4 Mon Sep 17 00:00:00 2001 From: Yuta Saito Date: Sun, 14 Apr 2024 16:48:59 +0000 Subject: [PATCH] [compiler-rt][wasm] Compile atomic.c with atomics and bulk-memory features The bulitin object file for atomic.c is only referenced by user code when the atomics and bulk-memory features are enabled and libcall is required. However, the atomic.c itself was compiled without those features and it leads to a linker error because all objects have to have the feature when `--shared-memory` is enabled. This patch compiles atomic.c with the atomics and bulk-memory features enabled only for the object file. --- compiler-rt/lib/builtins/CMakeLists.txt | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/compiler-rt/lib/builtins/CMakeLists.txt b/compiler-rt/lib/builtins/CMakeLists.txt index c139917d4f58..5af6c72a9102 100644 --- a/compiler-rt/lib/builtins/CMakeLists.txt +++ b/compiler-rt/lib/builtins/CMakeLists.txt @@ -839,11 +839,30 @@ else () set(deps_aarch64 lse_builtin_symlinks) endif() + set(BUILTIN_OBJECT_LIBS_${arch}) + # For WebAssembly, we need to compile the atomic.c with different flags than the rest + # to enable atomics and bulk-memory features only for the object file. + if((arch STREQUAL "wasm32" OR arch STREQUAL "wasm64") AND "atomic.c" IN_LIST ${arch}_SOURCES) + set(BUILTIN_ATOMIC_CFLAGS_${arch} ${BUILTIN_CFLAGS_${arch}}) + list(APPEND BUILTIN_ATOMIC_CFLAGS_${arch} -matomics -mbulk-memory) + add_compiler_rt_object_libraries(clang_rt.builtins.${arch}.atomic + ARCHS ${arch} + DEPS ${deps_${arch}} + SOURCES atomic.c + DEFS ${BUILTIN_DEFS} + CFLAGS ${BUILTIN_ATOMIC_CFLAGS_${arch}}) + # Include the atomic object file in the builtins archive + list(APPEND BUILTIN_OBJECT_LIBS_${arch} clang_rt.builtins.${arch}.atomic) + # Remove atomic.c from the main list of sources + list(REMOVE_ITEM ${arch}_SOURCES atomic.c) + endif() + add_compiler_rt_runtime(clang_rt.builtins STATIC ARCHS ${arch} DEPS ${deps_${arch}} SOURCES ${${arch}_SOURCES} + OBJECT_LIBS ${BUILTIN_OBJECT_LIBS_${arch}} DEFS ${BUILTIN_DEFS} CFLAGS ${BUILTIN_CFLAGS_${arch}} PARENT_TARGET builtins) -- 2.43.2 ```