tanishiking
commented
3 months ago WASI exchange data via Wasm linear memory
First of all, WASI (both preview 1 and preview 2) rexchange data between the Wasm module and the host environment via Wasm linear memory. For example, the fd_write function requires us to place some data on linear memory and pass the addresses and length as arguments. That means, we would need to allocate memory on WebAssembly linear memory, and put/get data on those allocated memory regions.
Direct ByteBuffer to allocate memory?
We've talked a bit about using (Direct)ByteBuffer to allocate off-heap memory on WebAssembly linear memory. Emulating a Direct ByteBuffer in pure WebAssembly would involve using WebAssembly's linear memory, which seems reasonable.
However, it relies on a JVM's garbage collector to handle memory deallocation, and I don't think there's a way to hook into when objects are garbage collected by the WasmGC and free memory in a similar way.
We would need to find a workaround to support Direct ByteBuffer in pure WebAssembly at some point if we want to support it, but at this point, Direct Bytebuffer seems too much for supporting WASI 🤔
Our own Memory Allocator (for wasm linear memory)
Instead of supporting Direct ByteBuffer for linear memory allocation, I'm thinking about having a simple linear memory allocator (we anyway need an allocator for Direct ByteBuffer though), something like:
withScopedMemoryAllocator { allocator =>
val ptr = allocator.alloc(byteSize)
ptr.writeBytes(bytes)
...
} // when we exit the scope, the allocated memory will be "free"ed.We can start with a simple memory allocation algorithm considering the those allocated memory should be short-lived only for exchanging data between wasm module and host environment (in WASI context).
tarsa
sjrd
Why
Currently, the Wasm modules generated by
scala-wasmdepend on helper functions in JavaScript, making it possible to run them on JavaScript engines like V8, but not on standalone Wasm runtimes such aswasmtimeorwasmedge.While WebAssembly was initially designed to run in web browsers, it has recently been adopted in various environments due to its portability, fast loading and execution times, and security benefits.
For example, WebAssembly is being used in:
(I personally believe the significant benefit of compiling Scala to WebAssembly lies in the ability to run it in these environments (as well as the Component Model))
However, to leverage WebAssembly in these environments, the binaries need to be executable on standalone Wasm runtimes, which is not possible with the current state of the generated binaries.
How
To achieve this, two major tasks are required (and the latter will likely be a significant challenge):
WASI (WebAssembly System Interface)(at least WASI preview 1 for now). This would enable access to the file system and other resources, regardless of the host environment.Kotlin/WasmRegExp engine reuses the implementation from Kotlin/Native.)js.Dynamic, would become unavailable (some of them might be replaceable with WASI?)State of WasmGC support on standalone Wasm runtimes