"Just" an API vs requiring upstream features

[penzn]

I think #549 and WebAssembly/wasi-native-threads#48, present a qualitative change, however minor, from 'just an imported API' to where WASI would need some core features. I didn't think this should've prevented #549 from going forward, but I think it would be good to start a discussion on this.

In my view WASI is taking these dependencies on upstream standard, we can debate whether one is causing the other or they are independent:

• Component Model itself is a core proposal, at least far as APIs are concerned
• Certain features must be implemented in core wasm instead of APIs, as we have seen with threads

This is concerning, because so far there is not much traction in supporting component model or other core features mandated by the current WASI direction in browsers (treads draft hasn't been presented yet, for example). This would create divergence between WASI and 'stock' web environment, while more convergence would be preferrable in my opinion, as there are currently some challenges with running the same code both ways.

Originally posted by @penzn in https://github.com/WebAssembly/WASI/issues/549#issuecomment-1707400369

[penzn]

@syrusakbary you've said you have some concerns as well, not sure if related to this

[lukewagner]

@penzn I think we should be careful not to extrapolate too much from wasi-threads: like functions, tables, memories and stacks, threads are core part of the computational model since threads significantly impact the way you compile wasm (particularly the shadow stack) in a way that most APIs don't and can't (because it requires changing All The Toolchains). Thus, wasi-threads is a bit of an outlier in the WASI world.

Importantly, the shift from wasi-threads to the Core WebAssembly proposal is motivated by changing the core runtime representation (from dynamic-linking-hostile O(MxN) function-table-duplication to O(N) function-table-sharing). If we were happy with the function-table-duplication approach of wasi-threads, there would be no problem from a Component Model perspective; we'd just build wasi-threads directly into the Component Model, thereby allowing core wasm to import wasi-threads without even having to care about components-vs-core-modules (b/c it would be the same core function import either way). Thus, the choice to depend on an "upstream feature" instead of an "API" is a choice to avoid committing for all time to a fundamentally inefficient core representation and not related to Wit, the Component Model or any other WASI-level decision.

Thus, yes, we'll have to push on threads as an upstream core wasm feature, and it will take longer than Just Shipping A Thing, but the reward will be much greater browser convergence (to your point), a wasm toolchain that works more like all other native architectures (reducing the wasm-specific tooling burden), and a more efficient runtime (both inside and outside the browser).

[penzn]

@lukewagner I am on board with your perspective about threads and component model (as an aside, a different perspective has been floated as well), and also agree a better core representation for threads is desirable. However I am not sure if that really changes a lot of things here: regardless of why we would needed a core threading feature we would now need it to do any standard threading in WASI (setting aside a situation when some would just keep current wasi-threads API around despite it being taken out of preview 1). Any core threading changes would impact JS semantics - do you have an idea how likely those would be accepted?

[lukewagner]

@penzn AFAICS, the proposal to add the shared attribute to core wasm already has lots of browser interest; it's only the addition of the thread.spawn instruction that crosses the JS-engine/Web-Platform boundary which then pulls in a wider set of stakeholders, requires more careful Web specification work and thus raises some concern. But the good news is that, if we can't get browsers to agree to add thread.spawn to Core WebAssembly, as long as we have shared (so that we can have (ref shared (func ...)) types), we can define thread.spawn as a core host import (defined as a component built-in with a core function signature or simply made available by core-wasm-only hosts). Thus, the only thing we really "need" from core wasm is shared.

Btw, there's an agenda item on this topic in the up-coming in-person CG meeting where we can all discuss this question more directly.

[penzn]

WASI is taking a dependency on this now, while changing core semantics usually takes a rather long time. Just like you point out, the basics of the core threading proposal are only going to be discussed at the in-person CG meeting next month, then there is the proposal itself and its stages and W3C/TC39 approval in case it affects other aspects of the web. It is a pretty long process and in the meantime we would need to have standalone runtimes implement a core functionality that browsers don't implement. Same story for the Component Model, regardless of whether the threads situation is related to it or not.

Just to re-emphasize, the situation when browsers would take a few years to implement new threading and Component Model is not entirely unrealistic.

[lukewagner]

If you're not worried about running natively in a browser without a polyfill and not worried about shipping very-early-stage proposals (e.g., wasi-threads is not implemented natively in browsers and is also a very-early-stage proposal), then you don't need to worry about it for the new core threading proposal either: it can be added as an experimental feature to standalone runtimes and shipped according to the same criteria as wasi-threads. Furthermore, I believe that, as long as we rule out mutating shared function tables (which was also a limitation of wasi-threads), one can polyfill the core-threading proposal on the Web today by a not-terrible binary rewrite (e.g., of shared global accesses into something that emulates their semantics, such as shared multi-memory).