[WASI] experiment - tracking

[SteveSandersonMS]

Description

This issue is to track known issues in the early WASI-enabled runtime builds. These need to be resolved in order to have a proper supportable WASI-ready release.

• [x] Add wasi-wasm RID
  • https://github.com/dotnet/runtime/issues/77780
  • https://github.com/dotnet/runtime/issues/78389 API change
• [x] Build System.Native using WASI SDK instead of using the Emscripten-built binary.
  • MSbuild files for it and generate wasm_m2n_invoke.g.h
  • Enable p/invokes to all APIs exposed by System.Native. Currently there's a small hardcoded list.
  • https://github.com/dotnet/runtime/pull/79046
• [x] Create nuget package for WASI-SDK (dotnet/wasi-sdk)
  • [ ] #95086
• [x] Add WASI runtime as XHarness target
  • https://github.com/dotnet/xharness/pull/977
• [x] Enable GC finalizers. Currently the finalizer thread isn't started (see mono_gc_finalize_notify in gc.c).
  • fix stack range https://github.com/dotnet/runtime/pull/80456
• [ ] Support .NET threads on threading-enabled wasm runtimes. This includes updating the synthetic-pthread.c code to support having separate pthread keys for each thread instead of a single global list.
• [x] #95137
• [x] create ICU transport package and use it
• https://github.com/dotnet/icu/pull/307
• https://github.com/dotnet/icu/pull/309
• https://github.com/dotnet/runtime/pull/81345
• [ ] make it work on wasmedge https://github.com/dotnet/dotnet-wasi-sdk/issues/5 https://github.com/dotnet/runtime/issues/85429
• [ ] make it work on wasmer https://github.com/dotnet/dotnet-wasi-sdk/issues/6
• [x] #86984
• [x] #86985
• [x] do we need vitual file system ? https://github.com/dotnet/runtime/issues/81418
• [ ] https://github.com/dotnet/runtime/issues/94803
• [ ] Mono AOT Support
  • [x] https://github.com/dotnet/runtime/pull/95051
  • [x] https://github.com/dotnet/runtime/pull/95074
  • [ ] #95136
• [x] https://github.com/dotnet/runtime/pull/106977
• [x] https://github.com/dotnet/runtime/pull/107351
• [x] https://github.com/dotnet/runtime/pull/103752
• [ ] WASI TLS https://github.com/dotnet/runtime/issues/109569
• [ ] WASI HTTP/2, gRPC see https://github.com/WebAssembly/wasi-http/issues/128
• [ ] WASI preview 3 async see https://github.com/orgs/bytecodealliance/projects/16
• [ ] WASI Crypto https://github.com/WebAssembly/wasi-crypto

Build

• [x] Have a real CI build (including some test) ~and a public distribution mechanism.~
  • [x] https://github.com/dotnet/runtime/issues/72641
  • [x] https://github.com/dotnet/runtime/pull/81052
• [x] Produce binaries into experimental workload , in similar way as we do for multi-threading WASM (https://github.com/dotnet/runtime/pull/81849)
• [x] wasi templates (https://github.com/dotnet/runtime/pull/81849)
• [x] WasmAppHost support (https://github.com/dotnet/runtime/pull/81849)
• [x] wasi.build.tests (https://github.com/dotnet/runtime/pull/81849)
• [x] App builder, merged targets etc (https://github.com/dotnet/runtime/pull/81849)

[kg]

My personal position as one of the original creators of WebAssembly is that "skating towards where the puck is going" has always been necessary to deliver a good developer experience, and it will continue to be necessary. It's not always possible to predict where the puck is going to end up, but following the standards process is decent way to go about it.

very few runtime maintainers are interested in supporting this moving target ... Note the actual downloads are WASI Preview 1, with barely any activity for the Preview 2 versions.

By the same logic used here to argue for staying on p1 forever (look at this linear scale graph showing usage of a widely deployed piece of technology vs an emerging technology), it would i.e. be appropriate to only support a single dominant web browser and ignore every other browser, based on current usage statistics. At one point, the proven technology was asm.js - widely supported and widely used - while the emerging technology was WASM. In the same way, one could argue that WASI itself shouldn't be supported, since WASI usage is tiny in comparison to usage of client web browsers. Surely that's not the goal?

Given the amount of time it takes for core runtime changes to percolate through the ecosystem into developers' hands, and then the time it takes for developers to adopt a new core runtime version and deploy it in their applications, it's not sufficient to ask "what does WASM look like today? what does WASI look like today?" because by the time your answer(s) end up on an end user's computer or on a server in the cloud, years have passed. It's not sufficient to just say "this is the wrong answer right now" and use that to close off any forward-looking development.

Some key questions are:

• What do end users want?
• What will end users' devices look like by the time this software ships?
• What changes need to be made in order to deliver what end users want on the devices and runtimes they'll have by the time this software ships?

Just off the top of my head, I don't see a good solution for HTTP in WASI p1, so I feel those three questions are illuminating in this context.

My personal advice in this context would be to consider stating what the user needs are and how those needs are best served by staying still instead of moving forward, or alternately, suggest which direction the forward movement should be aimed in and how that will serve users.

[yoshuawuyts]

@nilslice Hi there, Rust Project member here. I'd be careful not to read too much into the download data of one specific Rust package. The WASI 0.1 (nee wasm32-wasi) target has been a tier 2 stable target [^tier-2] for five years. The WASI 0.2 target has been on tier 2 for negative three weeks: it is going to be released next month as part of Rust 1.82 (October 17th, 2024).

[^tier-2]: A target in Rust being "tier 2" means it is guaranteed to build and pre-built compiler toolchains are available to be installed via the rustup tool. This is the support tier required to practically support running a toolchain in CI.

This is relevant because in order for crates to support both WASI 0.1 and also introduce support for WASI 0.2, a compiler target cfg is required. This allows crate authors to conditionally compile for different targets, and looks something like this:

 #[cfg(all(target_os = "wasi", target_env = "p1"))]
 mod wasip1 { /* WASI 0.1 items go here */ }

 #[cfg(all(target_os = "wasi", target_env = "p2"))]
 mod wasip2 { /* WASI 0.2 items go here */ }

In order for crates to test whether they work on CI they require a working WASI 0.2 toolchain, which in turn requires having a prebuilt compiler toolchain available. While it was technically possible to build your own WASI 0.2 toolchain for Rust before Rust 1.82, we've recommended against people doing that. That has meant that targeting WASI 0.2 from existing libraries essentially hasn't been possible. This will change with the upcoming Rust 1.82 release, at which point we will likely see a gradual roll-out of support for WASI 0.2 throughout the ecosystem.

I hope that provides some insight into the evolution of Rust's WASI targets. If you'd like to read more, please see the post: Changes to Rust's WASI targets, as well as the draft for the upcoming release: The wasm32-wasip2 Target has Reached Tier 2 Support.

[mattjohnsonpint]

We use Wazero as our WASM host runtime. I was planning to offer C# as a target guest language, but I guess since Wazero has voted no to WASI-p2 and .NET has voted no to WASI-p1, that sadly we'll just have to tell our customers to pick a different language.

See

• https://github.com/tetratelabs/wazero/issues/2200
• https://github.com/tetratelabs/wazero/issues/2289

[nilslice]

@yoshuawuyts, hi - I don't think the point you're making really lands here or is related to the .NET support concern, but thanks!

I'd be careful not to read too much into the download data of one specific Rust package.

I have been reading and following these closely for a few years, it's always a good proxy to measure interest. Considering it's my job to know what users want from Wasm, I read a lot of crates.io and other package manager data.

The WASI 0.2 target has been on tier 2 for negative three weeks: it is going to be released next month as part of Rust 1.82 (October 17th, 2024).

Wasip2 launched in January of this year (https://github.com/WebAssembly/WASI/pull/577), so I'd put the starting line for adoption metrics there. It's had nearly zero uptick, except from a few insiders within the Bytecode Alliance. This is despite countless strong pushes, conferences, and other evangelism. It's just not sticking.

I deeply care about WebAssembly, and have its best interest in mind, and I do not see the ecosystem having voted with their usage of Wasip2. Now, maybe if it really is that the Rust target alone is the winning solution, then that will change. But it still does NOT address anything to do with this thread, which is squarely about .NET tooling compiling to core standard Wasm Modules vs. non-standard unadopted Wasm components.

This is where the problem lays, and I worry will further fragment the ecosystem and at the same time strip .NET languages from their utility to Wasm projects.

If you'd like to read more, please see the post: Changes to Rust's WASI targets, as well as the draft for the upcoming release: The wasm32-wasip2 Target has Reached Tier 2 Support.

I have read this, and saw the problematic release and reversal of this announcement. It's not the most reassuring state of things to now think about depending on this target in any way.

[yoshuawuyts]

@yoshuawuyts, hi - I don't think the point you're making really lands here or is related to the .NET support concern, but thanks!

@nilslice ? - I was directly responding to your comment in https://github.com/dotnet/runtime/issues/65895#issuecomment-2373112531, where you claim that the downloads of a single Rust ecosystem crate are indicative of broader WASI 0.2 ecosystem adoption. You use that to further the argument that the dotnet maintainers should shy away from supporting WASI 0.2. I didn't make the connection between Rust and dotnet, you did.

Posting just the screenshot on its own can be misleading as it omits context. If you're going to discuss download counts, it seems relevant to explain that library authors can't practically add a dep on wasi@0.13 until Rust 1.82 which won't be released for another three weeks. Whereas the older version of the library has been around for five years. It should be obvious that download counts will be skewed and are not indicative of any broader trends or interests.

[nilslice]

I'm going to try and steer this thread back to reality, and to the core concern of neutering .NET's utility to the greater Wasm ecosystem.

@kg, much respect for your work on Wasm and continued interest, but:

(look at this linear scale graph showing usage of a widely deployed piece of technology vs an emerging technology)

This is precisely a reason not to drop targeting standardized core modules... it's widely deployed piece of technology that .NET will now lost traction with. I don't understand why this is the way you would approach the rollout of an emerging technology that is unproven, undesired, and unnecessary to a significant portion of the Wasm ecosystem.

By all means, go ahead and keep experimenting, but it would appear that the only reason you'd do this is to force components and wasip2 on unsuspecting users. It feels very hostile that if you can't lure users with quality, you force it this way.

Given the amount of time it takes for core runtime changes to percolate through the ecosystem into developers' hands, and then the time it takes for developers to adopt a new core runtime version and deploy it in their applications

Again, that amount of time equals the amount of time the latest versions of .NET become effectively useless to the greater Wasm ecosystem. It seems rather self-destructive to go this route, no?

Just off the top of my head, I don't see a good solution for HTTP in WASI p1, so I feel those three questions are illuminating in this context.

I think this is untrue, unless you'd care to specify that you mean "a good solution for running a virtual HTTP server in WASI p1"?

So:

Some key questions are: What do end users want?

First, users want their code to be useful, and usefulness starts with being able to run. This should make it obvious that supporting standard core modules which run everywhere is preferable to users, vs. running a non-standard component, which runs basically nowhere.

What will end users' devices look like by the time this software ships?

Largely like they do today, and the abstraction already provided by Wasm removes the need to worry. Perhaps I don't follow what this has to do with the overall concern though.

What changes need to be made in order to deliver what end users want on the devices and runtimes they'll have by the time this software ships?

None. However, I'm not saying that we stop trying to innovate and improve. I'm not saying the .NET team shouldn't support components. I'm just saying don't severely limit .NET's reach and utility to the vast existing ecosystem of Wasm users and runtimes that have no need or interest in components.

[bhelx]

As a user of dotnet and the wasip1 target. I'd ask for a reconsideration on components as well, as we won't be able to use this if it requires component model. A lot of the debate in this thread seems to be about wasp1 vs wasip2. I don't think people have a problem with the system interface evolving. It's actually a good thing and is very welcome. The concern is that, as I currently understand it, wasip2 requires the component model which locks it out of most of the runtimes we use. I seem to recall some wasi maintainers talk about not needing to couple both wasip2 and component model, but i'm not sure the status of that. But I think if you could just make the component model piece optional, then it would satisfy all parties.

[syrusakbary]

I've read the thread and spend a bit of time to reflect before adding my comments. I appreciate the time that has been put into this discussion.

I think for those who want it or need it, WASIp2 might be great. However, based on the the original post, it seems support for multiple runtimes was one of the original goals behind creating the issue:

• [ ] make it work on wasmedge https://github.com/dotnet/dotnet-wasi-sdk/issues/5 https://github.com/dotnet/runtime/issues/85429
• [ ] make it work on wasmer https://github.com/dotnet/dotnet-wasi-sdk/issues/6

WASIp2 is supported in only one Wasm runtime, while WASIp1 is supported in almost the totality of server-side Wasm runtimes. I'm afraid that the decision of not supporting WASIp1 might decrease the overall usage of .NET in the broader WebAssembly ecosystem, which is also hinted by other comments on this thread.

One of the concerns showcased is that only one target can be supported: either WASIp1 or WASIp2... which is turning the discussion into the wrong angle in my view. Please note that even the Bytecode Alliance (who are the champions behind WASIp2) have promised to keep updating and supporting WASIp1, specially because of industry needs.

As an example, if supporting the WASI target in Rust it would have meant not supporting the Emscripten target... that would turned the conversation on the wrong turn. Thankfully, the maintainers by then decided that WASIp1 was an additive change, and not predatory one against other target and thus... the community was happy, because they could choose the target that would better fit their needs. I believe we are in a similar situation here.

Personally, I think the best outcome would be to support both targets: WASIp1 and WASIp2. After doing a bit of research, I don't see a good reason why the two targets can't be supported, specially as the changes done in the PR to support WASIp2 (or p1) on the codebase are rather trivial: https://github.com/dotnet/runtime/pull/104683

We don't have dev bandwidth to support multiple build configurations, I'm sorry.

Of course, I don't think is reasonable to push the maintenance cost into someone else. Given the concerns presented, I think is only fair to put that maintenance cost on the interested parties rather than someone else.

Given that, I'm proposing the following:

• I will champion someone from the Wasmer team to support WASIp1 back into C#

We will eat the burden and bandwidth for supporting an extra build configuration so the Wasm community is able to use .NET in runtimes where WASIp2 is not supported, so .NET supports also WASIp1. For those that wants to use WASIp1 in most of the runtimes, they could. For those that want to use WASIp2 with the new binary format... they also could.

Will that work for you @pavelsavara @kg? Thanks a lot for your great work!

[RReverser]

@RReverser Did you ever get this workaround working? I'm in the same boat where I'm trying to code to an existing host that only supports modules, not components. .NET 9 preview 6 was working for me, but for some reason I couldn't get it to include native assets.

@enghch Yes, it's pretty ugly (as expected), but the core of it is

<Target Name="RemoveWasmComponents" BeforeTargets="_WasmWriteRspForLinking">
  <ItemGroup>
    <!-- hardcoded path because only pure paths can be excluded by wildcard in `Remove=...` while we want to remove a custom concatenated string -->
    <_WasmLinkStepArgs Remove="-Wl,--component-type,&quot;$([MSBuild]::NormalizePath('$(MicrosoftNetCoreAppRuntimePackRidNativeDir)', 'WasiHttpWorld_component_type.wit').Replace('\','/'))&quot;"/>
    <_WasmLinkStepArgs Include="-fuse-ld=lld" />
  </ItemGroup>
</Target>

This disables components from being linked in and produces raw Wasm MVP library - although you still need to deal with unknown imports depending on what you are doing.

It's far from ideal (and I don't have much to add to the discussion about the ideal state of things, aside from what has already been said above), but at least it hacks around the immediate problem.

[RReverser]

Semi-related: looks like Wasm imports are currently broken when used with LibraryImport for automatic marshalling. Raised here https://github.com/dotnet/runtime/issues/109181 (UPD: now fixed, thanks @jkoritzinsky).

[RReverser]

For others trying to build Wasm libraries, https://github.com/dotnet/runtime/issues/101434 is another issue that can be confusing to debug until you realise what's going on (it prevents UnmanagedCallersOnly exports from being preserved under the default compilation options).

Luckily, the workarounds are relatively simple - either use <TrimMode>partial or manually disable trimming for your exports via either TrimmerRootDescriptor/TrimmerRootAssembly.

Other than these two issues + a need for a proper component model opt-out flag (I'll still try to get around to making that PR for .NET 10), it's possible to make everything work at least as well as in .NET 8.

[RReverser]

One more pretty obscure issue: exports with 9+ arguments compile but crash in runtime. https://github.com/dotnet/runtime/issues/109338

[RReverser]

Also running into https://github.com/dotnet/runtime/issues/101276 when trying to use AOT (the main motivation for upgrading to .NET 9). Looks like there are more issues for now than I had hoped, so maybe we'll postpone the upgrade.

[mhmd-azeez]

@RReverser any conclusion regarding .NET 9?

[RReverser]

@RReverser any conclusion regarding .NET 9?

Well, I did get it to work, but there are a bit too many hacks and workarounds for the bugs mentioned above. At least a bit faster than .NET 8 thanks to AOT (around 20% in benchmarks that need to do a lot of work in C# itself rather than in the host imports).

[mhmd-azeez]

@RReverser ah I see, are those hacks open source? also, did it impact binary size?