May WASI turn out to harm Wasm use cases on the Web?

[dcodeIO]

A few thoughts:

Mandatory polyfill on the web

As of today, WASI is the most standard-ish API we have to perform common tasks like obtaining the current time or printing to console. As a result, if modules want to run everywhere (without recompiling for different environments), these will likely opt to use WASI APIs to be general enough for portability needs, but doing so may eventually lead to lots of websites shipping a WASI polyfill, which is unfortunate in terms of code size on the Web (and efficiency, see below).

What would be a reasonable expectation in this regard? Could browsers eventually provide a subset of WASI out of the box, or have there been thoughts already to make WASI a bit more Web-friendly?

Supporting UTF-8 only

As of https://github.com/WebAssembly/WASI/issues/322, an UTF-8 validation API is planned, and mentioning "UTF-8 is a very popular string encoding, for example, it's the encoding used by over 95% of all Web content" (allow me to note: one could also say that 100% of the code running on the Web today doesn't use UTF-8) makes me assume that WASI will most likely support exclusively UTF-8, which would make it suboptimal when polyfilled as outlined above.

For example, we may end up with a double-re-encoding scenario again when say Java, or another language using WTF-16 internally, wants to print to console or otherwise call through to a Web API using such a polyfill. Say, in Java, which is not aware of the polyfill, convert WTF-16 to UTF-8, validate with a WASI API and call the respective desired WASI API. Then, in the WASI polyfill, re-encode UTF-8 back to WTF-16 and call a browser API. Note that the intermediate step from WTF-16 to UTF-8 and back is unnecessary there, and since this is an API-level type restriction in the middle of a producer and a consumer, it also seems unlikely that Interface Types will be able to help there.

Alternative: Recompile per target

As mentioned above, the obvious alternative is to recompile modules to run either using WASI off the web, or say direct imports on the Web. Doing so would solve the problems mentioned, but introduce unfortunate fragmentation I think.

In general, these problems look like more unfortunate side effects from designing Wasm for non-Web first, and Web second, as seems to be quite the trend nowadays, so I figured it may be good to talk about these before committing to something we may (or perhaps we may not) regret. Or am I missing something? Let me know of your thoughts :)

[devsnek]

It is sort of expected that browsers will adopt at least a subset of wasi. bluetooth, sockets, fs, etc. all have current or wip apis bring exposed in browsers for js, so it's not a big leap.

[dcodeIO]

Where can I read more about these APIs? Do you have any references on browsers considering to support WASI APIs?

[sbc100]

Regarding web-friendliness: Unless I'm mistaken, interacting seamlessly with the web was never a primary goal of WASI. This design principle talks about this but perhaps could be made more clear? : https://github.com/WebAssembly/WASI/blob/main/docs/DesignPrinciples.md#relationship-to-the-web

Aside from strings, another example of where this is obvious is that use of i64 in the API itself this made it impossible to directly implement the API on the web which until recently didn't have support for passing i64 to/from JavaScript (still doesn't on some browsers).

[devsnek]

@dcodeIO it's more of a thing that has been discussed informally. like @sbc100 says wasi is happy to use functionality that isn't web friendly, but we are aware and respectful of the design space (many of us work on web standards too)

[dcodeIO]

I am aware that WASI is more like a libc layer underneath currently, and that of course makes a lot of sense to enable things. My question is rather: Is it wise to design it this way exclusively, given my concerns above, or would it be good to aim for more eventually? I also certainly appreciate that there is informal discussion already. Let me know once there is something concrete to share :)

[kripken]

It is sort of expected that browsers will adopt at least a subset of wasi.

I am not aware of such an expectation on the browser side. It's possible I'm not aware of everything, though.

bluetooth, sockets, fs, etc. all have current or wip apis bring exposed in browsers for js, so it's not a big leap.

Personally I would guess that overlapping existing Web APIs would be an argument against adding WASI APIs to the Web. (Browsers can avoid adding any surface area if users wrap around the existing Web APIs in userspace, as has already been done by various libraries.)

But, yes, it's possible that a future large ecosystem of WASI files could motivate browsers to add built-in support for them. Note that the opposite is possible as well: with Interface Types it will be possible for Wasm to use Web APIs without JS, and Wasm VMs may be motivated to support those Wasm files off the Web by supporting some Web APIs. It's hard to predict how the ecosystems will grow, and there is no standards-based argument against either direction (as both Web APIs and WASI APIs are standardized) so it's hard to make a guess here.

[sunfishcode]

wasi-filesystem as we currently know it is probably about as bad a fit for the Web as an API can be :wink: . It's synchronous, it hard-codes a filesystem data model which doesn't map very cleanly to existing Web storage subsystems, it hard-codes UTF-8 for some things, and it has a lot of historical baggage and portability concerns that would be unfortunate to expose to the Web.

That said, there are several different aspects to this topic.

One is that while wasi-filesystem looms large in terms of what people see when they look at WASI today, it doesn't represent our entire vision for WASI. In the big picture, capability-oriented security is about passing handles around instead of strings, and the more WASI grows to do that, the less important strings, passing strings, and string encodings become (at least for WASI itself). This helps with many problems, including portability problems (important on the Web!), virtualization problems, and of course performance problems when passing strings between languages with different string representations.

Another is that in recent WASI meetings, we've started discussing the foundations of a new I/O framework which will abstract away many of the low-level details of filesystems (while still working in contexts that do). I expect this is going to significantly change how we think about what kinds of environments it might make sense to embed WASI in, including the Web.

wasi-filesystem will continue to be important. There are important use cases that specifically want that kind of low-level access to a host filesystem. But when we talk about embedding WASI in new environments, we'll likely talk about subsets rather than the whole thing. Not everyone environment will have a need for wasi-nn, for example, and the same will be true of wasi-filesystem.

Another aspect is streams. WASI streams such as stdout/stdin are currently uninterpreted byte streams (in practice, likely most WASI programs that read and write text through streams today are using UTF-8, but that's not an explicit rule). I expect we'll eventually want to be more explicit about the encodings of text streams, and currently the obvious way to do this in the WebAssembly platform is interface types, which I expect will make it possible to have some form of "stream of char". Interface types is independent of WASI, so discussion of the strengths and weaknesses of interface types should take place in interface-types or WebAssembly-CG-level forums. It is off-topic here.

[trusktr]

Interface types is independent of WASI, so discussion of the strengths and weaknesses of interface types should take place in interface-types or WebAssembly-CG-level forums. It is off-topic here.

If interface types are an alternative to WASI-on-the-web, then it is valuable to keep it in the discussion because it can influence WASI-on-the-web. Or, another option in perspective is WASI-on-the-web is off topic here, but that raises the above concerns.

[sunfishcode]

Interface types are something that we'd expect to use at a layer underneath WASI, in both non-Web and (hypothetical) Web contexts. Interface types aren't an alternative to WASI.

[linclark]

I would like to remind those posting in this thread about the Code of Ethics and Professional Conduct, particularly the section about maintaining that courtesy, respect, and dignity when expressing disagreement.

[sunfishcode]

In WASI's APIs that take strings, today, we literally write string in our witx interface descriptions. We write it like this, today, even though string today is actually lowered to a pointer+length pair of i32s. We expect that in the future, WebAssembly will give us something better to lower string to. When that arrives, no matter what it looks like, interface types or not, WASI will change the lowering code to use it instead.

WASI's APIs today are called "snapshots" because WASI is still evolving, and string lowering is part of that.

The reason for working on WASI now, instead of waiting until all optimizations that all languages and use cases want are ready first, is that most of the work of WASI is independent of these optimizations. There isn't any need to block this work, as it's mostly a small change to swap out the string lowering at any time.

This is also the reason that, again, the strengths and weakness of interface types are off-topic in WASI forums. The WASI Subgroup isn't the group that will ultimately make those decisions, so it isn't productive to discuss it here.

[tlively]

I thought that @sunfishcode made some good points about why we can be optimistic about WASI's future Web-compatibility, including a new higher-level I/O interface and fewer strings in important interfaces over time. @sbc100's comment and the docs he linked to also make it clear that while Web interoperability is not a primary goal for WASI right now, neither is WASI going out of its way to make Web compatibility more difficult. It would be asking quite a lot of the folks working on WASI to ask them to compromise on their primary goals and potentially regress WASI's functionality to improve Web-compatibility in the short term. Yes, this results in some fragmentation between the WASI ecosystem and other WebAssembly ecosystems, but that's probably ok for now. As a loose analogy, there are no universal x86 or ARM ecosystems, either. It would be a great success for Wasm as a technology to be the foundation on which multiple important ecosystems are built.

[dcodeIO]

Sorry, I am not sure if your answer implies that we should think about it more and potentially improve it, or that my concerns are irrelevant since fragmentation is OK for now (which may then be forever)? Like, I'd argue it would be an even greater success for Wasm to avoid fragmentation, i.e. similar to how Wasm abstracts over x86 and ARM, it may be equally valuable to do our best to reasonably abstract over Web/non-Web as well, say in a way that doesn't hurt the Web where code size is most important?

For example, Web APIs like the following are often essential for basic Wasm modules just to function

• Date.now
• Date#getTimezoneOffset (say hi to new)
• performance.now
• crypto.getRandomValues (say hi to Uint8Array)
• console.log/debug/info/warn/error (say hi to WTF-16)
• Intl.* maybe
• ...

yet there is no way currently to target these in a portable way (be it because Non-Web doesn't support them or because Interface Types will not be available (on the Web) anytime soon), while all hands are on deck for WASI which imo prematurely promotes fragmentation, leaving everyone with an interest to create half-way portable modules no other choice than to use WASI with a polyfill on the Web, something we'll then be stuck with for a very long time.

[tlively]

Your concerns are certainly valid and reasonable, but it's important to recognize that your goals are not exactly the same as WASI's goals and that's ok. @sunfishcode and @sbc100 have indicated that they are open to considering proposals to promote WASI's Web compatibility, but only as long as those proposals are also consistent with WASI's primary goal of providing a capability-based security model for non-Web use cases. When you call WASI "premature," it sounds like you want it to stop in its tracks and rethink its core goals so that it supports Web use cases at the same level as non-Web use cases. I agree that it would be awesome if WASI did support Web use cases co-equally, but it does not and does not seek to do so right now, and it is not reasonable or productive to ask the entire project to change its goals to alternatives that have already been considered at length.

Since you want portability, but you don't want the code size burden of shipping a WASI polyfill on the Web, would it be feasible for you to instead target a Web-native set of APIs and ship a polyfill for use on WASI engines?

[dcodeIO]

Yeah, at the end of the day I am questioning not only WASI, but the general direction of Wasm here, that inevitably led us to WASI. It's all connected I guess, from WTF-16 over Universal Strings to WASI and I understand that me showing up with that in GitHub issues, which only can convey so much in text form, is unpleasant, especially when it opposes other party's goals (i.e. the core goals of WASI), and as such can easily be perceived as hubris because who am I even. I am nobody, I have no corporate agenda, no lobby, but for some weird reason I genuinely care about this more interconnected stuff to hopefully make Wasm the best it can be.

As such, yeah, approaching matters from the inverse perspective, that is design Wasm (and by extension WASI) with the Web in mind whenever a new proposal is made, and then do what we can to integrate with non-Web (as of this issue) in a portable way, is exactly what I am advocating for. Like, abstracting over CPU architectures is such an impressive feat, and imo we shouldn't stop there and just call it a day. See, I don't want to be like that, I only want my view on things to be part of this, with my voice heard instead of being fended off one epic uphill battle after the other, so we can eventually cooperate to achieve something great. And sadly, WASI doesn't quite cut it for me.

[devsnek]

You seem to be interested in a very prescriptive usage of wasm, while the cg/wg seem more interested in a more descriptive design where they attempt to provide interop with systems that people are actually using. I think discussion about that is worthwhile to some degree (both have pros and cons), but I don't think this is the place.

[dcodeIO]

It's quite the dilemma, isn't it. There is no place for a nobody like me in Wasm, but perhaps there should be.

[tlively]

On the contrary, I think it's very valuable to have you advocating for a Web-first system interface. Emscripten has deliberately had an unstable system interface so far, but only because there hasn't been an appropriate standard alternative. We also had the experience of trying to adopt WASI and finding that much of it was too inefficient on the Web for us to adopt fully. It looks like there is plenty of space in the ecosystem for something similar to WASI, but focused on the Web. Such a project might share tooling with WASI and even merge with WASI in the future once they are both more mature, but there's no reason they would have to be developed by the same group right now.

[sbc100]

WABI?

[sunfishcode]

@tlively As we've discussed, there are many optimizations possible in WASI. If you and your team would be interested in engaging with WASI, I believe we could accomplish a lot more together.

[tlively]

To be clear, I have no plans to work on a WASI alternative :) I'm not sure what WASI's stance is on having multiple ways to do the same thing, but if that would be acceptable, I think the simplest and most appropriate direction would be to create new Web-focused APIs under the WASI umbrella. That would not change the goals of any existing WASI work, but would add new WASI proposals with different, more Web-focused goals. I do agree that working together is better than working apart. On the other hand, if @dcodeIO wants to implement something that is not based on interface types and capability-based security, the WASI project is probably not the best place for that.

[kripken]

I agree that it's better to work together to improve Web APIs and WASI APIs rather than add anything new.

Regarding polyfilling, as mentioned earlier the best we can do now for a single build is something like this:

• project.wasi.wasm: WASI-using wasm module, optimized for and runnable in a WASI VM
• wasi.js: WASI-implementing JS shim for running on the Web

On the server you ship project.wasi.wasm, and on the Web you also ship wasi.js. This has the downsides @dcodeIO mentioned earlier. It may also be worth experimenting with the mirror opposite,

• project.web.wasm: Web API-using wasm module, optimized for and runnable on the Web
• web.wasm: Web API-implementing Wasm shim for running in a WASI VM

Concretely, in the latter case the wasm could import performance.now,

(import "performance" "now" (func (result f64)))

(very concise). And web.wasm would convert that into a call to wasi.clock_time_get as WASI requires.

This approach would be useful for having Wasm files portable between the two standard APIs, in this case with a more Web-optimized result. Another benefit, and the reason I expect we will see this eventually regardless of everything else, is that it would allow porting of existing Wasm files from the Web to WASI. That would be very useful I think.

One limitation this approach has is that web.wasm would need to be linked in statically atm, but when linking modules is possible in WASI then distributing web.wasm alongside a project could be very simple.

Another limitation is that while APIs like performance.now would just work, others might need some amount of JS. It's not large - that's what we are doing now, basically - but this could be improved in the Wasm JS API. So there is possible work on the Web side here.

On the WASI side, the largest blocker is that some Web APIs can't be polyfilled in WASI atm, even with reference types (and I don't think even with Interface Types - the issue isn't interfacing/sending values but rather representing and manipulating them). Adding new APIs to WASI could solve that. If that direction sounds interesting to people I can sketch out what I think such APIs might look like?

[linclark]

I’m going to close this out because this issue has been addressed in the broader CG forum. As many people explained during the course of those discussions, the concern that WASI will harm the web is not well supported.

I also wanted to provide a bit of a retrospective here:

There were a number of personal attacks and accusations thrown at various CG members by a particular individual in the discussions across different threads. My subtle CoC warning above was in response to an accusation that this individual made, though this individual has since deleted the message I was responding to. They also conducted personal attacks in other threads across the CG's repos.

This person has left the CG since then, so we don’t expect them to continue with these behaviors in this space. However, if anyone engages in CoC violating behavior towards you or others within the context of the CG, please do reach out. Part of my job as WASI co-chair is to make sure this is a safe space for you to collaborate, and I take that job seriously. In the wider CG, the chairs are also working on processes to better support people when they are the target of CoC violating behavior.

Thank you to everyone who has worked to make the collaboration within the WASI subgroup (and the collaboration in the wider CG) enjoyable and productive.