Add JS Packaging

[bgins]

Summary

Problem

We do not package rs-ucan for JavaScript environments.

Impact

rs-ucan cannot be used in web browsers, node, and other JavaScript environments.

Solution

Add a crate that acts as a thin layer over the ucan and ucan-key-support crates. The crate should compile to Wasm and provide JS packaging.

Detail

Is your feature request related to a problem? Please describe.

We want to use rs-ucan in the UCAN validator. See https://github.com/ucan-wg/rs-ucan/issues/19#issuecomment-1549929270 for more detail.

Describe the solution you'd like

Add a ucan-wasm crate that produces a JS-packaged version of rs-ucan.

Additional context

Implementing initial support with a limited set of features and targets will probably be best. This approach will let us get the machinery in place and support the UCAN validator as a first target.

ETA: 2023-09-01

[bgins]

Recording some notes on the wasm-pack and wasm-bindgen tooling available today.

Our goal in selecting a packaging approach is maximum support across targets with the best possible developer ergonomics. As noted above, our initial goal will be to support the UCAN validator, which lives in a web app. In the long run, we may need to use multiple approaches to support all targets.

---

wasm-pack

wasm-pack supports a few targets, including bundler, web, node, and no-modules.

The bundler target relies on the esm-integration proposal (https://github.com/WebAssembly/esm-integration), which currently has limited bundler support. For example, it does not have first-class support in Vite, though it can be used with a Vite plugin.

The web target forces downstream users to call an init function to instantiate the Wasm module, which is less than ideal.

The node and no-modules targets don't fit our initial use case, but they should definitely be considered in the future!

wasm-bindgen

wasm-bindgen can be used without wasm-pack to get more control over the package output. For example, we may want our own package.json instead of the one created by wasm-pack.

One benefit of this approach is that we can include a custom entry point that calls the init function for users, which seems like an ideal developer experience. A developer could import from the package like any JavaScript package without awareness of the underlying Wasm.

Unfortunately, this approach relies on top-level await to load the module. A custom entry point might look like this:

import init from './ucan-wasm.js';

await init();

export * from './ucan-wasm.js';

Top-level await makes a module asynchronous, which means init will finish before the exports are available. See https://exploringjs.com/impatient-js/ch_modules.html#top-level-await for more details.

Top-level await is available in all modern browsers, but support is still lacking in some bundlers like esbuild.

Let's pick one

wasm-pack with the web target seems the best place to start. It doesn't rely on bundler plugins, and it should work with most bundlers (I will verify this while implementing packaging support). If we decide to take another approach in the future, developers should be able to drop the init call from their code, and everything should work.

[fabricedesre]

The other option is to not use wasm-bindgen (it's quite opinionated and bloated imho) and use the wasm component model (start at https://github.com/bytecodealliance/wit-bindgen). I've had good results with it and found it more enjoyable to use than the reflection used with wasm-bindgen eg. to access WebCrypto.

[cdata]

The scope of JS packaging is somewhat narrow compared to packaging in general, and with that in mind it can pay off to have opinions. We don't currently have a recorded use case for packaging as Wasm for use outside of JS (although I'm sure such use cases exist).

@fabricedesre could you share any examples that come to mind that may show off the advantages of the less bloated wit-bindgen approach?

The node and no-modules targets don't fit our initial use case, but they should definitely be considered in the future!

The issue states that Node.js and "other" JS environments are in scope for JS packaging. Should I understand that the initial use case is scoped to main-thread web (excluding web workers, I suppose, since we're assuming ESM)?

As a general comment: JS bundling is a fraught exercise. There are about as many opinions as there are tools available for bundling, and no clear consensus in the community on the topic of tool interoperability. So, even if we bless a bundler strategy such as wasm-pack, we'll most likely be doing so with trade-offs on the consumption side.

In other projects I have worked on, we have assumed that the NPM package consumer is probably using a bundler of their choice. This frees the package maintainer from having a strong opinion on bundling. Instead, they can worry about distributing a package that has a very good chance of being easily bundled. And today we have such niceties as ESM and Import Maps on the web, and configurable package entrypoints in Node.js, making this an easier strategy than it has been historically.

With the above in mind, it seems like wasm-bindgen is the right tool for the job, since it does not concern itself with bundling per se. Maybe there is a similarly interesting approach to consider with wit-bindgen.

This is an aside: this article crossed my feed today from a former colleague; it may contain some topical pointers: https://www.donmccurdy.com/2023/05/24/publishing-webassembly-modules-with-assemblyscript/

[bgins]

Thanks for the Don McCurdy article! Good to see this approach. 🙏

The issue states that Node.js and "other" JS environments are in scope for JS packaging. Should I understand that the initial use case is scoped to main-thread web (excluding web workers, I suppose, since we're assuming ESM)?

Yep, the initial use case is main-thread web only. My initial thought was to only package for that, but it does seem like it would be better to package for a couple of environments.

wasm-pack doesn't seem up to that task because it can target one environment or another, but not multiple environments in a single package.

@hugomrdias mentioned Automerge, which also packages for multiple environments. From their package.json, it looks like they target browsers, node, deno, and Cloudflare workers: https://github.com/automerge/automerge/blob/main/rust/automerge-wasm/package.json

The other option is to not use wasm-bindgen (it's quite opinionated and bloated imho) and use the wasm component model (start at https://github.com/bytecodealliance/wit-bindgen). I've had good results with it and found it more enjoyable to use than the reflection used with wasm-bindgen eg. to access WebCrypto.

I would be open to trying wit-bindgen! A working example would be helpful for getting a sense for it. We are definitely going to need WebCrypto, so anything that makes that more pleasant is appreciated. 💯

[fabricedesre]

Let me try to re-vive my wit-bindgen branch. When I first tried I hit https://github.com/bytecodealliance/wit-bindgen/issues/305 but that got fixed since.

[cdata]

We are definitely going to need WebCrypto, so anything that makes that more pleasant is appreciated.

FWIW we already have support for RSA keys generated with Web Crypto in the ucan-key-support crate, via web-sys: https://github.com/ucan-wg/rs-ucan/blob/main/ucan-key-support/src/web_crypto.rs