Support importing wasm modules

[ry]

import * as foo from "./foo.wasm";

where foo will correspond to the exports of the wasm module.

[kitsonk]

For reference, here is the README about the integration. It is a Stage 1 proposal (and the WebAssembly committee follows the TC39 stage process), which means it is still quite early. I think this could be important to Deno, so maybe getting involved early would be a good thing, but is likely to be a bit of a rough road.

This will present interesting challenges for TypeScript and trying to understand the exports of a WASM module, but I suspect this would be an iterative process, where it would be just an untyped import, but eventually could try to figure out a way to evaluate/type on the fly the imports and exports of a WASM Module. It is going to be fairly complex I suspect, with some mediation between V8 and the compiler.

[kitsonk]

Just an update, the standard has moved to Stage 2. An important note in the proposal is that TLA is potentially needed to evaluate the module and hand it back to the ESM importer, so it is likely we need a version of TLA in V8 before we can really address this.

[dsseng]

I have some ideas and I probably will work on this one. Compiler for that should produce smth like this:

const { instance } = await WebAssembly.instantiate(Uint8Array.from(atob("<WASM as base64>"), c => c.charCodeAt(0)));
module.exports = instance.exports;

.wasm file gets transformed to Base64 in compiler file (written in Rust) and then decoded & instantiated by Deno runtime, then exported. Instantiation function reference is here P.S. I checked this manually, it works just fine. P.P.S. module.exports seems to be invalid in Deno, maybe we should use an API like this that'll allow imports from JS to WASM too? Or we can just export the exports object. @ry what do you think?

[kevinkassimo]

We should be able to close this with #3328

[hayd]

It'd be great to have a toy example in the docs/manual.

[kevinkassimo]

@hayd that could be a good first issue 😄

[kevinkassimo]

@sh7dm Ooops, did not realize that you are working on this too... Only noticed this issue just now. But basically I also thought of the base64 solution (and I guess it might work well with bundling). I implemented the thing by generating a dynamic script.

Would love to hear your feedback since currently in #3348 we have double compilation issue (and I did a hack to placate TS compiler... which might not be the most elegant thing)

[hayd]

@kevinkassimo I will add an example to the manual.

[hayd]

@kevinkassimo am i missing something here:

 $ cat fib.as.ts
export function fib(n: i32): i32 {
  var a = 0, b = 1;
  for (let i = 0; i < n; i++) {
    let t = a + b; a = b; b = t;
  }
  return b;
}

 $ asc fib.as.ts -b fib.wasm -O3

 $ cat fib.ts
import { fib } from "./fib.wasm";

console.log(fib(5));

 $ ./target/release/deno fib.ts
Compile file:///Users/hayd/OSS/deno/fib.ts
error: Uncaught ImportPrefixMissing: relative import path "env" not prefixed with / or ./ or ../
► $deno$/dispatch_json.ts:40:11
    at DenoError ($deno$/errors.ts:20:5)
    at unwrapResponse ($deno$/dispatch_json.ts:40:11)
    at sendAsync ($deno$/dispatch_json.ts:91:10)

Or is asc not doing the right thing?

[kevinkassimo]

@hayd can you get asc output .wat format?

[kevinkassimo]

@hayd I tried asc -t, and saw the following lines:

(module
  ;; ...
  (import "env" "abort" (func $~lib/builtins/abort (param i32 i32 i32 i32)))
  ;; ...
)

Looks like this result is prepared for npx asinit ., and running this command generates script index.js of the following format:

const fs = require("fs");
const compiled = new WebAssembly.Module(fs.readFileSync(__dirname + "/build/optimized.wasm"));
const imports = {
  env: {
    abort(_msg, _file, line, column) {
       console.error("abort called at index.ts:" + line + ":" + column);
    }
  }
};
Object.defineProperty(module, "exports", {
  get: () => new WebAssembly.Instance(compiled, imports).exports
});

This is why there is an attempt to import env. It seems asc does not handle the possibility of wasm imports

[kevinkassimo]

@hayd Figured out. You need to compile with the following flags:

asc --use "abort=" fib.as.ts -b fib.wasm -O3

This basically disables builtin abort thus removing the invalid import statement.

Now fib.ts works for me:

$ ./target/debug/deno ../test/wasm/fib.ts
Compile file:///Users/kun/Projects/Deno/test/wasm/fib.ts
8

[dsseng]

@kevinkassimo no problem, just had an idea, didn't work on anything yet.

[rsp]

Could it be useful to run/import text files (wat/wit/witx) in addition to binary files?

• by Deno bundling a wat to wasm compiler, like it bundles the tsc
• by having a wat to wasm compiler in std
• by bundling a fast s-expression parser

Some formats to keep an eye on:

• wat (wasm text format)
• wit (for module types)
• witx (for WASI APIs)

The formats that describe the types might be especially useful.

[SyrupThinker]

Any reason why this hasn't been closed?

[sithumonline]

Any reason why this hasn't been closed?

Manual documentation for wasm import is not complete. https://github.com/denoland/deno/pull/3351#pullrequestreview-317411415

[SyrupThinker]

The manual notes that wasm can be imported as it had been added later that day: https://github.com/denoland/deno/commit/411f53f7bbff54ff8b16f4d4c0f6d2ef573926f1 https://deno.land/std/manual.md#wasm-support

[sithumonline]

@SyrupThinker , I think @ry will close #2552 & #3351

[icyJoseph]

As of https://github.com/denoland/deno/pull/5135 this has been temporarily removed. Search engine results however still do not show #5135 above this issue.

[Ciantic]

Can we re-open this? Or add some tag at least to indicate it's waiting for ECMAScript Stage 3.

I had to manually go through the thread to see why this was closed, only to find out the support was once added and then removed.

[kitsonk]

Update

This is the status page on the platform status: https://www.chromestatus.com/feature/5987156463976448#details

There is no active development in Chromium and there doesn't look like there is a lot of interest or activity there. Seems everyone has accepted the "fetch" instantiate streaming method. The big challenges with that for Deno are:

• Requires explicit permissions to be able to load web assembly
• fetch() does not currently support local files, which means you have to offload it to Deno.readFile() but that isn't currently easily interoperable with web streams.

It won't come down the TC39 proposal route, because it has "nothing" to do with ECMAScript.

Currently the proposal does not use ECMAScript import assertions: https://webassembly.github.io/esm-integration/js-api/index.html#esm-integration. ~I haven't found a relevant discussion where the Web Assembly committee is considering it either.~ It was briefly discussed here, but no clear actions: https://github.com/WebAssembly/esm-integration/issues/42 nor any obvious progress.

[guybedford]

Has there been any progress on discussion on https://github.com/denoland/deno/issues/2150?

[kitsonk]

Has there been any progress on discussion on #2150?

Yes. Within the core team. It is something we are going to discuss in depth soon. We just have to get over how we deal with window.location and related APIs when a --location is not provided.

[lucacasonato]

There has been considerable movement on this (thanks @takikawa). There is now an experimental implementation of WASM imports in WebKit, behind a flag. We should implement WASM imports again as per the most up to date specs: https://webassembly.github.io/esm-integration/js-api/index.html#esm-integration and https://github.com/whatwg/html/pull/4372.

I think the general approach for us would be the following:

1. deno_graph needs to understand WASM modules. It needs to be able to analyze imports from within WASM.
2. deno_core needs to have a WASM import type.
   • We need to parse the module.
   • Instantiate the module with it's "imports" set to the imports parsed out by deno_graph.
   • Create a synthetic module with it's module namespace set to the exports of the instantiated module.

Part 2 could be simplified once V8 implements WASM modules itself. The deno_graph changes are required regardless.

[bartlomieju]

I can work on the deno_core part, should be straight-forward. @kitsonk could you look into deno_graph support?

[sant123]

I'm curious in how the wasm module will be imported. If using the assert import or not.

[redradist]

Is there any progress on it ?

Wasm Modules is currently at Stage 2 and usually Deno implement interesting features at this stage ...

[nicolo-ribaudo]

Fyi, https://github.com/whatwg/html/pull/10380 has been merged and the Wasm proposal is at phase 3 ("implementation phase").