loading web assembly files (.wasm)

[shaunlebron]

I saw a question on #clojurescript about this.

Maybe we want to load a Web Assembly module in cljs like this:

(ns my.example
  (:require ["./foo.wasm" as foo]))  ;; <-- hypothetical wasm loader needed

There is work happening to integrate wasm into the normal ecmascript module loading system via:

import "./foo.wasm";  // <-- (hypothetical static import)
import("./foo.wasm"); // <-- (hypothetical dynamic import)

In the meantime, cljs core will probably wait to load these until google closure does, but maybe shadow-cljs can add a loader that does the following (source):

1. Get the .wasm bytes into a typed array or ArrayBuffer
2. Compile the bytes into a WebAssembly.Module
3. Instantiate the WebAssembly.Module with imports to get the callable exports

For reference—how folks are doing it in the webpack world:

• official wasm loader in webpack
• third party wasm-loader

[thheller]

I'm not against adding support for .wasm but its probably not going to be part of ns :require.

The .wasm files I have seen to date are either tiny toy examples or gigantic (megabytes) of compiled C/WhateverLang. Tiny files could be inlined into the JS via base64 encoded strings but that loses viability quickly as files tend to be become way larger than the binary form. So it is probably safer to assume async as the default and that could probably all the done via some CLJS helper function with maybe a macro or two around js/fetch and the normal JS API.

As far as the build aspect is concerned it probably is closed to the planned support for other "static" assets such as CSS so that they are copied from the classpath to the output-dir but that's probably it.

Would help if you have some concrete example .wasm files I can experiment with. Ideally something practical thats not just a hello world style dummy function.

[thheller]

Did some experiments yesterday since this example was posted in the #shadow-cljs channel. It is a .wasm file generated by Rust tools and as expected the problem is not so much with the .wasm but rather how it is "pre-bundled" by the tools.

It generates a .js file and a .wasm file that are coupled together pretty tightly.

The .js file will import the .wasm file directly and it also has a couple of exported functions that in turn the .wasm file will import directly via an import of its own.

import * as wasm from './game_of_life_3d_bg';
...
const __wbg_timeEnd_5ba22134470392e6_target = console.timeEnd;

export function __wbg_timeEnd_5ba22134470392e6(arg0, arg1) {
    let varg0 = getStringFromWasm(arg0, arg1);
    __wbg_timeEnd_5ba22134470392e6_target(varg0);
}

(import "./game_of_life_3d" "__wbg_time_ff03c978915ebe44" (func $./game_of_life_3d.__wbg_time_ff03c978915ebe44 (type $t2)))
(import "./game_of_life_3d" "__wbg_timeEnd_5ba22134470392e6" (func $./game_of_life_3d.__wbg_timeEnd_5ba22134470392e6 (type $t2)))
(import "./game_of_life_3d" "__wbg_random_86efc8986c8a8805" (func $./game_of_life_3d.__wbg_random_86efc8986c8a8805 (type $t3)))
(import "./game_of_life_3d" "__wbindgen_throw" (func $./game_of_life_3d.__wbindgen_throw (type $t2)))

So we can't just import the JS and modify it in any way without also modifying the .wasm file. All the other basic examples always provide an import object where you basically pass the required functions into the .wasm on instantiate but the other tools all seem to have their own idea about how wasm should be instantiated.

It would be much easier to work with this if the binding generators just output a .json file that describes the expected imports/exports so that tools can consume it without having to rewrite everything that was generated. I have never used .wasm and don't know how practical that would be though. Also no idea if tools have other options other than the example above.

[thheller]

Most recent update looking at .wasm. Getting better but still not smooth. Likely still ways off till I can actually support this in shadow-cljs directly.

https://github.com/thheller/wasm-pack-cljs

[saskenuba]

Hey, did you have the time to look at this recently? Wasm support is getting better and better, perhaps something changed?

[thheller]

It is still solely dependent on what the "glue" JS looks like. Until this is standardized each X-to-WASM compiler will do its own thing and that hasn't changed much AFAICT.