Support for customizing the URL that is loaded (for Chrome Extensions)

[Pauan]

I'm using cargo-web + stdweb to create Chrome Extensions. It generally works well, but there is one issue.

To explain, here is what the .js looks like when compiling with the wasm32-unknown-unknown target:

"use strict";

if( typeof Rust === "undefined" ) {
    var Rust = {};
}

(function( root, factory ) {
    if( typeof define === "function" && define.amd ) {
        define( [], factory );
    } else if( typeof module === "object" && module.exports ) {
        module.exports = factory();
    } else {
        Rust.saltybet = factory();
    }
}( this, function() {
    ...

    if( typeof window === "undefined" ) {
        const fs = require( "fs" );
        const path = require( "path" );
        const wasm_path = path.join( __dirname, "saltybet.wasm" );
        const buffer = fs.readFileSync( wasm_path );
        const mod = new WebAssembly.Module( buffer );

        return __initialize( mod, false );
    } else {
        return fetch( "saltybet.wasm" )
            .then( response => response.arrayBuffer() )
            .then( bytes => WebAssembly.compile( bytes ) )
            .then( mod => __initialize( mod, true ) );
    }
}));

The important part is fetch( "saltybet.wasm" ). Because of the way that Chrome Extensions work, that causes it to load the wrong URL, so it doesn't work.

In order to fix this, I have to manually edit the .js file and change it to use fetch(chrome.runtime.getURL("saltybet.wasm")), and now it works correctly.

Obviously needing to manually edit the .js file every time I make a change is very annoying, so it would be nice to be able to have cargo-web use the correct URL.

Since this only applies to Chrome Extensions, it will need to be a flag (or Web.toml setting).

[Pauan]

Alternatively, rather than hard-coding in support for Chrome Extensions, cargo-web can instead export a function:

"use strict";

if( typeof Rust === "undefined" ) {
    var Rust = {};
}

(function( root, factory ) {
    if( typeof define === "function" && define.amd ) {
        define( [], factory );
    } else if( typeof module === "object" && module.exports ) {
        module.exports = factory();
    } else {
        Rust.saltybet = factory();
    }
}( this, function() {
    ...

    return {
        loadURL: function ( url ) {
            if( typeof window === "undefined" ) {
                const fs = require( "fs" );
                const buffer = fs.readFileSync( url );
                const mod = new WebAssembly.Module( buffer );
                return __initialize( mod, false );

            } else {
                return fetch( url )
                    .then( response => response.arrayBuffer() )
                    .then( bytes => WebAssembly.compile( bytes ) )
                    .then( mod => __initialize( mod, true ) );
            }
        }
    };
}));

Then I can call that function to load whatever URL I want:

Rust.saltybet.loadURL(chrome.runtime.getURL("saltybet.wasm"));

[koute]

This is something I do want to support, however I don't want to have to support a gazillion different runtimes.

Currently cargo-web can actually emit two different runtimes using the hidden (and unstable) argument --runtime - the non-default runtime is used by my Parcel plugin, although for e.g. Webpack it's not ideal (https://github.com/koute/cargo-web/issues/35). What I'd like to have eventually is (if possible) two runtimes - the default one, which does everything automatically, and a semi-manual one where you have to fetch and instantiate the wasm module yourself (so it'd be appropriate for bundlers and in cases where you need to do something custom).

[Pauan]

@koute I don't think you would need a gazillion runtimes, since it can export multiple functions:

return {
    loadURL: function ( url ) {
        if( typeof window === "undefined" ) {
            const fs = require( "fs" );
            const buffer = fs.readFileSync( url );
            const mod = new WebAssembly.Module( buffer );
            return __initialize( mod, false );

        } else {
            return fetch( url )
                .then( response => response.arrayBuffer() )
                .then( bytes => WebAssembly.compile( bytes ) )
                .then( mod => __initialize( mod, true ) );
        }
    },
    instantiate: function ( instance ) {
        return __instantiate( instance );
    }
};

Now there's two ways to use it: the user can handle everything themself and call instantiate, or they can call loadURL which does everything (except they can specify the URL).

[koute]

The problem is that different people have different ideas on how this should look so it's hard to please everybody; also JS bundlers really don't like code like this (e.g. some of them try to be smart and scan the source for stuff like fs.readFileSync and do the wrong thing). Having two runtimes - a default one, and one you could instantiate yourself manually seems like the least painful way to do this.

[Pauan]

Well, whatever you decide for the manual runtime, it should support WebAssembly.instantiateStreaming (since that's the most efficient way to load wasm).

Maybe something like this:

WebAssembly.instantiateStreaming(
    fetch(chrome.runtime.getURL("saltybet.wasm")),
    Rust.saltybet.imports
).then((x) => Rust.saltybet.instantiate(x.instance));

Note: the above code is what I (or other users) would use, it wouldn't be inside of the runtime.

[hodlbank]

FYI we had to use following bash script to replace Module.export with function that accepts custom fetchWasm function, so web application would provide correct URL to wasm loader:

#!/bin/bash

rustup run nightly-x86_64-unknown-linux-gnu cargo web build --target wasm32-unknown-unknown --release
if [ $? -ne 0 ]; then { echo "Build failed"; exit 1; } fi
cp target/wasm32-unknown-unknown/release/someproject_wasm.{js,wasm} ./build/
if [ $? -ne 0 ]; then { echo "Copy failed"; exit 2; } fi
sed -i 's/return fetch(/return (fetchWasm) => { return fetchWasm(/' ./build/someproject_wasm.js
sed -i 's/\.then( mod => __initialize( mod, true ) );/&}/' ./build/someproject_wasm.js

So then in my application I have SomeLibLoader class with method as following:

import * as somelib_wasm from '../build/someproject_wasm.js';

export class SomeLibLoader {
  private static somelib_wasm_instance: any | null = null;

  public static load(arg_fetchWasmFn: Function): Promise<SomeLib> {
    return new Promise<SomeLib>((resolve, reject) => {
      if (this.somelib_wasm_instance != null) return Promise.resolve(this.somelib_wasm_instance);
      // asynchronous environment (i.e. browser)
      if (typeof somelib_wasm == 'function') {
        somelib_wasm(arg_fetchWasmFn)
        .then((res_somelib_wasm_instance) => {
          this.somelib_wasm_instance = res_somelib_wasm_instance;
          resolve( new SomeLib(this.somelib_wasm_instance) );
        });
      } else {
        this.somelib_wasm_instance = somelib_wasm;
        resolve( new SomeLib(this.somelib_wasm_instance) );
      }
    })

  }

}

Hope this helps. References: #35 , #71 .

[madmaxio]

Does any plan exist for this issue?

[Pauan]

@madmaxio This issue has already been fixed: you can use --runtime web-extension to compile for Chrome/Firefox/Edge/Opera extensions.

If you need more control, you can instead use --runtime library-es6 and then handle fetching/instantiation yourself.

[madmaxio]

Ah, looks like I choosed the wrong issue. I need customize wasm fetch path for standalone runtime (default one). Is this possible now?

[Pauan]

@madmaxio I don't think so. You'll have to use --runtime library-es6 and then handle the loading yourself:

import factory from "./path/to/my-module.js";

const url = "./path/to/my-module.wasm";

const instance = factory();

WebAssembly.instantiateStreaming(fetch(url, { credentials: "same-origin" }), instance.imports)
    .then(function (x) { instance.initialize(x.instance); });

[Pauan]

(Personally, I agree that there should be some sort of Web.toml flag to customize the folder/URL for the standalone runtime. It's a common need. And other bundlers like Webpack allow for folder loading customization, so cargo-web should too.)