Explore AssemblyScript integration

[mvaligursky]

We have a PR https://github.com/playcanvas/engine/pull/2213 which explores one way of using AssemblyScript (WASM) modules by Playcanvas. I would like to explore a way to build custom functions using AssemblyScript, which can be called by the main engine in JS to optimize hot functions / but perhaps do more in the future.

Main objectives:

• build system should build the engine as currently, but build wasm module along it from separate ts files.

  perhaps https://www.npmjs.com/package/rollup-plugin-assemblyscript could be useful here

  but also this is interesting as it allows to embed the wasm module into main script https://blog.scottlogic.com/2017/10/30/migrating-d3-force-layout-to-webassembly.html https://github.com/rollup/plugins/tree/master/packages/wasm

• AssemblyScript allows code to be compiled to WASM, but also generate TS code. It would be good to have this as an option during development / debugging, allowing easy debugging of AssemblyScript code in form of TS.

Later:

• AssemblyScript has support for SIMD and atomics (experimental features), it'd be great to see if these can be used. I'm not sure what the impact is of the code using these still being able to compile to TS.

[kungfooman]

As far I know AssemblyScript allows code to be compiled to WASM, but also generate TS code is not exactly true. AS has these outputs:

asc --help

Output

  --outFile, -o         Specifies the output file. File extension indicates format.
  --binaryFile, -b      Specifies the binary output file (.wasm).
  --textFile, -t        Specifies the text output file (.wat).
  --jsFile, -j          Specifies the JavaScript (via wasm2js) output file (.js).
  --idlFile, -i         Specifies the WebIDL output file (.webidl).
  --tsdFile, -d         Specifies the TypeScript definition output file (.d.ts).

And the wasm2js output looks like this:

function assembly_Quat_Quat_clone($0) {
  $0 = $0 | 0;
  return assembly_Quat_Quat_constructor(0, HEAPF32[$0 >> 2], HEAPF32[($0 + 4 | 0) >> 2], HEAPF32[($0 + 8 | 0) >> 2], HEAPF32[($0 + 12 | 0) >> 2]) | 0;
 }

 function assembly_Quat_Quat_conjugate($0) {
  $0 = $0 | 0;
  HEAPF32[$0 >> 2] = Math_fround(HEAPF32[$0 >> 2] * Math_fround(-1.0));
  HEAPF32[($0 + 4 | 0) >> 2] = Math_fround(HEAPF32[($0 + 4 | 0) >> 2] * Math_fround(-1.0));
  HEAPF32[($0 + 8 | 0) >> 2] = Math_fround(HEAPF32[($0 + 8 | 0) >> 2] * Math_fround(-1.0));
  return $0 | 0;
 }

(not exactly debug friendly)

The other option you might refer to is: https://www.assemblyscript.org/portability.html

In this case AS generates nothing, but the .ts-files are consumed directly by e.g. rollup and AS should provide polyfills for stuff like Mathf.sincos

Small problem could be: The portable standard library is still pretty much a work in progress and we are extending it as we go while working on the compiler.

Just the info I found so far, I like your idea to test the AS code from pure JS (because that makes it way easier to debug/test), I just need to investigate further how to do it best in my PR 😅

[mvaligursky]

Hi @kungfooman - thanks for taking the interest in this.

I was reading this yeserday https://blog.scottlogic.com/2017/10/30/migrating-d3-force-layout-to-webassembly.html see this part.

Being able to run the same code as both JavaScript (via TypeScript) and WASM is great for debugging, I found quite a few errors that in my module code that would have been really hard to track down if I only had access to the compiled module.

[kungfooman]

I integrated your idea to use AssemblyScript from pure JavaScript for debugging purposes in my PR now. It was a bit more tricky than expected, because I tried to optimize and strip down everything that isn't "really" needed for the AS code:

• Only allow new pc.Vec3(1, 2, 3), not new pc.Vec3([1, 2, 3])
• Remove toString() methods to not mess around with strings in AS
• Removed optional arguments (every argument is required in AS math code)
• Normal Mat3/Mat4 uses Float32Array .data internally, which I replaced with direct access fields named m0, m1, m2, ... to have the least amount of indirect data access possible

So the job of the JavaScript wrapper over the AssemblyScript code resulted in readding these four points. As an example to readd the old abilities, this is how the pc.Quat JS wrapper over the AS code looks like:

import { Vec3 } from "./vec3";

import { Quat as Quat_AS } from "../../assembly/Quat";

class Quat extends Quat_AS {
    constructor(x?: any, y?: any, z?: any, w?: any) {
        if (x && x.length === 4) {
            super(
                x[0],
                x[1],
                x[2],
                x[3]
            );
        } else {
            super(
                (x === undefined) ? 0 : x,
                (y === undefined) ? 0 : y,
                (z === undefined) ? 0 : z,
                (w === undefined) ? 1 : w
            );
        }
    }

    getEulerAngles(eulers) {
        if (eulers === undefined) {
            eulers = new Vec3();
        }
        return Quat_AS.prototype.getEulerAngles.call(this, eulers);
    }

    transformVector(vec, res) {
        if (res === undefined) {
            res = new Vec3();
        }
        return Quat_AS.prototype.transformVector.call(this, vec, res);
    }

    toString() {
        return '[' + this.x + ', ' + this.y + ', ' + this.z + ', ' + this.w + ']';
    }

    toStringFixed(n) {
        return '[' + this.x.toFixed(n) + ', ' + this.y.toFixed(n) + ', ' + this.z.toFixed(n) + ', ' + this.w.toFixed(n) + ']';
    }
}

Object.defineProperty(Quat, 'IDENTITY', {
    get: (function () {
        var identity = new Quat();
        return function () {
            return identity;
        };
    }())
});

Object.defineProperty(Quat, 'ZERO', {
    get: (function () {
        var zero = new Quat(0, 0, 0, 0);
        return function () {
            return zero;
        };
    }())
});

export { Quat };

[mvaligursky]

@kungfooman - I tried to integrate rollup-plugin-assemblyscript, to allow assembly script file (.as) to build into wasm module, and be usable inside the engine, but have issues with this. Any idea how this could be made to work? (my rollup / node build skiils are zero).

https://github.com/playcanvas/engine/commit/5c3a124f5ce0a37ef71479f90e15e979b9cfceea

[kungfooman]

@mvaligursky Thank you for trying! I am also trying around to see what would make most sense for easily/quickly testing all possible permutations in the example browser, and I am not sure if rollup-plugin-assemblyscript is the best way to go yet (e.g. a custom plugin for bundling .wasm into .js could be a better fit).

My example browser currently looks like this:

Because we have so many different builds and flavors now... JS/NORMAL, JS/PROFILER, X64/DEBUG or X32/PROFILER etc.

To exhaust every possible permutation the build step should create 6 .wasm integrated single-request .js files:

AS32 
AS64 
AS32 (DEBUG)
AS64 (DEBUG)
AS32 (DEBUG PROFILER)
AS64 (DEBUG PROFILER)

List could increase in future, when WebAssembly supports 16bit and/or 128bit floating points. 😅

And then another question, should we also differentiate between .wasm bundled/unbundled versions? Browsers are optimized to load/prepare the .wasm code straight as the data comes over the network (called Streaming Compilation), and bundling it into .js de-optimizes that path while increasing the .js parsing time and size (it may be just 10 KB and a few milliseconds extra parsing time or so).

All ideas are welcome, I am still fleshing this out for my PR