webonnx / wonnx

A WebGPU-accelerated ONNX inference run-time written 100% in Rust, ready for native and the web
Other
1.65k stars 60 forks source link

Compile wonnx to WASM, usable in WebGPU-capable browsers #61

Closed pixelspark closed 2 years ago

pixelspark commented 2 years ago

Still very much a work in progress, but a simple test works!

pixelspark commented 2 years ago

@haixuanTao this works pretty well now (tested on Chrome Canary with WebGPU enabled). API looks like this at the moment:

<script type="module">
import init, { Session, Input } from "/target/pkg/wonnx.js";

async function fetchBytes(url) {
    const reply = await fetch(url);
    const blob = await reply.arrayBuffer();
    const arr = new Uint8Array(blob);
    return arr;
}

async function run() {
    try {
        const [modelBytes, initResult] = await Promise.all([fetchBytes("models/single_relu.onnx"), init()])
        const session = await Session.fromBytes(modelBytes);

        const input = new Input();
        input.insert("x", [13.0, -37.0]);
        const result = await session.run(input);
        console.log({result});
        session.free();
    }
    catch(e) {
        console.error(e, e.toString());
    }
}

run();
</script>
haixuanTao commented 2 years ago

Wow! Incredible feature! Just for your information, I have been able to run the model on Firefox Nightly.

I'll add reviews once you unmark draft from the PR.

pixelspark commented 2 years ago

@haixuanTao So I managed to get SqueezeNet running in the browser, using the webcam as input! https://pixelspark.nl/wp-content/public/squeeze/squeeze.html

image

haixuanTao commented 2 years ago

Wow! That is incredible!

Fantastic work!

It does not work from scratch on Firefox Nightly Linux, not sure why.

Both Camera and the inference. But it might be linked to the camera.

P.S: I actually haven't installed any camera. Could be the problem

P.S 2: With a Camera the camera stream works! :rofl: But the inference is always Tinca as if the result is always 0.

pixelspark commented 2 years ago

It does not work from scratch on Firefox Nightly Linux, not sure why.

Doesn't work here either in Firefox Nightly on Mac. I am seeing some warnings which might be related:

image

It works for me in Chrome Canary (on Mac). To make it work there I had to fix some issues in WGSL first (the Chrome WGSL compiler is a bit more strict than wgpu's), there might be other things that need to be fixed for Firefox.

haixuanTao commented 2 years ago

So, I know some of the tests are not working due to the git lfs limit. I feel like several small OSS project have migrated there static data to another provider like dropbox or else. I can open a discussion on that.

For me this PR is fine for merge.

pixelspark commented 2 years ago

So, I know some of the tests are not working due to the git lfs limit. I feel like several small OSS project have migrated there static data to another provider like dropbox or else. I can open a discussion on that.

This is a bit weird, as we are not using that much storage... not sure about the alternatives (Dropbox is likely traffic-limited as well?)

For me this PR is fine for merge.

:+1: will merge!