tvm-wasm: Build pure WebAssembly from DL model

This repository contains

• a script to compile a pre-trained model into a DL library in the pure WebAssembly format (no need for the emscripten layer)
• example Rust codes to use the generated library (e.g. WASI, waSCC, ...).

Install

• Clone repo

  git clone --recursive https://github.com/kazum/tvm-wasm

• Setup TVM

  • Build

  LLVM 9.0 or later is required.

  mkdir tvm/build
  cd tvm/build
  cmake -DUSE_LLVM=/usr/bin/llvm-config-9 ..
  make -j8
  cd -

  • Install Python packages

  pip install cython
  cd tvm/python; python setup.py install; cd -
  cd tvm/topi/python; python setup.py install; cd -
  pip install onnx

  See https://tvm.apache.org/docs/install/from_source.html for details.

• Setup Rust

  • Add wasm32 targets

  rustup target add wasm32-unknown-unknown
  rustup target add wasm32-wasi

  • Build a static library for tvm runtime.

  cd tvm/rust/runtime; cargo build --target wasm32-unknown-unknown --release; cd -

Build model

• Build DL library in the WebAssembly format.

  • Download model

  wget https://s3.amazonaws.com/onnx-model-zoo/resnet/resnet50v1/resnet50v1.onnx

  • Compile

  AR=llvm-ar-9 python build_lib.py -O3 resnet50v1.onnx

Example: WASI

• Run inference from command line

  • Build

    $ cd examples/wasi
    $ cargo build --release

  • Run inference

    $ wget -O cat.png https://github.com/dmlc/mxnet.js/blob/master/data/cat.png?raw=true
    $ wasmtime target/wasm32-wasi/release/tvm-wasi.wasm --dir . cat.png
    original image dimensions: (256, 256)
    resized image dimensions: (224, 224)
    input image belongs to the class `tabby, tabby cat`

Example: waSCC

• Run HTTP inference server

  • Build an actor to handle inference request

    $ cd examples/tvm-serving
    $ make release

  • Build a host with the wascc-host library

  Follow the tutorial in https://wascc.dev/tutorials/first-actor/create_host/

  • Send a inference request

    $ curl -s -T cat.png localhost:8081
    "tabby, tabby cat"