[Bug][WASM] Cannot find function tvm.contrib.sort.argsort_nms in the imported modules or global registry

[AlexanderSerov]

We have a network which contained nms layer. I can build a module from this using relay.build and run inference using graph_runtime.GraphModule(lib"default") on host using python tvm interface flawlessly. But inference stuck with an error during inference when run inference of wasm file on node.js.

Steps to reproduce: 1) Compile wasm file using following code:

target = "llvm -mtriple=wasm32-unknown-unknown-wasm -system-lib"
if not tvm.runtime.enabled(target):
    raise RuntimeError("Target %s is not enbaled" % target)

mod, params = relay.frontend.from_mxnet(sym, shape=shape_dict, arg_params=arg_params, aux_params=aux_params)
func = mod["main"]
func = relay.Function(func.params,
                  func.body, None,
                  func.type_params, func.attrs)

with tvm.transform.PassContext(opt_level=3):
    graph_json, mod, params = relay.build(func, target=target, params=params)

2) Run wasm module using folowing node.js code:

const path = require("path");
const fs = require("fs");
const assert = require("assert");
const tvmjs = require("./dist");

const basePath = "/tvm_master/web/tvm_master";
const wasmPath = tvmjs.wasmPath();
const EmccWASI = require(path.join(wasmPath, "tvmjs_runtime.wasi.js"));
const wasmSource = fs.readFileSync(path.join(basePath, "bld210.wasm"));

const graphJson = fs.readFileSync(path.join(basePath,"builder210_relay.json"), "utf-8");
const paramsBinary = fs.readFileSync(path.join(basePath,"builder210_relay.params"));

let tvm = new tvmjs.Instance(new WebAssembly.Module(wasmSource), new EmccWASI());

// console.log(tvm.listGlobalFuncNames());
// console.log(tvm.getGlobalFunc('tvm.graph_runtime.create'));

ctx = tvm.cpu(0);

const syslib = tvm.systemLib();

// syslib.getFunction("fused_subtract_multiply_layout_transform");

const executor = tvm.createGraphExecutor(graphJson, syslib, ctx);
executor.loadParams(paramsBinary);

const inputData = tvm.empty([1, 3, 128, 128]);
const outputData = tvm.empty([1, 676, 16]);

const output = executor.getOutput(0);
executor.run();
ctx.sync();

function randomArray(length, max) {
  return Array.apply(null, Array(length)).map(function () {
    return Math.random() * max;
  });
}

// inputData.copyFrom(randomArray(49152, 255));
inputData.copyFrom(Array(49152).fill(125));

executor.setInput("data", inputData);
executor.run();
outputData.copyFrom(output);

3) Get an error:

Environment: TVM : main branch or v0.7 Environment: ubuntu 18.04, tlcpack docker image

Notes:

• Only reproduced with wasm module. Not reproduced on host
• All operators resolved successfully during tvm.createGraphExecutor in above node.js code. Means fused_vision_non_max_suppression successfully resolved in syslib. As i can understand nms rely on additional operators, i.e argsort_nms, which he can't resolve
• The error iccurs when executor.run(); , means runtime.
• Similar issues:

  • Meet an error when deploy nms on target
  • [VTA] A Workaround for Deploying Faster R-CNN on Target ext_dev(VTA and ARM CPU)

[tqchen]

Thanks for easking the question, please use https://discuss.tvm.apache.org/ for related questions.

In your particular case, it could due to the original runtime bundle does not include the argsort_nms function. Changing the file https://github.com/apache/tvm/blob/main/web/emcc/wasm_runtime.cc#L44 to include https://github.com/apache/tvm/blob/main/src/runtime/contrib/sort/sort.cc might resolve the problem

[AlexanderSerov]

@tqchen Thanks, that works. Should we open PR to add this to wasm_runtime.cc on permanent basic?

[tqchen]

Assuming this is a common usecase, a PR(with comment on why) would be helpful. Thank you @AlexanderSerov !