[Image Classification] Failed to execute 'dispatch' on 'MLContext': Invalid inputs

[ibelem]

Failed to execute 'dispatch' on 'MLContext' after landing https://github.com/microsoft/webnn-developer-preview/pull/45

Repro Steps A

1. Enable WebNN in browser
2. Visit https://microsoft.github.io/webnn-developer-preview/demos/image-classification/?provider=webnn&devicetype=gpu&model=mobilenet-v2&run=1
3. Click "Classify" button
4. It will show performance results, click "Classify" button again

Repro Steps B

1. Enable WebNN in browser
2. Visit https://microsoft.github.io/webnn-developer-preview/demos/image-classification/?provider=webnn&devicetype=gpu&model=mobilenet-v2&run=1
3. Click "Classify" button
4. It will show performance results, click "ResNet50" button
5. Click "Classify" button

Actual Result

Failed to execute 'dispatch' on 'MLContext': Invalid inputs: The context of MLGraph doesn't match the context of the MLTensor with name "pixel_values".

@egalli do you have any clue to fix this issue, thanks!

CC @Honry @fdwr @huningxin

Note

This issue didn't occur by using compute() in ORT Web before.

How to build the Transformers.js dists and apply for developer preview image classification demo

1. Clone https://github.com/xenova/transformers.js/tree/v3 // "onnxruntime-web": "1.20.0-dev.20241016-2b8fc5529b",
2. npm install
3. Apply the patch https://github.com/ibelem/transformers.js/blob/v3-webnn-2/0001-v3-perf-code.patch or follow the code change in https://github.com/ibelem/transformers.js/commit/7cb760f43a6b5e9c616e755b159dd57ab118d05d
4. npm run build
5. Copy the dists from /transformers.js/dist to webnn-developer-preview/assets/dist_transformers/dynamic-runs-1.19-dev/

[egalli]

It seems to be related to how we created a new MLContext per sessions. I have created a PR that adds a cache for MLContexts so that InferencesSessions created with the same parameters will use the same MLContext.

FYI, while testing change to the ORT, I noticed that transformers.js defaults to downloading the .wasm file from a CDN. If we want to use the version this repo, we'll need to set that variable to

transformers.env.backends.onnx.wasm.wasmPaths = "../../assets/dist_transformers/dynamic-runs-1.20-dev/";

[ibelem]

Thank you @egalli , I will re-build transformers.js dist once merging the https://github.com/microsoft/onnxruntime/pull/22510 , also set transformers.env.backends.onnx.wasm.wasmPaths.

[fdwr]

Thank you @egalli , I will re-build transformers.js dist once merging the microsoft/onnxruntime#22510 , also set transformers.env.backends.onnx.wasm.wasmPaths.

@ibelem - Tis merged.

[ibelem]

Hold the dists update since the https://github.com/microsoft/onnxruntime/pull/22278 and https://github.com/microsoft/onnxruntime/pull/22556 cause demos regression.