Open ggaabe opened 3 weeks ago
It might also help for me to add this is originating in onnx-runtime resolveBackendAndExecutionProviders
implementation in the InferenceSession
class. This might ultimately boil down to being a onnx-runtime issue though I haven't seen any issues raised in that repo around service workers.
it looks like this really might be an onnx-runtime issue and it's being worked on over at that end; might needs a version bump on transformers.js when that gets resolved
https://github.com/microsoft/onnxruntime/issues/20876#issuecomment-2143655319
System Info
MacOS 14.4.1, Chrome 125.
Environment/Platform
Description
I'm trying out the Phi3 WebGPU chat example based on transformers.js v3, but inside of the chrome extension example
But I keep getting this error, no matter what:
Which occurs here, after downloading the model:
If I remove the device param, it tries to use [wasm] as the backend, but this also fails.
Chrome recently fixed this issue and made the WebGPU API available to service workers..
Here is an example extension from the mlc-ai/web-llm package that implements WebGPU usage in service workers successfully: https://github.com/mlc-ai/web-llm/tree/main/examples/chrome-extension-webgpu-service-worker
Here is some further discussion on this new support from Google itself: https://groups.google.com/a/chromium.org/g/chromium-extensions/c/ZEcSLsjCw84/m/WkQa5LAHAQAJ
If I'm doing something wrong, would love some help here; Otherwise this might be an issue with onnx or some other part of how transformersjs uses it.
Reproduction
You can put this in the extension's background.js, mostly copied from the phi3 WebGPU-chat and extension examples: