ricky0123 / vad

Voice activity detector (VAD) for the browser with a simple API
https://www.vad.ricky0123.com
Other
773 stars 120 forks source link

onnx not being found in react native expo metro bundler #130

Open Cornelivl opened 1 month ago

Cornelivl commented 1 month ago

Encountered an error while loading model file. Please make sure silero_vad.onnx, included with @ricky0123/vad-web, is available at the specified path: "right path" If need be, you can customize the model file location using the modelsURL option.

const workletURL = 'http://localhost:8081/assets/vad/vad.worklet.bundle.min.js'; const modelURL = 'http://localhost:8081/assets/vad/silero_vad.onnx';

const workletURL = 'http://localhost:8081/assets/vad/vad.worklet.bundle.min.js';

const modelURL = 'http://localhost:8081/assets/vad/silero_vad.onnx';

console.log('Worklet URL:', workletURL); console.log('Model URL:', modelURL);

const vad = useMicVAD({ startOnLoad: true, // Start manually onSpeechStart: () => { console.log('Speech started'); startStreaming(); }, onSpeechEnd: (audio) => { console.log('Speech ended'); stopStreaming(); // Do something with the audio data if needed }, workletURL, modelURL, });

Also tried using require and import

lc-spxl commented 3 weeks ago

Hi adding here that a similar problem also shows in vite / sveltekit bundles.

It seems ORT fails to load the actual location of ort-wasm-simd-threaded.mjs however it used to work until a couple of weeks ago.

Trying now combinations of onnxruntime-web and ricky0123/vad-web version to hopefully spot a regression.

From the log it seems ort loads env at http://localhost:8080/node_modules/vite/dist/client/env.mjs

But then tries to go up a folder and fails for ort-wasm-simd-threaded

GET http://localhost:8080/node_modules/.vite/deps/ort-wasm-simd-threaded.mjs?import net::ERR_ABORTED 504 (Gateway Timeout)

ERROR [AudioDetection] Failed to start VAD: no available backend found. ERR: [wasm] TypeError: Failed to fetch dynamically imported module: http://localhost:8080/node_modules/.vite/deps/ort-wasm-simd-threaded.mjs?import, [cpu] Error: previous call to 'initWasm()' failed.

Maybe related to https://github.com/microsoft/onnxruntime/issues/21785

lc-spxl commented 3 weeks ago

As a temporary fix, this seems to work in my case

          ort.env.wasm.wasmPaths =
            'https://unpkg.com/onnxruntime-web@dev/dist/';
Cornelivl commented 2 weeks ago

Thanks for the tip. In the end is tried to use the microsoft onnxruntime directly.

Both the web and react native.

However didn't work in the end. My guess for expo web is that the combination of react native on web causes the problem. So the libraries are inconsistent.