RangeError: offset is out of bounds - Githubissues

xenova / transformers.js

State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!

https://huggingface.co/docs/transformers.js

Apache License 2.0

11.04k stars 678 forks source link

RangeError: offset is out of bounds #806

Open jrieken opened 3 months ago

jrieken commented 3 months ago

System Info

"@xenova/transformers": "^2.17.2"
"onnxruntime-web": "1.18.0",

Environment/Platform

[X] Website/web-app
[ ] Browser extension
[ ] Server-side (e.g., Node.js, Deno, Bun)
[ ] Desktop app (e.g., Electron)
[ ] Other (e.g., VSCode extension)

Description

Receiving the RangeError when disabling quantization

Reproduction

embed the code snippet below in a webpage
🐛 you get the range error
set quantized to true
😄 no error

import * as transformers from '@xenova/transformers';

const pipe = await transformers.pipeline('feature-extraction', 'mixedbread-ai/mxbai-embed-large-v1', {
    quantized: false,
});

try {
    const output = await pipe("Hello", { pooling: 'cls' });
    console.log(output);
} catch (err) {
    console.error(err);
}

xenova commented 3 months ago

Hi there 👋 That is a WASM out-of-memory error, which makes sense since you only get the error when using the unquantized (fp32) version of the model that is ~1.34GB in size. If you would like to use the unquantized version, I would recommend you use it with the WebGPU execution provider, which you can try with the (experimental) v3 branch: https://github.com/xenova/transformers.js/pull/545. See the first comment for installation and usage instructions.