HuggingFace failure

[sroussey]

What version of Bun is running?

1.0.20

What platform is your computer?

Darwin 23.2.0 arm64 arm

What steps can reproduce the bug?

import { pipeline } from "@xenova/transformers";

const generateEmbedding = await pipeline(
  "feature-extraction",
  "Supabase/gte-small"
);

console.log(generateEmbedding);

What is the expected behavior?

Something not an error...

What do you see instead?

TypeError: BuildMessage: ModuleNotFound resolving "undefined" (entry point)
      at new Worker (node:worker_threads:124:27)
      at yc (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:6:7890)
      at Cc (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:6:7945)
      at lt (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:6:5687)
      at Et (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:6:9720)
      at WASM  ([wasm code])
      at WASM  ([wasm code])
      at WASM  ([wasm code])
      at WASM  ([wasm code])
❯ bun -v
1.0.20
❯ uname -mprs
Darwin 23.2.0 arm64 arm

Additional information

No response

[sroussey]

Slightly more detail

22 |         __classPrivateFieldSet(this, _OnnxruntimeSessionHandler_inferenceSession, new binding_1.binding.InferenceSession(), "f");
23 |         if (typeof pathOrBuffer === 'string') {
24 |             __classPrivateFieldGet(this, _OnnxruntimeSessionHandler_inferenceSession, "f").loadModel(pathOrBuffer, options);
25 |         }
26 |         else {
27 |             __classPrivateFieldGet(this, _OnnxruntimeSessionHandler_inferenceSession, "f").loadModel(pathOrBuffer.buffer, pathOrBuffer.byteOffset, pathOrBuffer.byteLength, options);
                 ^
error: Error
      at new OnnxruntimeSessionHandler (/Users/steve/Code/elmers/node_modules/onnxruntime-node/dist/backend.js:27:13)
      at /Users/steve/Code/elmers/node_modules/onnxruntime-node/dist/backend.js:64:29
      at processTicksAndRejections (:61:77)

Something went wrong during model construction (most likely a missing operation). Using `wasm` as a fallback. 
✖ DATA [FAILED: BuildMessage: ModuleNotFound resolving "undefined" (entry point)]
1 | /*!
2 | * ONNX Runtime Web v1.14.0
3 | * Copyright (c) Microsoft Corporation. All rights reserved.
4 | * Licensed under the MIT License.
5 | */

[[[[[ lots of minified code]]]]]

TypeError: BuildMessage: ModuleNotFound resolving "undefined" (entry point)
      at new Worker (node:worker_threads:124:27)
      at yc (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:6:7890)
      at Cc (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:6:7945)
      at lt (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:6:5687)
      at Et (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:6:9720)
      at WASM  ([wasm code])
      at WASM  ([wasm code])
      at WASM  ([wasm code])
      at WASM  ([wasm code])

[sroussey]

When debugging, before the line is question is even run, I have a breakpoint and use the debugger to look at things.

I type _OnnxruntimeSessionHandler_inferenceSession into the debug console and get this:

I click on the weakmap to open:

I then click on OnnxruntimeSessionHandler:InferenceSession and get this:

Just looking at the properties segfaults the process (and causes the WebSocket closed message).

[sroussey]

Digging around this seems to be the culprit, but it makes no sense to me:

1 | export default "native";

Exception has occurred.
TypeError: BuildMessage: ModuleNotFound resolving "undefined" (entry point)
    at Worker (native)
    at new Worker (node:worker_threads:124:14)
    at yc (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:252:32)
    at Cc (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:254:45)
    at lt (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:196:24)
    at Et (/Users/steve/Code/elmers/node_modules/onnxruntime-web/dist/ort-web.node.js:332:147)
    at <?>.wasm-function[10917] (native)
    at <?>.wasm-function[1580] (native)
    at <?>.wasm-function[2786] (native)
    at <?>.wasm-function[5903] (native)
    ^
error: Session is not initialized.

[sroussey]

I went to the onnx which is where the errors are happening. I found a simple example that uses node:

https://github.com/microsoft/onnxruntime-inference-examples/blob/main/js/api-usage_inference-session/inference-session-create.js

It works fine.

However...

When I change it to require onnxruntime-web instead of onnxruntime-web bun gives me almost the same error!

Weird.

I get this error:

failed to create inference session: TypeError: BuildMessage: ModuleNotFound resolving "undefined" (entry point)

Run the same thing in node and I get:

failed to create inference session: TypeError [ERR_WORKER_PATH]: The worker script or module filename must be an absolute path or a relative path starting with './' or '../'. Received "blob:nodedata:be2e77f3-2beb-4779-845a-4ff54d158079"

That is obviously a different error. And it has to deal with Workers.

[sroussey]

There is an issue for the above:

https://github.com/microsoft/onnxruntime/issues/18933

[sroussey]

Note here: "It's a known issue that onnxruntime-web multi-thread not loading in node.js."

https://github.com/microsoft/onnxruntime/issues/17274#issuecomment-1691050653

"@xenova/transformers" is surely using the web version and not the node version and thus the cause of this issue.

[Edit: this is false, it determines which to use and correctly does so.]

[sroussey]

OK, it was dealing with workers and wasm as a backup if the native code failed. ONNX on node with WASM can only use one thread, so I have a one line fix for "@xenova/transformers". With that fix, it appears to work just fine but it is much slower than it should be.

So the bug is a failure in the native code that works fine with node but fails with bun.

Things run slower as it runs the native code first, loads the model and bails, then loads the wasm, loads the model, and succeeds. On a browser it can use multiple threads. So the browser version is faster than bun and the node version is faster than bun unless bun can run the native module (assuming using my fix).

[sroussey]

I went through building onnx runtime and the node build both in debug and found everything to work and not error.

Long story short, onnx has already fixed the issue but "xenova/transformers" is using an old version.

[sroussey]

Main fix: https://github.com/xenova/transformers.js/pull/488

[Nnadivictory25]

heyy man @sroussey what worked for you ? I am currently getting this error in bun trying to run embeddings , please I really need help , this has me stuck for a few days now

[sroussey]

Use my version temporarily

@sroussey/transformers

[MariuzM]

@sroussey thank you i had same issue https://github.com/xenova/transformers.js/issues/558 your version fixed

[sroussey]

It's just temporary until fixed upstream.

[sroussey]

But glad you have it working!

[TimPietrusky]

As mentioned in https://github.com/xenova/transformers.js/pull/489 it should be solved in the v3 branch of transformers.js!