Error: Unsupported model type: whisper

[lsiem]

When uploading an mp4 file I get the error: failed fetch.

The docker log output shows the following:

2024-01-05 09:53:44 -- Working The PisaSales user interface.mp4 --
2024-01-05 09:53:44 [Conversion Required] .mp4 file detected - converting to .wav
2024-01-05 09:53:45 [Conversion Processing]: 4KB converted
2024-01-05 09:53:46 [Conversion Processing]: 37376KB converted
2024-01-05 09:53:46 [Conversion Processing]: 54596KB converted
2024-01-05 09:53:46 [Conversion Complete]: File converted to .wav!
2024-01-05 09:58:45 fetch failed
2024-01-05 09:49:08 ┌─────────────────────────────────────────────────────────┐
2024-01-05 09:49:08 │  Update available 5.3.1 -> 5.7.1                        │
2024-01-05 09:49:08 │  Run the following to update                            │
2024-01-05 09:49:08 │    npm i --save-dev prisma@latest                       │
2024-01-05 09:49:08 │    npm i @prisma/client@latest                          │
2024-01-05 09:49:08 └─────────────────────────────────────────────────────────┘
2024-01-05 09:49:09 fatal: not a git repository (or any of the parent directories): .git
2024-01-05 09:49:09 getGitVersion Command failed: git rev-parse HEAD
2024-01-05 09:49:09 fatal: not a git repository (or any of the parent directories): .git
2024-01-05 09:49:09 
2024-01-05 10:01:27 Failed to load the native whisper model: Error: Unsupported model type: whisper
2024-01-05 10:01:27     at AutoModelForCTC.from_pretrained (file:///app/collector/node_modules/@xenova/transformers/src/models.js:4667:19)
2024-01-05 10:01:27     at async file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2677:33
2024-01-05 10:01:27 node:internal/process/promises:288
2024-01-05 10:01:27             triggerUncaughtException(err, true /* fromPromise */);
2024-01-05 10:01:27             ^
2024-01-05 10:01:27 
2024-01-05 10:01:27 Error: Unsupported model type: whisper
2024-01-05 10:01:27     at AutoModelForCTC.from_pretrained (file:///app/collector/node_modules/@xenova/transformers/src/models.js:4667:19)
2024-01-05 10:01:27     at async file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2677:33
2024-01-05 10:01:27 
2024-01-05 10:01:27 Node.js v18.19.0

I am running anythingllm with my Apple M1 Max running macOS 14.2.1 (23C71)

[timothycarambat]

Related: https://github.com/xenova/transformers.js/issues/314

[timothycarambat]

Can you shell into your container and see if there are any files in /app/server/storage/models/Xenova/whisper-small. The model should only download when needed for the first time and be loaded from the disk on subsequent runs. However, if for whatever reason HuggingFace blocks the download, then the model would fail to download and thus, inference would fail

I do not have an M-series chip so I am unable to replicate this issue currently. We have the "patched" version of transformers running already so that should be resolved.

[shatfield4]

I get a similar fetch failed error in my docker logs as well on an M1 Pro running macOS 12.4

2024-01-05 09:45:41 -- Working jfk.wav --
2024-01-05 09:45:41 [INFO] The native whisper model has never been run and will be downloaded right now. Subsequent runs will be faster. (~250MB)
2024-01-05 09:45:41 
2024-01-05 09:45:41 
2024-01-05 09:46:17 Failed to load the native whisper model: TypeError: fetch failed
2024-01-05 09:46:17     at Object.fetch (node:internal/deps/undici/undici:11730:11)
2024-01-05 09:46:17     at async getModelFile (file:///app/collector/node_modules/@xenova/transformers/src/utils/hub.js:471:24)
2024-01-05 09:46:17     at async getModelJSON (file:///app/collector/node_modules/@xenova/transformers/src/utils/hub.js:575:18)
2024-01-05 09:46:17     at async AutoProcessor.from_pretrained (file:///app/collector/node_modules/@xenova/transformers/src/processors.js:1752:44)
2024-01-05 09:46:17     at async Promise.all (index 2)
2024-01-05 09:46:17     at async loadItems (file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2694:5)
2024-01-05 09:46:17     at async pipeline (file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2640:19)
2024-01-05 09:46:17     at async LocalWhisper.client (/app/collector/utils/WhisperProviders/localWhisper.js:34:14) {
2024-01-05 09:46:17   cause: SocketError: other side closed
2024-01-05 09:46:17       at TLSSocket.onSocketEnd (node:internal/deps/undici/undici:8280:26)
2024-01-05 09:46:17       at TLSSocket.emit (node:events:529:35)
2024-01-05 09:46:17       at endReadableNT (node:internal/streams/readable:1400:12)
2024-01-05 09:46:17       at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
2024-01-05 09:46:17     code: 'UND_ERR_SOCKET',
2024-01-05 09:46:17     socket: {
2024-01-05 09:46:17       localAddress: '172.17.0.2',
2024-01-05 09:46:17       localPort: 40518,
2024-01-05 09:46:17       remoteAddress: undefined,
2024-01-05 09:46:17       remotePort: undefined,
2024-01-05 09:46:17       remoteFamily: undefined,
2024-01-05 09:46:17       timeout: undefined,
2024-01-05 09:46:17       bytesWritten: 263,
2024-01-05 09:46:17       bytesRead: 0
2024-01-05 09:46:17     }
2024-01-05 09:46:17   }
2024-01-05 09:46:17 }
2024-01-05 09:46:17 node:internal/process/promises:288
2024-01-05 09:46:17             triggerUncaughtException(err, true /* fromPromise */);
2024-01-05 09:46:17             ^
2024-01-05 09:46:17 
2024-01-05 09:46:17 TypeError: fetch failed
2024-01-05 09:46:17     at Object.fetch (node:internal/deps/undici/undici:11730:11)
2024-01-05 09:46:17     at async getModelFile (file:///app/collector/node_modules/@xenova/transformers/src/utils/hub.js:471:24)
2024-01-05 09:46:17     at async getModelJSON (file:///app/collector/node_modules/@xenova/transformers/src/utils/hub.js:575:18)
2024-01-05 09:46:17     at async AutoProcessor.from_pretrained (file:///app/collector/node_modules/@xenova/transformers/src/processors.js:1752:44)
2024-01-05 09:46:17     at async Promise.all (index 2)
2024-01-05 09:46:17     at async loadItems (file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2694:5)
2024-01-05 09:46:17     at async pipeline (file:///app/collector/node_modules/@xenova/transformers/src/pipelines.js:2640:19)
2024-01-05 09:46:17     at async LocalWhisper.client (/app/collector/utils/WhisperProviders/localWhisper.js:34:14) {
2024-01-05 09:46:17   cause: SocketError: other side closed
2024-01-05 09:46:17       at TLSSocket.onSocketEnd (node:internal/deps/undici/undici:8280:26)
2024-01-05 09:46:17       at TLSSocket.emit (node:events:529:35)
2024-01-05 09:46:17       at endReadableNT (node:internal/streams/readable:1400:12)
2024-01-05 09:46:17       at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
2024-01-05 09:46:17     code: 'UND_ERR_SOCKET',
2024-01-05 09:46:17     socket: {
2024-01-05 09:46:17       localAddress: '172.17.0.2',
2024-01-05 09:46:17       localPort: 40518,
2024-01-05 09:46:17       remoteAddress: undefined,
2024-01-05 09:46:17       remotePort: undefined,
2024-01-05 09:46:17       remoteFamily: undefined,
2024-01-05 09:46:17       timeout: undefined,
2024-01-05 09:46:17       bytesWritten: 263,
2024-01-05 09:46:17       bytesRead: 0
2024-01-05 09:46:17     }
2024-01-05 09:46:17   }
2024-01-05 09:46:17 }
2024-01-05 09:46:17 
2024-01-05 09:46:17 Node.js v18.19.0

[timothycarambat]

Okay, here is the root cause.

On AMD machines, Docker runs AnythingLLM in an x86 emulator on the container. This emulator overhead causes ONNX models to run very slowly if at all. This slowness also leads to timeouts on the HTTPS call to HF when downloading the models needed to embed,whisper, etc.

The fix for this is to publish an AMD64 image for AnythingLLM so that AMD machines can download an appropriate image and not run in a compatibility mode.

• [ ] Implement arm64 build and publish for anythingllm docker image

This support causes issues with lanceDB, who require Rust binaries for it to run.

• [ ] We need to bump LanceDB to a version that publishes support for linux_arm64 binaries so that the non-emulated docker container can use LanceDB.
• [ ] puppeteer does not publish an arm86 compatible chromium with it - so we need to patch this as well.