search

hoarder-app / hoarder

A self-hostable bookmark-everything app (links, notes and images) with AI-based automatic tagging and full text search
https://hoarder.app
GNU Affero General Public License v3.0
6.34k stars 226 forks source link

Custom fetch wrapper #656

Open sbarbett opened 1 week ago

sbarbett commented 1 week ago

I added a custom wrapper for fetch to control the headersTimeout value. The default value is 5 minutes, and slower instances of ollama can take longer than that, so the fetch fails. The following environment variable allows you to up it (the default is still 5 min).

services:
  web:
    environment:
      - INFERENCE_FETCH_TIMEOUT_SEC=3600  # 1 hour fetch timeout

There's more info on this in issue 628. I built this locally and tested it against my ollama instance. With the default timeout:

Nov 14 02:35:33 llama ollama[3733]: [GIN] 2024/11/14 - 02:35:33 | 200 |          5m0s |        10.0.0.2 | POST     "/api/chat"

Hard stop at 5 min. The inference job fails.

workers-1      | 2024-11-14T02:35:33.346Z error: [inference][10] inference job failed: TypeError: fetch failed
workers-1      | TypeError: fetch failed
workers-1      |     at node:internal/deps/undici/undici:13392:13
workers-1      |     at async post (/app/node_modules/ollama/dist/shared/ollama.9c897541.cjs:114:20)
workers-1      |     at async Ollama.processStreamableRequest (/app/node_modules/ollama/dist/shared/ollama.9c897541.cjs:232:25)
workers-1      |     at async OllamaInferenceClient.runModel (/app/packages/shared/inference.ts:2:3386)
workers-1      |     at async OllamaInferenceClient.inferFromText (/app/packages/shared/inference.ts:2:4136)
workers-1      |     at async inferTagsFromText (/app/apps/workers/openaiWorker.ts:6:3135)
workers-1      |     at async inferTags (/app/apps/workers/openaiWorker.ts:6:3370)
workers-1      |     at async Object.runOpenAI [as run] (/app/apps/workers/openaiWorker.ts:6:6863)
workers-1      |     at async Runner.runOnce (/app/node_modules/liteque/dist/runner.js:2:2578)

Here is the same job after upping the fetch timeout.

Nov 14 02:52:23 llama ollama[3733]: [GIN] 2024/11/14 - 02:52:23 | 200 |         7m31s |        10.0.0.2 | POST     "/api/chat"
workers-1      | 2024-11-14T02:55:55.614Z info: [search][13] Completed successfully

One other small tweak: The docker-compose.dev.yml file needed the env file declared under the worker service.

Thanks for making this useful app.