search

firebase / genkit

An open source framework for building AI-powered apps with familiar code-centric patterns. Genkit makes it easy to integrate, test, and deploy sophisticated AI features to Firebase or Google Cloud.
Apache License 2.0
607 stars 74 forks source link

GoLang Ollama plugin uses a fixed 30s timeout #719

Open lesv opened 1 month ago

lesv commented 1 month ago

Is your feature request related to a problem? Please describe. Yes - Running Ollama on my M2 Mac it times out.

time=2024-07-30T15:28:09.584-10:00 level=DEBUG msg="span start" name=ollama/llama3.1:70b
time=2024-07-30T15:28:39.592-10:00 level=DEBUG msg="span end" name=ollama/llama3.1:70b
time=2024-07-30T15:28:39.592-10:00 level=DEBUG msg=Action.Run name=0x22e13a0 output=(*ai.GenerateResponse)(nil) err="failed to send request: Post \"http://127.0.0.1:11434/api/generate\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
time=2024-07-30T15:28:39.601-10:00 level=DEBUG msg="span end" name=dev-run-action-wrapper
time=2024-07-30T15:28:39.602-10:00 level=ERROR msg="request end" reqID=9 err="failed to send request: Post \"http://127.0.0.1:11434/api/generate\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"

Describe the solution you'd like Either a longer timeout - 1000s or better a configurable timeout.

Describe alternatives you've considered It appears to be set here

Additional context

I'm really enjoying using this. Thanks.

lesv commented 1 month ago

If you'd be receptive to a PR, I'd propose modifying genkit/go/plugins/ollama/ollama.go: