search

livekit / agents

Build real-time multimodal AI applications 🤖🎙️📹
https://docs.livekit.io/agents
Apache License 2.0
4k stars 412 forks source link

OpenAI timeouts are too low / aren't handled well #807

Closed jezell closed 1 month ago

jezell commented 1 month ago

The LLM client is configured with a 5 second read timeout. If the client times out (which it does very often with a short timeout), the stream is not resumed.

The error handling needs to be more graceful, but the timeout really shouldn't be 5 seconds by default.

davidzhao commented 1 month ago

that's a good point.. we set it on the lower side mostly because it would seem awkward in the flow of the conversation to be stuck for awhile.

but I understand your point, by default it probably should be higher.

martin-purplefish commented 1 month ago

I'm sure you guys are thinking about this, it'd be really nice to have some failover mechanisms - e.g. azure vs. openai.