Closed Ludwig-Graef closed 2 months ago
No response
I need to be able to wait a certain amount of time before I get a connection timeout.
I've done this before: AsyncAzureOpenAI(timeout=httpx.Timeout(timeout=300.0, connect=20.0), **kwargs). See httpx.Timeout details here: https://www.python-httpx.org/advanced/timeouts/
AsyncAzureOpenAI(timeout=httpx.Timeout(timeout=300.0, connect=20.0), **kwargs)
Now I want to use it as a parameter in the litellm router. Is there any way to get similar behaviour?
Is your feature request related to a problem? Please describe.
No response
Describe the solution you'd like
I need to be able to wait a certain amount of time before I get a connection timeout.
I've done this before:
AsyncAzureOpenAI(timeout=httpx.Timeout(timeout=300.0, connect=20.0), **kwargs). See httpx.Timeout details here: https://www.python-httpx.org/advanced/timeouts/Now I want to use it as a parameter in the litellm router. Is there any way to get similar behaviour?
Describe alternatives you've considered
No response
Additional context
No response