Closed khaledsulayman closed 2 weeks ago
Here are the OpenAI library error types: https://help.openai.com/en/articles/6897213-openai-library-error-types-guidance
It seems like we may be able to specifically use APIError
, Timeout
, RateLimitError
, etc. to match the specific cases where we'd want to wait and retry, but might be worth further discussing which of these are relevant in this case.
In trying to work on the eval CI I noticed that the library doesn't raise any errors when there is no model being hosted at the requested port.
As I understand it, our current OpenAI error handling prints exceptions to stdout instead of failing because of some expected behavior with the API where it may be ok to fail temporarily while we keep retrying. However, currently, we are catching
openai.OpenAIError
, which is more general thanopen.APIConnectionError
, which is what we are actually seeing in this case (no model being served).I believe we can fix this one of two ways:
open.APIConnectionError
, we raise it. We could do this either immediately or after the max retries are completed.openai
exception type for this scenario but would need to do more digging to confirm if that's the only scenario in which we'd want to carry out this retry functionality.