Closed mentatai[bot] closed 3 months ago
I'm realizing what I asked for is an abuse of contextmanagers. Instead of doing it this way can you give spice a "call_llm" method which sets up the context manager and implement the retry logic? And change callers of the client in spice to go through this function?
This update introduces retry functionality to the
catch_and_convert_errors
context manager. The new parametersmax_retries
,base_delay
, andmax_delay
have been added to theSpice
client to configure the retry strategy. On encounteringAPIError
orAPIConnectionError
, the client will now retry the operation with exponential backoff, starting frombase_delay
and capping atmax_delay
. The operation will be retried up tomax_retries
times before raising the exception.Closes #89