AbanteAI / spice

accelerant
Apache License 2.0
4 stars 2 forks source link

Add retry support to catch_and_convert_errors #90

Closed mentatai[bot] closed 3 months ago

mentatai[bot] commented 4 months ago

This update introduces retry functionality to the catch_and_convert_errors context manager. The new parameters max_retries, base_delay, and max_delay have been added to the Spice client to configure the retry strategy. On encountering APIError or APIConnectionError, the client will now retry the operation with exponential backoff, starting from base_delay and capping at max_delay. The operation will be retried up to max_retries times before raising the exception.

Closes #89

jakethekoenig commented 4 months ago

I'm realizing what I asked for is an abuse of contextmanagers. Instead of doing it this way can you give spice a "call_llm" method which sets up the context manager and implement the retry logic? And change callers of the client in spice to go through this function?