AbanteAI / spice

accelerant
Apache License 2.0
4 stars 2 forks source link

Replace validators/converters with retry_strategy #87

Closed jakethekoenig closed 4 months ago

jakethekoenig commented 4 months ago

Currently get_response supports converters and validators as ways to coerce model response and retry it it's bad. But we want to support more complicated behaviors such as:

To support these let's make get_response accept a retry_strategy argument which is of abstract class RetryStrategy which should have one function decide (name TBD, feel free to choose the name of you have a good idea) which accepts the call_args the number of the previous attempt (0 indexed) and the model output as text. It then returns a tuple, (behavior, next_call_args, result) where behavior is a new enum Behavior which can be either RETRY or RETURN. If it is set to Return the text and result are added to the spice response and returned. Otherwise we try again with next_call_args. It's up to the strategy to throw an exception or otherwise define a failure case after a certain number of iterations.

This will deprecate the current converter/validator/retries arguments. We should support them for now and if passed create a RetryStrategy called Default which reproduces the current behavior.

jakethekoenig commented 4 months ago

@mentatbot can you do this?

mentatai[bot] commented 4 months ago

I will start working on this issue