At the moment, the DeepInfraClient has readonly fields about the retry params:
export class DeepInfraClient {
private axiosClient: AxiosInstance;
private readonly maxRetries: number = MAX_RETRIES;
private readonly initialBackoff: number = INITIAL_BACKOFF;
private readonly subsequentBackoff: number = SUBSEQUENT_BACKOFF;
...
}
Ideally these will be adjustable from the user. So maybe create a config interface with all relevant fields and pass it to the Model/Client constructors.
const model = new TextGeneration(MODEL_NAME, TOKEN, {maxRetries: 1});
i.e optional ClientConfig 3rd argument to overwrite the defaults.
On a related note backoff is normally defined with min, max and coef -- i.e starts from min, then min * coef, then min * coef * coef, etc, but never more than max.
At the moment, the
DeepInfraClient
has readonly fields about the retry params:Ideally these will be adjustable from the user. So maybe create a config interface with all relevant fields and pass it to the Model/Client constructors.
Then the Model class (assuming #17 goes through):
i.e optional ClientConfig 3rd argument to overwrite the defaults.
On a related note backoff is normally defined with
min
,max
andcoef
-- i.e starts frommin
, thenmin * coef
, thenmin * coef * coef
, etc, but never more thanmax
.