Open Jeff-ZYX opened 6 months ago
Hi @Jeff-ZYX
We acknowledge and appreciate the information you brought to our attention. Currently, we do not have a built-in rate restriction capability in our architecture for API instances. We understand that controlling the rate of generation is crucial to prevent errors and ensure error-free operation for GPT completions. We thank you for recommending the addition of a programmable sleep function to efficiently regulate the generating rate. We value your input and will prioritize this feature request for future updates, even though we are unable to offer a quick fix.
Is your feature request related to a problem? Please describe. Current architecture doesn't seem to support API instances with inbuilt rate limits, causing errors in generation
Describe the solution you'd like A sleep function that can be configured to change rates of generation of GPT completions.