search

FSoft-AI4Code / AgileCoder

Incorporating Agile methodology into agents to create complex real-world softwares
MIT License
374 stars 52 forks source link

RateLimitIssue with 30k per minute on Azure OpenAI #15

Open hoangvnh24 opened 1 month ago

hoangvnh24 commented 1 month ago

I am running the demo code using Azure Open AI, tier pay as you go, the limit is 30k per minute. However, with such a large limit, my prompt was "Build a Cargo game in Python" The system keeps giving me RateLimitErrors. I believe that 30k per minute is quite large already. Not sure if the system is technically feasible if it cannot run with the above token rate limit.

minhngh commented 1 month ago

Hi hoangvnh24,

Thank you for your concern. This issue may stem from your Azure OpenAI key, which can be easily fixed by increasing the TPM. Also, you can check whether the key you're using is shared with anyone. Technically, our method will run seamlessly from phase to phase, which can lead to a noticeable number of requests within a short time.

hoangvnh24 commented 1 month ago

Hi there, what is the new TPM you are suggesting to request from Azure? Let's say 100k TPM? Let me know about this with more details if you can. I am just afraid that if we ask for more tokens, the designed system might be too expensive/inefficient to run?