search

Youssefbenhammouda / openai-ratelimiter

MIT License
8 stars 6 forks source link

Azure OpenAI Service Support? #3

Closed josephtutera closed 1 year ago

josephtutera commented 1 year ago

Hey! I am currently building with GPT-4 and our API endpoint uses the Azure OpenAI service. We need to implement a ratelimiter like the one you have here as we run many calls in parallel. Will this work with the Azure Endpoints?

Thanks!

Youssefbenhammouda commented 1 year ago

Hi, Thanks for checking the repo.

The library is separated from the openai library, so it must work. You just need to feed it the prompts and parameters you are using with a context manager.

But, I have not updated the library for months, I don't know if there are any modifications in the system.

I suggest you test it first.

Until then, I can't guarantee that the current rate limiting implementation is still correct.

Thanks.

josephtutera commented 1 year ago

I'll give it a try!

Thanks

josephtutera commented 1 year ago

This worked on Azure, just FYI