How do you rate limit calls to llms configured through langchaingo?

tmc / langchaingo

LangChain for Go, the easiest way to write LLM-based programs in Go

https://tmc.github.io/langchaingo/

MIT License

3.73k stars 518 forks source link

How do you rate limit calls to llms configured through langchaingo? #940

Open vivekrao1985 opened 2 days ago

vivekrao1985 commented 2 days ago

We're currently using langchaingo to connect to a gpt-4o model deployed on Azure. Is there any documentation or examples I can refer to on how to set token rate limits?

Thanks!