Handle rate limit errors correctly

sourcegraph / sourcegraph-public-snapshot

Code AI platform with Code Search & Cody

https://sourcegraph.com

Other

10.1k stars 1.27k forks source link

Closed mrnugget closed 1 year ago

mrnugget commented 1 year ago

Since #50925 we have rate limiting for the completions API. We need to make sure that the web client handles this correctly.

thenamankumar commented 1 year ago

@taylorsperry hey, what will be the best message to display to the user in case Cody hit rate limits?

taylorsperry commented 1 year ago

@mrnugget has a suggestion for the actual rate limits here and we should be aligned/have a decision after our sync on Wednesday.

For error handling, I'm inclined to not reinvent the wheel, but cc @danielmarquespt for input. The ChatGPT UI looks like this:

eseliger commented 1 year ago

@thenamankumar I have tried it locally just now to see if it is a problem and seems like we just hang and retry indefinitely.

The Cody VSCode client renders the rate limit error message in a banner correctly now.