search

konstructio / kubefirst

The Kubefirst Open Source Platform
https://kubefirst.konstruct.io/docs
MIT License
1.79k stars 140 forks source link

API may get rate limited by GitHub when retrieving the catalog #1778

Open fharper opened 1 year ago

fharper commented 1 year ago

Which version of kubefirst are you using?

ALL

Which cloud provider?

None specific

Which installation type?

UI (Console app)

Which distributed Git provider?

None specific

What is the issue?

Right now, when we create a cluster using the UI, the gitops-catalog content is fetched, and saved in MongoDB. This is happening every time we create a new cluster using the UI, which mean that we can get rate limited by GitHub since we do not use any API token when fetching the repository content.

Happened to me while testing [2023-08-28T16:42:11-04:00] error reading gitops catalog apps at startup: error retrieving gitops catalog repository content: GET https://api.github.com/repos/fharper/gitops-catalog/contents/: 403 API rate limit exceeded for 74.58.149.38. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) [rate reset in 19m09s], but I could see this issue happening to our users when they create multiple clusters... or not, since any calls to GitHub without a token by the same IP count toward the limit.

Code of Conduct

fharper commented 1 year ago

This can be easy to implement for users creating clusters with GitHub as their Git provider, since we'll already have, and use their token, but it will need a bit more work, and probably requirement a GitHub token all the time even for folks using GitLab as their Git provider.