Spore-Community / ModBrowser

A tool for browsing and downloading Spore mods from GitHub
https://mods.sporecommunity.com/
MIT License
7 stars 1 forks source link

Practically non-functional behind a CGNAT internet connection #12

Closed Techjar closed 1 year ago

Techjar commented 2 years ago

This tool is using client-side unauthenticated requests to the GitHub API, which have a rate limit of 60 requests per hour per IP address. CGNAT shares an IP address with thousands of users doing who knows what, so chances are the rate limit will be exhausted very quickly every time it resets, rendering this tool essentially unusable.

The only solution is, unfortunately, to move the API requests to the server side and use GitHub's OAuth2 flow to obtain a token which can be sent in requests to the API. You'll likely also need to implement some sort of server-side caching to avoid hitting the authenticated rate limit. Re-reading the documentation, I think you can allow the user to "sign in" with their GitHub account, and grant the application an OAuth2 token unique to them which can be held in local storage. This way, the requests can remain client-side and are tied to each user rather than the whole server.

Here's a link to the relevant documentation: https://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate-limiting

Splitwirez commented 2 years ago

This "60 requests per hour per IP address" rate limit...do you know at what scope it applies? With respect to a given repo? With respect to all repos owned by a given GitHub user? With respect to..the whole of GitHub...?

Techjar commented 2 years ago

Ah sorry, I've added a link to the documentation. As I understand it applies globally to any kind of request, only discriminating by IP address, per this paragraph:

For unauthenticated requests, the rate limit allows for up to 60 requests per hour. Unauthenticated requests are associated with the originating IP address, and not the person making requests.

So the rate limit is likely being exhausted by various other apps that make client-side API requests in the same way as this one (I doubt that many people behind the same IP are using the Spore mod browser). I also updated the second paragraph of the top post after going over the docs again.

Kade-N commented 1 year ago

Fixed with 0d8dae6

A new proxy server was added that authenticates with GitHub to get a higher rate limit