No documentation when a server hits the rate-limit

[games647]

As I already posted over Twitter, there is no documentation how you display a rate-limit.

Previously I didn't even know that you use a 429 code and that this code in the JSON format. In order to detect this properly we need to know how a JSON would look like in this case. (i.e. What's the JSON key for that code)

I know this a temporary solution, but there should be at least a small documentation about it. I currently use it for one of my plugins where it first tries:

Only for new players a Name-> UUID check

1. Check the in-memory cache (expires 3 hours)
2. If fails, Check the local database
3. If fails, Check the Mojang servers
4. If fails because only of rate-limiting, check your API for the next 10 minutes instead of step 3.

It would be also good to know, how many requests we could make until we reach the rate-limit of your API.

I assume I never hit your limit, but I want to have fail-safe solution to properly handle the error. (like not asking your server again for the next x-minutes)

[nathantowell]

That solution would just leave us with an every increasing backlog of requests. Also: Our api is not rate limiting you... we are getting rate limited by Mojang.

[nathantowell]

Ref #40

[games647]

https://twitter.com/mcapi_status/status/738298880148803584

Due to extremely high usage by certain users, a temporary rate limit is being imposed for high volume users.

I only want get notified when your api got rate-limited and so the request failed.

[nathantowell]

We can't do that since we manually rate limit people in our Nginx configs... That has only happened to ~20 IP addresses. That notification basically means that we are limiting people who are effectively dos'ing us.

[njb-said]

We can't notify people, because we actually have no way of contacting you/the user. All we have is the IP and User agent which helps us narrow down what is sending the requests, the new api-key system I am working on will ease this as we will have a twitter account we could potentially contact (api keys are obtained by signing in with twitter)

We get notified if an ip sends an abnormal amount of traffic (very high) by the api server automatically and then we manually rate limit in nginx as @ImNatho said. It's not the most elegant solution but it's what we have had to do, as we were essentially being DOS'd. I am adding as many extra servers as financially viable.

To understand the scale of the problem as of the time I am writing this we are receiving a consistent 230 requests per second, which is happening 24/7. It is extremely hard for us to isolate who is essentially hogging the api, hence the solution we have implemented.

The new system won't be the "silver bullet" solution but it should reduce the load (or at least distribute it more evenly) on our servers and mean we can tailor the API to the needs of it's users. All I can do is apologise about this and hope you can bear with it until the new system is done.

[njb-said]

For reference, we currently return a JSON message like this: {"status": 429, "http_reason" : "429 Too Many Requests: The user has sent too many requests in a given amount of time. ", "error": "We have detected an unusual amount of requests from your IP address, and have had to block it temporarily to protect our users.", "notice" : "We are working on an api-key system for the future."}

A Retry-After header is also set with the value 15. (Retry after 15 seconds)

If you have any questions feel free to comment below.

[games647]

Thank you very much.

One last question: What happens if one of your servers hit the rate-limit against the mojang servers. Will we receive a "Unknown Username" as it's a unknown account or is there something different.

[njb-said]

@games647 Depending on the situation we either return "Unknown Username" on /v3/uuid queries or "Unable to reach Mojang API" for others.

[games647]

Great. Thanks a lot.

[njb-said]

As an update: I am about 70%-75% done with the new api server 👍