Random API Key reset

[Picsou993]

One of my API Key has randomly reseted, this is the one of the playername JamiePotterLive (UUID 2f0f1190f9774e2185b00a28bc201763). The key has reseted between 5:50am and 6:50am ET as the execution of my script at 5:50am was good but the one at 6:50am returned {"success":false,"cause":"Invalid API key"}.

The issue is not related to a compromised account, mute or ban because as we can see on my recent servers, I haven't logged on and I'm not muted: https://i.imgur.com/2TGgqS0.png

The key before the reset ends with 7147.

[Strengthless]

I'm glad to hear that I'm not alone - I've been having this issue since 2 days ago, it keeps resetting every once in a while.

Autotip also has been having this issue as well (@Sk1er).

[ConnorLinfoot]

If your API key was reset it could be the case that the key was either compromised or being abused in some way, such as multiple keys being used.

[robere2]

If your API key was reset it could be the case that the key was either compromised or being abused in some way, such as multiple keys being used.

Since when has this been a rule? Can we at least be tactful and publish guidelines before breaking applications for hundreds of thousands of players due to randomly invalidating API keys and not giving information as to why? ~It has previously been encouraged to use multiple API keys if you need more than the 120 requests.~

[robere2]

Attempting to find a source for my "encouraged" claim, unfortunately can't find one... Might've been said in passing during a private convo with an admin, or I'm misremembering. My comment was written in the heat of the moment, should've waited a few minutes and done some research. Regardless, there are users who use multiple keys or have claimed they have been granted above the 120-request limit. If you're going to start enforcing this new rule then perhaps there should be a way to request a higher limit.

[mdashlw]

But why would you care about it? Key privacy is up to user, if they think it's compromised, they can freely regenerate it via /api new. I don't think you should try to detect compromised keys (how is it even possible?) with some weird checks that don't even fully work or trigger a false alarm. Invalidating api keys is quite a breaking "change", it breaks stuff, it should not be a thing if your code thinks it somehow is compromised.

key was either compromised

Since when is this not allowed? It always has been. I won't find an official statement, but you won't find one about it being disallowed either. If you want to disallow multiple api keys, then allow users somehow upgrade their key to more limit? 120 req/m is pretty small i would say, if you have a semi-big app, you will exceed it and will have to use multiple keys, because there's no other solution provided by hypixel at all.

being abused in some way, such as multiple keys being used.

[mew]

I shouldn't have to worry about keys belonging to accounts that I purchased myself from Microsoft being invalidated because some algorithm you guys wrote thinks the keys were "compromised".

[Dance-Dog]

But why would you care about it? Key privacy is up to user, if they think it's compromised, they can freely regenerate it via /api new. I don't think you should try to detect compromised keys (how is it even possible?) with some weird checks that don't even fully work or trigger a false alarm.

Pretty sure its because someone along the line decided it would be a great idea to use API keys to link forum accounts (tbh, the worst that could happen is a troll changing your guild MOTD) instead of generating a separate key for that. Even if your key is "compromised" though, there's no reason resetting it isn't on us.

[WolfDWyc]

Is there any update on why this happened?

[ConnorLinfoot]

It has always been stated that abuse of the API could lead to keys being banned. To my knowledge we have never said that users may use multiple keys, that would defeat the whole purpose of the limit.

We are seeing a lot of abuse from users where up to hundreds of API keys are being used for single applications. Because of this, we had to implement detections and reset keys that were being abused to bypass the limit. If users require an API key limit they can contact us via https://support.hypixel.net with their use case and why they need a higher limit.

We will add this to the Readme to make sure new users of the API are aware of this.

[robere2]

It has always been stated that abuse of the API could lead to keys being banned. To my knowledge we have never said that users may use multiple keys, that would defeat the whole purpose of the limit.

I think the confusion arises from the fact that "abuse" is super subjective. If any other clarifications about what "abuse" means, such as examples, could be provided then that would help avoid issues in the future.

[WolfDWyc]

We are seeing a lot of abuse from users where up to hundreds of API keys are being used for single applications. Because of this, we had to implement detections and reset keys that were being abused to bypass the limit. If users require an API key limit they can contact us via https://support.hypixel.net with their use case and why they need a higher limit.

Do you have an estimate on what are the "requirements" to get that higher limit?

Obviously, if you gave it to everyone, that would defeat the purpose, so who are you gonna give it to? What purpose should a use-case have to receive a limit increase?

Even if you can't answer all of these questions, please provide more information about API Limits and abuse in the readme, as this was the first and only place I ever saw this mentioned.

[mdashlw]

I guess it's not getting reverted. How about improving your algorithmes? They don't work well, people are complaining about legit api keys being invalidated for no reason, just because it thought they were abused in some way. Also what about the compromised part of the reason keys are invalidated? Is it gonna remain? The only reason keys are sensitive is because they are used for account linking. If it was changed, there would be no reason to implement weird checks to detect compromised keys which will partly close this issue.

It also was never said that users may not use multiple keys, it was never enforced

To my knowledge we have never said that users may use multiple keys,

[ConnorLinfoot]

Do you have an estimate on what are the "requirements" to get that higher limit?

It's hard to give a specific list cause of the large array of applications and services that can use the API. We usually look at what your tool does and if it implements caching correctly and make a judgment per request.

How about improving your algorithmes?

For the legit keys that may have been reset, we recently eased the method that we use to detect abuse with multiple keys so it should no longer be a problem.

Also what about the compromised part of the reason keys are invalidated?

For compromised accounts that were more of a generic warning, we do nothing to target them specifically but a high amount of keys abusing the API were found to belong to accounts that were also compromised.

[WolfDWyc]

It's hard to give a specific list cause of the large array of applications and services that can use the API. We usually look at what your tool does and if it implements caching correctly and make a judgment per request.

First of all, thank you for the quick answer, it's really appreciated. I'm just wondering how I could get my limit increased, I don't really want a specific list, but just a vague example/estimate on a project that would get accepted

Thanks in advance!

[ConnorLinfoot]

I can't really give a perfect example, but as something to go off if you're hitting the 120 requests per minute and you're doing all you can in regards to caching data for a reasonable amount of time then it would be worth requesting an increase.

[mew]

I think if you're going to go this route then the least you can do is provide clear examples of software that would be permitted an increased limit and go into details on what that increased limit would look like.

[Kuba314]

@ConnorLinfoot I have a suggestion. Most of the time I find that I'd really appreciate retrieving a whole guild. With your current system, it'd be up to 126 requests. Why not just make it a single request to retrieve a full guild with 1 SQL command(supposing both guilds and players are in the same database). Of course it'd take a few seconds worth of time to retrieve from your database, but I think it'd be way faster than making 126 distinct requests

[puppy0cam]

Hypixel does not use an SQL database, they use MongoDB.

[mdashlw]

Actually, requesting guild members is like the only reason I could exceed one api key limit. It would be very nice if something could be done with this, however I don't have any suggestions, returning all player objects in /guild would make the response big as hell.

[Kuba314]

@puppy0cam Is it possible to only select only certain columns in Mongo? Or would it be computationally intensive to only send certain columns? (I'm not a MongoDB expert)

If so, could the response be stringed together using this $lookup thing?

I found with some testing, that each player could be only 7000 bytes long if unnecessary attributes were omitted (e.g. levelingReward_109). Sadly, the biggest portion of the response is stats so they would've to be omitted as well

[Dance-Dog]

@Kuba314 I'm pretty sure this could be done using $lookup in combo with $project. In fact, their website API (different from PublicAPI) is a perfect example of a projection that includes the essential fields needed to display a basic member list, like the one on Plancke.

Example (from https://hypixel.net/api/players/hypixel):

{
  "success": true,
  "_id": "516398d00cf273d9c97152c3",
  "playername": "hypixel",
  "displayname": "hypixel",
  "uuid": "f7c77d999f154a66a87dc4a51ef30d19",
  "rank": "ADMIN",
  "lastLogin": 1589161847767,
  "lastLogout": 1589162197698,
  "networkExp": 65817686,
  "exactLevel": 226,
  "packageRank": "NONE",
  "newPackageRank": "MVP_PLUS",
  "monthlyPackageRank": "NONE"
}

It's not enough information to construct a whole player profile, but it's insanely useful information if you want to display a basic member list without querying every player:

And if they wanted to improve performance, caching the projection result might help.

[mdashlw]

Or if it's too much to add basic info for every guild member, rate limit could be updated from 120req/m to 240req/2m or even 600req/5m, basically the same but would allow to make more requests at once to fetch all 125 members and not get throttled, also mojang uses rate limit like this 600req/10m

[mdashlw]

Everything was fine until yesterday. I'm getting my api keys on different applications randomly invalidated for no reason again. Can we get an update on this?

[Kuba314]

@mdashlw are you sure that you're not throttling your keys? Are you strictly under the limit? I finally found an error in my script and fixed it so it doesn't throttle and since then all my keys are fine and valid.

[mdashlw]

@mdashlw are you sure that you're not throttling your keys? Are you strictly under the limit? I finally found an error in my script and fixed it so it doesn't throttle and since then all my keys are fine and valid.

No. When it throttles, it retries the request after static amount of time (since hypixel doesn't have proper rate limits) and notifies me too. The key was randomly invalidated for no obvious reason. And as I said everything was just fine until yesterday.

[Kuba314]

@mdashlw If I were you, I'd just try not to throttle. It took me some times to figure out, but it's definitely possible. The queriesInPastMin resets every minute. Why don't you then restrict your script to send only 120 reqs / minute?

[TheNullicorn]

Some insight into how "abuse" is detected would be very helpful. Is repetitive throttling considered abuse? Is using the same key in different applications considered abuse? Is using different API keys for different applications considered abuse? We don't know if the detection is faulty or if we are breaking some built-in rules if we don't know what those rules are.

[mdashlw]

The queriesInPastMin resets every minute.

There's no way not to throttle. queriesInPastMin is reset every minute relative to the start time of the api service that we can't know. That's the main problem about hypixel api rate limiting.

Why don't you then restrict your script to send only 120 reqs / minute?

I don't have any script, I have a big discord bot.

I fully agree with @TheNullicorn message, there are no any clear statements about what is abuse. You cannot invalidate a key that potentially violates the rules if no one knows the rules. And to add to it, is asking the user for their personal api key in a centralized application for performing heavy tasks intended for them considered an abuse? I have seen it many times, including from hypixel staff.

[Kuba314]

@mdashlw It doesn't matter whether you know when it resets. You could have a list of all your previous times of requests and before every request check if the number of requests in the past minute is less than 120. I don't see why this wouldn't work. I'm using it in my discord bot also. I can send you a code snippet if you're also using python.

[mdashlw]

@mdashlw It doesn't matter whether you know when it resets. You could have a list of all your previous times of requests and before every request check if the number of requests in the past minute is less than 120. I don't see why this wouldn't work. I'm using it in my discord bot also. I can send you a code snippet if you're also using python.

If I understand you correctly, then if you send 120 requests at once, and the next second api resets it, then your snippet will think that it is still 120 requests and will not make a request, even though if'd work. Either way I don't want to do anything tricky like this about rate limits on my side. It's hypixel's goal to provide proper rate limits.

[TheNullicorn]

@mdashlw It doesn't matter whether you know when it resets. You could have a list of all your previous times of requests and before every request check if the number of requests in the past minute is less than 120. I don't see why this wouldn't work. I'm using it in my discord bot also. I can send you a code snippet if you're also using python.

If I understand you correctly, then if you send 120 requests at once, and the next second api resets it, then your snippet will think that it is still 120 requests and will not make a request, even though if'd work. Either way I don't want to do anything tricky like this about rate limits on my side.

Not needed, but would still be nice to have more information about ratelimits. Looks like #248 addresses this.

[ConnorLinfoot]

Please see our last response on this topic here.