search

discordjs / discord.js

A powerful JavaScript library for interacting with the Discord API
https://discord.js.org
Apache License 2.0
25.38k stars 3.97k forks source link

client.login has no ratelimit backoff #3890

Closed MattIPv4 closed 2 years ago

MattIPv4 commented 4 years ago

Please describe the problem you are having in as much detail as possible:

Well, I got one of my bots ratelimited on /gateway/bot it seems. Watching the debug event, it seems to be continually returning 429s on /gateway/bot whilst trying to log in.

It seems to be getting these at a very consistent rate, which would suggest to me the login method has no logic to consider a ratelimited response and back-off etc.?

Include a reproducible code sample here, if possible:

// This requires you be ratelimited
const client = new Client();
client.on('debug', console.log);
await client.login(token);
// Observe cotinual ratelimit logs

Logs from a slightly more fleshed out bot script:

 MNGR | 20/3/3 22:9:33  INFO  Launched shard 0
 S0 | 20/3/3 22:9:34  INFO  Logging in...
 S0 | 20/3/3 22:9:34  DEBUG  Provided token: [SNIP]
 S0 | 20/3/3 22:9:34  DEBUG  Preparing to connect to the gateway...
 S0 | 20/3/3 22:9:34  DEBUG  429 hit on route /gateway/bot
 S0 | 20/3/3 22:9:34  DEBUG  429 hit on route /gateway/bot
 S0 | 20/3/3 22:9:35  DEBUG  429 hit on route /gateway/bot
 S0 | 20/3/3 22:9:35  DEBUG  429 hit on route /gateway/bot
 S0 | 20/3/3 22:9:35  DEBUG  429 hit on route /gateway/bot
 S0 | 20/3/3 22:9:35  DEBUG  429 hit on route /gateway/bot
 S0 | 20/3/3 22:9:36  DEBUG  429 hit on route /gateway/bot
 S0 | 20/3/3 22:9:36  DEBUG  429 hit on route /gateway/bot
 S0 | 20/3/3 22:9:36  DEBUG  429 hit on route /gateway/bot
 S0 | 20/3/3 22:9:37  DEBUG  429 hit on route /gateway/bot
 S0 | 20/3/3 22:9:37  DEBUG  429 hit on route /gateway/bot
 S0 | 20/3/3 22:9:37  DEBUG  429 hit on route /gateway/bot
 S0 | 20/3/3 22:9:37  DEBUG  429 hit on route /gateway/bot
^C

Further details:

MattIPv4 commented 4 years ago

Possibly related to #3560

gc commented 4 years ago

Getting this once every week or two, is causing me problems.

Koyamie commented 4 years ago

Getting this once every week or two, is causing me problems.

Same for me, bumping but not having any updates.

PhoenixShay commented 4 years ago

Having the same problem on a ~101k guild bot, getting IPbanned as a result of this.

mykola-bihun commented 4 years ago

Causing consistent problems for a bot that I commonly use.

wilyt1 commented 4 years ago

I am being affected as well.

HypheX commented 4 years ago

This issue greatly affects a bot present in well over 100k servers. Fixing the issue should be a priority.

epicexcelsior commented 4 years ago

Still affected by this, the bot has well over 100k guilds.

SudhanPlayz commented 4 years ago

This issue affecting the small bots too like 50 servers

ghost commented 4 years ago

This issue is actually affecting my server’s own dedicated bot too which is only in the two servers (main and management).

ahtinker commented 4 years ago

This issue is affecting my bot which is in 2 servers

Ichigo271 commented 4 years ago 
Package     Current  
discord.js   12.0.2  
erlpack       0.1.3  
zlib-sync     0.1.6

invited in 2 discord guilds

Hi, I just had this problem, my heroku dyno restarted and my bot was unavailable for one hour (at this point i manually restarted and it connected without problem). discord.js.log

429 hit on route /gateway/bot loop every 15 seconds after "Preparing to connect to the gateway..."

I've updated discord.js and i'll see if it happens again

vladfrangu commented 4 years ago

I'd like to clarify and ask a few things about this issue, if possible 😄

We do have backoffs for login ratelimits, /gateway/bot goes through the same API route/ratelimit parsing like everything else. This error you're encountering is highly likely to happen if you attempt to login with multiple shards in parallel. This shouldn't be affected by internal sharding however.

I've also been simply unable to replicate this with curl, and with discord.js as a whole. Most I've gotten was 1 429 at boot due to me reconnecting too soon (the ratelimit right now is 2/5 [2 every 5 seconds])

So.. how can we replicate this issue? What's your system? Are you sharding, if so how? What are some steps you can take that causes this to happen every time you take them? This information is going to help us a lot in debugging, and either fixing it on our side, or possibly guiding you into fixing it on your side 😄

MattIPv4 commented 4 years ago

So, I haven't had this happened in a while, but I think this actually comes from being IP banned from the API for a while (so, you could probably reproduce this by hammering the API for a short while until you get tempbanned). When the happens and you try to connect, you'll get a 429 on /gateway/bot. The debug logs from the library when this happens are very consistent in timing, so it doesn't feel like any backoff is happening.

MattIPv4 commented 4 years ago

As for my specific setup, using ShardingManager to handle shard spawning. This was happening when a single shard restarted, and would then just sit being ratelimited seemingly forever.

vladfrangu commented 4 years ago

I think this actually comes from being IP banned from the API for a while

@MattIPv4 could this be related to https://github.com/discordjs/discord.js/issues/3560#issuecomment-692330833? (the API ban possibly coming from Cloudflare, which returns in seconds but we don't handle that yet)? It would kinda make sense if so, and it's probably highly related if so...

MattIPv4 commented 4 years ago

Yes! I saw your comment on that in Discord last night but wasn't auth'ed into GitHub. I think these two are probably intertwined, as this would be a Cloudflare ban, where Retry-After actually follows the spec and is in seconds.

One thing to note though, is that I'm still not convinced this is going through the exact same ratelimiting logic, as the ratelimit event doesn't get fired for this (#3891).

vladfrangu commented 4 years ago

Wrong, it IS going through the same ratelimit logic, HOWEVER...

https://github.com/discordjs/discord.js/blob/01ceda5b0cd6a14a49862a91b14c714bca4898a9/src/rest/RequestHandler.js#L148-L153

As you can see, on 429's we wait our the retryAfter. The check for emitting the rateLimit event,

https://github.com/discordjs/discord.js/blob/01ceda5b0cd6a14a49862a91b14c714bca4898a9/src/rest/RequestHandler.js#L69-L98

checks that this.limited is truthy. limited returns true under these conditions:

https://github.com/discordjs/discord.js/blob/01ceda5b0cd6a14a49862a91b14c714bca4898a9/src/rest/RequestHandler.js#L50-L52

Notice the Date.now() < this.reset. We already waited the supposed retryAfter, so Date.now() > this.reset.. this is because, as mentioned in https://github.com/discordjs/discord.js/issues/3560#issuecomment-692330833 we don't handle Cloudflare bans yet

This won't be a problem once we move to API v8, as everything is according to the spec, and uses seconds (or..shouldn't be anymore) 👍

vladfrangu commented 2 years ago

@MattIPv4 is this still an issue you encounter with newer releases?

MattIPv4 commented 2 years ago

👋 I haven't been in a situation where Discord has banned my token in a long time, but given d.js now uses v10 (I think) I imagine this has resolved itself

vladfrangu commented 2 years ago

Then I'll be closing this 🎉. Follow up if you encounter this again 😄