Ratelimits for FredBoat

[schnapster]

Ratelimit for users. Especially stuff that does a lot of requests/heavy calculation stuff (example: loading a spotify playlist)

This needs some kind of (temporary) data keeping on users. What kind of DB is the current one used?

[Nanabell]

postgresql iirc

[freyacodes]

I don't see why this could not be achieved from just a HashMap?

[schnapster]

Some general thoughts:

• Rate limiting on a per-guild level and a (global) per-user level -> take out abusers without affecting the guilds they use for their actions
• Even if a user/guild has depleted credits to add songs, music control commands should keep working.
• I don't really want to stop someone from loading some crazy 500 songs or whatever playlist, but if he decides to do, he should be blocked for an appropriate time from adding more songs.
• There are two possible ways of abuse: spamming expensive commands (like loading songs), and just spamming a lot of commands (no one needs to pause/unpause or request meme pictures 100 times in a minute). The latter one should not block for a long time (especially not for guilds), some per-minute rate of commands seems appropriate.
• song limit for the guilds should replenish more often and be rather generous than the one for a single user. we want to punish individual abusers in the first place, not the guild they are home to. guilds should only be punished when they are home to several abusers that they aren't taking care of.

Leaves me with a fourfold system of rate limiting: (all numbers made up, I think you have more experience than me to adjust them properly; it can be done when the code is written, this is more about exploring what kind of rate limiting we want)

song limit per user (adding 100 songs per hour, updated every hour) song limit per guild (adding 1000 songs per hour, updated every 10 mins) command limit per user (issuing 100 commands per minute, updated every minute) command limit per guild (issuing 200 commands per minute, updated every minute)

Some examples:

• A user adds a 500 song troll playlist (we should still add some kind of hard limit to playlists, maybe only allow moderators on a guild add really long ones): he will have to wait 5 hours until he can add more songs, while his guild still has enough limit left to use the bot
• A spammer spams ;;np or whatever commands and gets limited at 100 commands per minute; other users of the guild still have access to 100 commands per minute to use; if it's a guild of spammers they get limited at 200 commands per minute, and all users get affected, even if it's only until the minute ends to give guild admins just enough motivation to get rid of the abusers without making it unusable for everyone else entirely.

Additional things: Daily limits, example: 10000 commands per day for guilds: allows guilds to peak at 200 commands per minute, but if they continue to abuse it for a longer period (10000 = 200 * 50 minutes), bars them from using the bot for a longer period.

Too complex? Something missing? Entirely in the wrong direction? Let me know what you think.

[Nanabell]

100 commands per minute for a user ? that would be over 1.6 Commands per seconds for a FULL minute to reach that. do we really have spammer of that constancy ?

[freyacodes]

I've been thinking of some buckets:

User buckets:

Any command: 5/10s
Skip*: 5/20s 
Queue playlist: 4/20s
(m)shards*: 2/30s

Guild buckets:

Any command: 10/10s
Slow playlist import (such as hastebin): 1000 tracks / 2m

No changes to the 10000 tracks restriction we currently have. These buckets are all on a user basis. Bear in mind that loading youtube playlists is very fast, as lavaplayer only needs to make a request for each page with up to 100 tracks each.

*=Manage messages or otherwise admin bypasses this

[schnapster]

Queue playlist: 4/20s

this is about youtube playlists, right? or using the ;;queue command?

[freyacodes]

Yes, using standard playlists

[schnapster]

Several problems with that rule:

• its doesn't stop abusers, who can load up giant playlists 4 times every 20 seconds. they will be stopped much more effectively by the 10000 tracks limit. I don't see a special reason to implement a ratelimit for regular users which is rather hard to detect, coming to the next point:
• it's very hard to detect if something is a playlist, without completely loading it - at which point the damage has been done (hogging the bot's resources) and the ratelimiter merely only prevents the songs from being added to the queue. the latter should be a decision the guilds should handle individually, not us setting an arbitrary and abusable (see point 1) standard.

Why is it hard to detect if something is a playlist? Because links just get thrown at lavaplayer, and we only see what's behind them when it returns them (except maybe for our self written importers, but those aren't the issue here). Lavaplayer also works with url shortners: try ;;play https://goo.gl/MohJIK, so we can't just throw a regex at links and filter for youtube playlists.

Please rethink that approach. Preferably to something that translates to "amount of some command per time". Can also do "songs added per time", just not in a perfectly strict way.

About that, some minor clarification for this rule needed: Slow playlist import (such as hastebin): 1000 tracks / 2m What happens if someone loads a playlist with 1001 tracks?

Does he get denied? * If not, when is the guild allowed to add more slowly loaded playlists? 2 minutes? 4 minutes? 2 minutes and 1 second? If allowed to go over 1k tracks: after loading a playlist with 8000 tracks, how long does the guild has to wait? 2 minutes, 16 minutes?

[freyacodes]

On second though it's not really a very good bucket, but the idea was more to prevent someone's queue from getting raided

[schnapster]

Both playlist buckets a bad idea?

[schnapster]

This has been deployed a while ago and has been working fine, both from a user perspective (I am not aware of any major complaints or improvement suggestions) and a resource perspective (database, memory, response times; caveat: no hard numbers support that claim, just a general absence of issues).

The ratelimits might need to be revisited in the future as we add more things.