Rotating action logs

[gdude2002]

I had planned on looking into this on a later date, but some concerns were raised with me in private so I'm going to enumerate it now.

We don't need to keep action logs permanently. I think 30 days is plenty, and had considered deleting messages monthly, but there are some issues with that:

• Discord won't let you bulk-delete messages older than 2 weeks, which is not enough retention time
• Monthly rotations have windowing issues, making it harder for people to know the exact duration we're storing things for
• Direct deletion of messages requires a ton of database work, plus maintaining rows, deleting old ones when messages are deleted, etc

Instead, I'd like to propose the following solution:

• We keep a rotation of 5 channels for action logs
• Action log channels are rotated weekly - with the oldest one being deleted when a new one is created
• We formalize our informal action log policy, along the lines of "don't leak anything from the action log channels"

I think this is a relatively simple solution that retains action logs as long as we need them for (35 days), whilst stil respecting users' expectation of privacy (messages are always eventually removed if users delete/edit them). That said, I'd be happy to take constructive feedback in the comments of this issue.

---

Things that have been considered and decided against:

• Per-user opt-out: We simply can't trust users with this. Most users don't really care enough, and many people that decide to opt-out are likely to turn out to be troublemakers, even if there are people that behave themselves and would like to opt-out.

[jamierocks]

Why not just keep the logs forever, or for so long as you legally can (not sure on the GDPR implications of doing this)?

Per-user opt-out: We simply can't trust users with this. Most users don't really care enough, and many people that decide to opt-out are likely to turn out to be troublemakers, even if there are people that behave themselves and would like to opt-out.

Are there no GDPR concerns on this?

[gdude2002]

GDPR isn't relevant as we don't exfil any data - everything is kept on Discord. You could say there's an argument for us removing user profile info from them if a user deletes their account, though.

To answer the GDPR question though, the cutoff is around 30 days. That said, it's not applicable to what we're doing.

[Minenash]

Discord won't let you bulk-delete messages older than 2 weeks, which is not enough retention time

What do you mean by bulk-delete? Is it a specific api thing or is describing mass individual deletion of messages? I ask, because if it allows you to delete messages individually, then every week you could have the bot loop through all the messages in the channel, and if it's over 30 days old, delete it. If the thing you get back when you request a list of messages is sorted by date, then you could also stop the loop once you reached under 30 days. It wouldn't require any database work.

[2xsaiko]

Discord won't let you bulk-delete messages older than 2 weeks, which is not enough retention time

What do you mean by bulk-delete? Is it a specific api thing or is describing mass individual deletion of messages? I ask, because if it allows you to delete messages individually, then every week you could have the bot loop through all the messages in the channel, and if it's over 30 days old, delete it. If the thing you get back when you request a list of messages is sorted by date, then you could also stop the loop once you reached under 30 days. It wouldn't require any database work.

Yes, bulk delete is an API call. Not sure what the problem is with using the single delete call for older messages though, I did the same in my own discord bot.

[gdude2002]

I mostly want to avoid doing that because it's pretty slow and inefficient, and there are a lot of action log messages on a server with 10,000 people in it so the bot would end up getting rate limited quickly and be unable to delete other messages on the server

[gdude2002]

This has been fully implemented and is now running in production.