Redis-Queus: Auto-removal of jobs

[Behzad-rabiei]

For the Discord bot, we have the following queues:

• ChannelMessage: (Announcement feature)
• DirectMessage: (Announcement feature)
• CronJob: 1 job per day
• GuildEvent: 9 Discord events are associated with this queue (create, update, delete -> Role, Channel, GuildMember)
• GuildExtraction: 1 job per Discord platform per day
• UserEvent: Used only for userUpdate events (tracks global username changes, not related to GuildEvents)

We can automatically remove jobs for both completed and removed jobs. For each type, we can consider the age and the maximum number of jobs we can keep for each queue.

Proposed configuration for each queue:

• channelMessage & directMessage:

  • Complete: max: 1000 - age: 1 day
  • Fail: max: 1000 - age: 7 days

• CronJob:

  • Complete: max: 365 - age: 365 days
  • Fail: max: 365 - age: 365 days

• GuildExtraction:

  • Complete: max: 1000 - age: 7 days
  • Fail: max: 500 - age: 30 days

• GuildEvent:

  • Complete: max: 6000 - age: 3 hours
  • Fail: max: 600 - age: 7 days

• UserEvent:

  • Complete: max: 2000 - age: 3 hours
  • Fail: max: 200 - age: 7 days