Threads are not getting pruned

[Gummikavalier]

Description:

Pruning of threads is not working at least on channels that have channel based separate shorter retention time.

Steps to reproduce:

1. Set global pruning in admin settings admin -> settings -> retention policy for 1 year and apply it for everything. Disable option Do not prune threads. (Meaning do prune threads.)
2. Create a test channel and set retention time on that channel separately into 1 day.
3. Create a thread and couple of messages under that thread on the channel. Also type some individual messages on the channel for comparison later.
4. Two days later check the messages on the channel.

Expected behavior:

No messages can be found on the channel.

Actual behavior:

Individual messages have been pruned correctly but the thread and all messages in it remain on the channel at least for one year.

Server Setup Information:

• Version of Rocket.Chat Server: 6.9.3
• License Type: Enterprise
• Number of Users: -
• Operating System: RHEL8
• Deployment Method: tar
• Number of Running Instances: 2
• DB Replicaset Oplog: Yes
• NodeJS Version: 14.21.3
• MongoDB Version: 6.0

Additional information:

The thread pruning functionality works in RC 6.10.0 for newly created channels.

The bug would be in the upgrade process that does not add ignoreThreads: false into channel's properties for the existing channels that have already had the channel based pruning enabled in RC 6.9.x. (Which did not have such option at all.)

[Gummikavalier]

@dougfabris FYI. Observed on two separate older channels I used for testing. These channels were not completely new though, and I'm yet to do the third test with a completely new channel.

[Gummikavalier]

@dougfabris I have now tested with new and old channels and with both RC 6.9.3 and RC 6.10.0.

The result is that channel based retention policy doesn't prune threads with default settings, at least.

After creation of a new channel, and enabling channel based pruning and setting it to one day, you get these entries into the database for that channel:

    retention: {
      overrideGlobal: true,
      maxAge: 1,
      enabled: true,
      excludePinned: false,
      filesOnly: false
    },

After enabling Do not prune Threads option on that particular channel you get one more entry into the database:

    retention: {
      overrideGlobal: true,
      maxAge: 1,
      enabled: true,
      excludePinned: false,
      filesOnly: false,
      ignoreThreads: true
    },

After disabling Do not prune Threads on the channel you get:

    retention: {
      overrideGlobal: true,
      maxAge: 1,
      enabled: true,
      excludePinned: false,
      filesOnly: false,
      ignoreThreads: false
    },

I'll see and report back if the pruning of threads starts working after this.

Edit: Actually I was wrong. Old channels in RC 6.10.0 do not have that ignoreThreads: false. The new ones do. I have not tested yet on RC 6.10.0 channels that were created with this latest version. I'll do so.

Also RC 6.9.3 does not have this switch at all so this seems to be work in progress. If it now works as designed on RC 6.10.0, the bug is only that after the upgrade old channels do not get their settings updated.

[Gummikavalier]

Confirmed that the thread pruning functionality works in RC 6.10.0 for newly created channels.

The bug would be in the upgrade process that does not add ignoreThreads: false into channel's properties for the existing channels that have already had the channel based pruning enabled in RC 6.9.x. RC 6.9.x did not have a threads pruning option for single channels at all.

Edit: Those who have already started using independent channels pruning, simply update to RC 6.10.0, enable the threads pruning setting on such channels, save, and then disable it again, and save.

[github-actions[bot]]

This issue has been marked as stale because there has been no further activity in the last 10 days. If the issue remains stale for the next 4 days (a total of 14 days with no activity), then it will be assumed that the question has been resolved and the issue will be automatically closed.

[Gummikavalier]

Bump

[dougfabris]

Hey @Gummikavalier sorry for the delay, I was really busy here last days! Indeed in 6.10.0 we introduced the ability to Do not prune Threads in retention override but since channels hasn't the prop previously it won't work for old channels as you correctly said. I'm almost sure we'll have to add a migration to add this property to the old channels to fix it.

We usually wait for major releases to create migrations, but since it's just a addition I'm going to take a look with the team if it's possible to do that right away.

I really appreciate your help finding this one 🚀

[Gummikavalier]

@dougfabris Sorry, I got that bot stale message so I just bumped to keep this open. I keep one old channel and thread on our production system reserved for checking, and then confirm this fixed when it gets pruned.

But thanks for the response, appreciated! This is not actually a biggie as the feature was so recent it was not widely used in the past, and workaround is easy to do if required. :heart:

[github-actions[bot]]

This issue has been marked as stale because there has been no further activity in the last 10 days. If the issue remains stale for the next 4 days (a total of 14 days with no activity), then it will be assumed that the question has been resolved and the issue will be automatically closed.

[dougfabris]

@Gummikavalier well, good to know there's an easy workaround. We'll have to release it on the next major because of the migration. Thanks again!