Per feed retention policy

[shizunge]

Today there is only one global retention policy, which is controlled by a global parameter CLEANUP_ARCHIVE_READ_DAYS.

For some blogs, I want to keep articles as long as possible. For some news feeds, I do not need to keep many items that long.

It would be great if there is an option let us set per feed retention policy: 1. days to keep, 2. max items to keep.

I think this can be done in two steps:

1. Add maximum items per feed as a globle retention policy.

   Update the cleanup routine: Iterate all feeds and archive items for each feed. (Like the fresh routines, but this can be done using only 1 worker I think.) Then add a new global parameter: CLEANUP_MAX_READ_ITEMS_PER_FEED to control the max items per feed to keep. Set the parameter to 0 to store unlimited items. (Still subject to CLEANUP_ARCHIVE_READ_DAYS)

2. Add an option to set per feed retention policy. If not set, use the global policy default. This step probably can be done together with https://github.com/miniflux/miniflux/issues/412 (Fresh frequency setting for single feed)

[unwaivering]

I'd like to keep them all forever unless I mark them as read, then they can go away. So maybe have an option to keep them permanently unless otherwise noted.

[somini]

I'd like to keep them all forever unless I mark them as read, then they can go away. So maybe have an option to keep them permanently unless otherwise noted.

Do you run your instance for a long time? I have hundreds of feeds with a retention rate of 3 years now, and still have a DB of 3GB only, so this sounds a good feature.

[shizunge]

I'd like to keep them all forever unless I mark them as read, then they can go away. So maybe have an option to keep them permanently unless otherwise noted.

The cleanup policy should apply to read items only. Unread items should be kept forever. This should be true even in current version.

[unwaivering]

Just an FYI I'm using the hosted instance for now.

[shizunge]

In the latest version, we had two global config options: CLEANUP_ARCHIVE_READ_DAYS and CLEANUP_ARCHIVE_UNREAD_DAYS. We can set them to -1 to keep all entries.

The original proposal said we want to set the number of entries to keep per feed. It is difficult to write the SQL query. So we can let user to set the number of days to keep (read/unread) per feed, which is easier.

[Sleepful]

Is this feature approved to implement? @fguillot

[unwaivering]

There should just be a global one for all feeds. Why should I ever have to go into the terminal to change that?

[unwaivering]

In other words, it should just continue to persist the way it is.

[Kairixir]

@shizunge how does it look like with this feature? I find it extremely important for my feed since I am following multiple sources with differing posting frequency — some post multiple times per hour and some once per two weeks — and this feature bugs me.

I am willing to look into it. Would it be accepted if I do?

[shizunge]

I don't have time on it right now.

I believe we need to

1. Update database scheme to store the new user configuration. at internal/database/migrations.go
2. The cleanup codes go to runCleanupTasks
3. UI updates

I am also thinking that probably we should only let user set a shorter retention period (than the global ones) to avoid users abuse the system. Then we should also support unlimited global retention policy (e.g. CLEANUP_ARCHIVE_READ_DAYS=-1 or something similar)

[Kairixir]

Awesome, thanks for guidance :)

I'll try my best

[woj-tek]

(context: I have miniflux deployed on my RPi and I'm sole user).

As in the original message - it would be awesome to mark some feeds as "retain forever" (mostly those "low traffic", to have complete archive). Currently for those low traffic feeds I simply mark all entries with star but it's not really convenient and it may slip by me sometime thus a feed-option "keep items forever" would be very welcomed.