FoxxMD / context-mod

an event-based, reddit moderation bot built on top of snoowrap and written in typescript
https://contextmod.dev
MIT License
49 stars 11 forks source link

"non-actioned" events #41

Closed OmgImAlexis closed 2 years ago

OmgImAlexis commented 2 years ago

I'd like to see a page with the results for every check that's run as well as a button to rerun the check on that post/comment.

FoxxMD commented 2 years ago

It's a good idea but the blocking issue for this is scaling. For low-volume subreddits this would be a non-issue but CM runs on subreddits which do checks daily across 13k-30k activities. It's not feasible to hold all of those results in memory (the default cache/store configuration). Redis would work but still doesn't scale across multiple days...I've got one subreddit that's done 9.6 million activities in the last month.

Issues I see with this and some off-the-cuff remedies:

Scaling

How to store results in a way that works from 10 to 10k activities per day?

Finding useful information

At lower volumes it should be easy to find the result you are looking for. But at higher volumes will need to be able to filter results or else it'll be as useless as fast logging for high-volume subreddits

Additionally, may be prudent to fold all results to this change. So actioned/non-actioned alike. Then actioned can just be another filter (maybe still store separately so tailing limit doesn't apply?)

Retaining useful information

A user may want to retain a certain result but if the tailing limit is low or subreddit is high volume it could potentially be dropped quickly. Will need a way to store and retrieve user-defined results.

OmgImAlexis commented 2 years ago

I’ll expand more into this but redis. It’ll mean using redis. I’ve dealt with this kind of volume.

The other option would be to allow this to be enabled/disabled so smaller subs or subs needing to debug can use it.

FoxxMD commented 2 years ago

Implemented in 0.11.0 using Recording Options