Closed astyfoo closed 5 months ago
Hello, first of all happy holidays!
Second, I'm hoping this can be merged with upstream - if there is any additional documentation or refactoring needed please just say the word.
Sorry it took so long to get to this!
Setting up some bots for scraping RSS and pushing to Mastodon recently, I notice many larger sites (e.g. CNBC) update their entries either to make updates to make changes, or just push old entries to the top. This causes feediverse to re-toot entries, sometimes several times in a row.
Added a command line flag (by default disabled) to add some basic de-duplication and maintain its state in the .feediverse file.