search

Xyrio / RSSOwlnix

RSSOwlnix is a fork of RSSOwl a powerful application to organize, search and read your RSS, RDF & Atom news feeds in a comfortable way. Highlights are saved searches, notifications, filters, fast fulltext search and a flexible, clean user interface.
Eclipse Public License 1.0
237 stars 22 forks source link

Feature request: clean up duplicate news within a feed #47

Open goaud27 opened 5 years ago

goaud27 commented 5 years ago

Some websites refresh their feeds partially and my news bin often has 2-8 duplicates of the same news differing only by date and time. Attached is an example. newsDuplicates

Xyrio commented 5 years ago

34

goaud27 commented 5 years ago

The issue #34 was about duplicate feeds. I think some functionality regarding this issue was already implemented in the original rssowl. I am talking about duplicate news within a single feed, which is a different problem. It should be possible to delete the duplicate news since a simple "group by name" already shows when there are several news with the same name. One just needs to implement deleting all extra copies within such a group (keeping only the oldest e.g.)

Xyrio commented 5 years ago

i see duplicate feeds/news detection as the same feature. also both are diffiucult to compare.

duplicate news title does not always mean duplicate news. the shorter the title the more false positives.

those duplicates can also appear when the rss creator corrects something that was used to identify the news uniquely. some might do this intentionally.

TroyDanielFZ commented 3 years ago

i see duplicate feeds/news detection as the same feature. also both are diffiucult to compare.

duplicate news title does not always mean duplicate news. the shorter the title the more false positives.

those duplicates can also appear when the rss creator corrects something that was used to identify the news uniquely. some might do this intentionally.

How about mark it duplicate by the links? For most people, the entires refer to the same page are duplicates.

Xyrio commented 3 years ago

menu: Tools/Preferences/Feeds/Reading make sure the checkbox is on for "automatically mark duplicate news as read" thats all there is. there might be cases where it wont work.

it is unlikely that i add any more heuristics for this if they are too complicated, error prone or time consuming.