filipedeschamps / rss-feed-emitter

Super RSS News Feed aggregator written in Node.js and ES6
MIT License
1.06k stars 77 forks source link

Detection of new and updated feed items #221

Open mikesouza opened 3 years ago

mikesouza commented 3 years ago

Great package! Very useful, easy to use, and well tested and documented.

Issue

Currently the FeedManager identifies a feed item as being new by comparing it to the existing Feed items with the following logic:

// if feed is RSS 2.x, check existence of 'guid'
if (item.guid) return entry.guid === item.guid;
// if feed is Atom 1.x, check existence of 'id'
if (item.id) return entry.id === item.id;
// default object with 'link' and 'title'
return entry.link === item.link && entry.title === item.title;

In a perfect world, everyone who publishes an RSS or Atom feed would apply best practices. However, this is not the case for the feed I am consuming, which does not set a guid or id on items, so the title is used to determine whether an item is new or existing. The feed I'm consuming essentially publishes "updates" to existing items with the title unchanged and changes to various custom elements. Because the title stays the same, the FeedEmitter does not identify it as a new item and emit an event. As far as I can tell, there is no way for me to handle "updates" to these feed items.

Proposal

There are couple of solutions I that first come to mind that would solve the issue I'm having:

Option 1: Allow user to specify and override how a new item is identified (i.e. delegate the responsibility to a custom function) Option 2: Emit a separate event for when an existing item is identified

I'm open to other solutions and willing to help out and implement it, but wanted to get buy-in and feedback before spending the time and effort doing so.

Anyways, thanks again for this very nice package.

TobiTenno commented 2 years ago

feel free to submit a pr! Sadly, right now, any work i do is delayed behind getting approvals from filipe, so i'm not doing any additional work until I can get it set up so i can merge my own after others review it