Some RSS feeds are technically invalid but still want to be fetched. In these cases we want to make sure we can still fetch them and properly handle when the feed item does not contain correct tags for dates.
Previously we used published and internally last_digest to determine if a feed has already been sent to the user. Well some feed items do not contain published so we would always send the feed item to the user in their email digest.
Now we are storing the feed items we have fetched in a separate table along with some of its metadata in order to properly mark a feed item as "already sent." Now it doesn't matter if the feed item has the correct tags or not, we will try to fetch the items, send them to the user, and them mark them as already sent.
Some RSS feeds are technically invalid but still want to be fetched. In these cases we want to make sure we can still fetch them and properly handle when the feed item does not contain correct tags for dates.
Previously we used
publishedand internallylast_digestto determine if a feed has already been sent to the user. Well some feed items do not containpublishedso we would always send the feed item to the user in their email digest.Now we are storing the feed items we have fetched in a separate table along with some of its metadata in order to properly mark a feed item as "already sent." Now it doesn't matter if the feed item has the correct tags or not, we will try to fetch the items, send them to the user, and them mark them as already sent.