search

mediacloud / rss-fetcher

Intelligently fetch lists of URLs from a large collection of RSS Feeds as part of the Media Cloud Directory.
https://search.mediacloud.org/directory
Apache License 2.0
5 stars 5 forks source link

incorrectly marking duplicate file hash as a fetch failure #12

Closed rahulbot closed 1 year ago

rahulbot commented 1 year ago

Right now if the code fetches the RSS feed and it has a duplicate hash to the last time (ie. the file hasn't changed), it marks that as a fetch failure. That logic is wrong. We need to fix that logic and also reset all the failure counts so that every feed in the database is checked again.