search

mediacloud / rss-fetcher

Intelligently fetch lists of URLs from a large collection of RSS Feeds as part of the Media Cloud Directory.
https://search.mediacloud.org/directory
Apache License 2.0
5 stars 5 forks source link

file occasionally not showing up on s3 #13

Closed rahulbot closed 1 year ago

rahulbot commented 1 year ago

Each night after we generate the RSS file, we sync the whole folder of files to S3 in a separates script. The file is sometimes not showing up on S3 until a day later. We need to figure out if it is the sync that is failing occasionally, or the file generation.

rahulbot commented 1 year ago

This seems to have been resolved when we switched to the new (UMass) server.