integrate new user-agent string stored in mcmetadata

mediacloud / rss-fetcher

Intelligently fetch lists of URLs from a large collection of RSS Feeds as part of the Media Cloud Directory.

https://search.mediacloud.org/directory

Apache License 2.0

5 stars 5 forks source link

integrate new user-agent string stored in mcmetadata #37

Closed rahulbot closed 4 months ago

rahulbot commented 4 months ago

Splitting action off of #36. The new v0.12.0 of mcmetadata includes mcmetadata.webpages.MEDIA_CLOUD_USER_AGENT. We want to integrate that as the user-agent in wherever the rss-fetches requests content over http.