mediacloud / metadata-lib

How Media Cloud approaches extracting metadata from online news stories
Apache License 2.0
12 stars 5 forks source link

store MC user-agent for use by our other libraries #81

Closed rahulbot closed 9 months ago

rahulbot commented 9 months ago

(forked from https://github.com/mediacloud/rss-fetcher/issues/36) We should store our MC user agent string in one place so we can use it in other apps. That said, perhaps it shouldn't be default in here for fetching. Related change would be to make this more generic (it already supports passing in your own UA for fetches done by this library): https://github.com/mediacloud/metadata-lib/blob/b67c651550e5a68fa1531b7699c96178f2dd663a/mcmetadata/webpages.py#L9