search

alan-turing-institute / misinformation-crawler

Web crawler to collect snapshots of articles to web archive
MIT License
5 stars 2 forks source link

Update date extraction for redstate.com #342

Closed edwardchalstrey1 closed 5 years ago

edwardchalstrey1 commented 5 years ago

Some articles don't have a meta tag for the date, so switch to using the date on the actual page, see #341

edwardchalstrey1 commented 5 years ago

Decided to keep using the meta tag for dates since this works for almost all articles anyway