search

alan-turing-institute / misinformation-crawler

Web crawler to collect snapshots of articles to web archive
MIT License
5 stars 2 forks source link

add extra urls weeklyworldnews #350

Closed edwardchalstrey1 closed 5 years ago

edwardchalstrey1 commented 5 years ago

Managed to get a few extra articles, 370 to 386

2019-08-05 11:16:47     INFO: Processed 388 pages in 0:00:54.894659 => 7.19 Hz
2019-08-05 11:16:47     INFO: Found articles in 386/388 pages => 99.48%
2019-08-05 11:16:47     INFO: ... of these 0/386 had no date => 0.00%
2019-08-05 11:16:47     INFO: ... of these 0/386 had no byline => 0.00%
2019-08-05 11:16:47     INFO: ... of these 0/386 had no title => 0.00%
2019-08-05 11:16:47     INFO: Including skipped pages, there are articles in 386/388 pages => 99.48%