search

ArchiveTeam / grab-site

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
Other
1.32k stars 129 forks source link

imgur images are not grabbed after imgur redirects #127

Open ivan opened 5 years ago

ivan commented 5 years ago

Reported in the Edit in https://github.com/ludios/grab-site/issues/126#issuecomment-410379513 and confirmed by me, the actual images are neither in gs-dump-urls wpull.db todo nor in the WARC.