ArchiveTeam / NewsGrabber

Grabbing all news.
62 stars 32 forks source link

Support ignoreregex for each site #12

Open PressStartandSelect opened 8 years ago

PressStartandSelect commented 8 years ago

Sites should have an ignoreregex parameter to avoid regrabbing pixel trackers and other unwanted files (such as files that consistently give 4xx errors).