Add random sleep period for crawling

jijames / electionWatch

A collection of crawlers and social media analysis tools to track and investigation concept trends

2 stars 0 forks source link

Open jijames opened 7 years ago

jijames commented 7 years ago

Slow down crawler with random sleep period. Do not DoS the sites!

imugi-games commented 7 years ago

added to both news and board generic crawlers