Web crawler with very flexible crawling options. Can either use standalone or can be used with resque to perform clustered crawls.
227
stars
45
forks
source link
Forced UTF-8 codification of request.body so it can be serialised correc... #22
Closed
wuiscmc closed 3 years ago