stewartmckee / cobweb

Web crawler with very flexible crawling options. Can either use standalone or can be used with resque to perform clustered crawls.
MIT License
227 stars 45 forks source link

Add missing config options, fix default values and layout #20

Closed fqxp closed 10 years ago

fqxp commented 10 years ago

Hi,