How can I start stop crawling website

stewartmckee / cobweb

Web crawler with very flexible crawling options. Can either use standalone or can be used with resque to perform clustered crawls.

MIT License

227 stars 45 forks source link

How can I start stop crawling website #46

Open qiun opened 8 years ago

qiun commented 8 years ago

It's possible to som how start, stop crawling website or pause from comand line?

stewartmckee commented 8 years ago

Hi,

Currently there isn't a way from the command line. I've started adding in some quick stop/status/list commands to enable you to do this, you need to specify redis-options to point to the right redis (defaults to localhost) and crawl-id which is the crawl you want to stop etc.

Not ready for release yet, but shortly, is this what you are looking for?