node-web-crawler

A web crawler with a web user interface which shows crawling stats in realtime. Uses Node.JS, jQuery, Socket.IO, Express, Twitter Bootstrap.

Node web crawler

Install

To use node web crawler, you need to have installed Node.js and MongoDB. When you have them both installed, just clone the repo:

git clone git@github.com:ecdeveloper/node-web-crawler.git

And then install dependencies:

npm install

That's it. Run the app:

node app