search

chris-yoon90 / Automated-facets

0 stars 0 forks source link

Evaluate Web crawlers #10

Closed chris-yoon90 closed 11 years ago

chris-yoon90 commented 11 years ago

I did some research on some possibilities of crawling apps. General consensus is that if you don't have money to hire a lawyer, don't crawl craigslist whether it's legal or illegal. So I don't really like the sound of that, so crawling craigslist is not a good idea. However, we can definitely try with other web sites, but I'm not really sure WHAT we can crawl. I may need more ideas..........................

colinmleung commented 11 years ago

http://addyosmani.com/blog/spotlight-issue1/

Combines both the node library and web crawler projects. Web scraping with node.