implemented store-by-domain, refactored a bit

PDX-Capstone-Team-C / scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

http://scrapy.org

BSD 3-Clause "New" or "Revised" License

0 stars 4 forks source link

implemented store-by-domain, refactored a bit #35

Closed sgarciapdx closed 8 years ago

sgarciapdx commented 8 years ago

Sources are actually stored by spider name for now. Also started to compartmentalize source selection and delta recomputing for source response changes.

ericalmadovapsu commented 8 years ago

Thumbs up from me