storing original response length in db

PDX-Capstone-Team-C / scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

http://scrapy.org

BSD 3-Clause "New" or "Revised" License

0 stars 4 forks source link

storing original response length in db #30

Closed sgarciapdx closed 8 years ago

sgarciapdx commented 8 years ago

Turns out you can't pickle a tuple, so I'm storing the length of an original serialized response in the DB under its own key. Ended up writing a _read_delta method as a result.