PDX-Capstone-Team-C scrapy issues

PDX-Capstone-Team-C / scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

http://scrapy.org

BSD 3-Clause "New" or "Revised" License

0 stars 4 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Doc update

#60 mjsiegfried closed 8 years ago
0
updated documentation for delta backend

#59 mjsiegfried closed 8 years ago
1
Added functions to handle http compression

#58 trevoreyre closed 8 years ago
0
Added DeltaLeveldbCacheStorage class

#57 mjsiegfried closed 8 years ago
0
Topic/recompute source

#56 ericalmadovapsu closed 8 years ago
0
Ignore Time on read_data when not calling from Retrieve_Response

#55 mjsiegfried opened 8 years ago
0
Updated _parse_domain_from_url function

#54 trevoreyre closed 8 years ago
0
Run tests in Python 3

#53 mjsiegfried opened 8 years ago
0
added unit test for DeltaLeveldbStorage

#52 mjsiegfried closed 8 years ago
0
added 2 unit tests DeltaLeveldbStorageTest and DeltaLeveldbStorageBsd…

#51 mchichou2015 closed 8 years ago
0
Update test_downloadermiddleware_httpcache.py

#50 mchichou2015 closed 8 years ago
1
removed unnecessary delta check from store_response

#49 sgarciapdx closed 8 years ago
0
:fixed issue of sources not being retrieved properly

#48 mjsiegfried closed 8 years ago
1
Topic/recompute deltas

#47 ericalmadovapsu closed 8 years ago
0
Bug: _recompute_deltas always triggers

#46 sgarciapdx closed 8 years ago
2
Topic/try bsdiff4

#45 sgarciapdx closed 8 years ago
0
Remove dirty hack from setup.py

#44 sgarciapdx opened 8 years ago
1
Decompress a file that was compressed by the server before calculating deltas

#43 mjsiegfried opened 8 years ago
0
Use BSDiff4 in place of Xdelta3

#42 mjsiegfried closed 8 years ago
1
Get domain from request url

#41 trevoreyre closed 8 years ago
0
Sloppy, temp solution for recompute

#40 ericalmadovapsu closed 8 years ago
0
Store cache by domain

#39 trevoreyre closed 8 years ago
0
Topic/refactor cleanup

#38 sgarciapdx closed 8 years ago
0
changed method call, added comments

#37 sgarciapdx closed 8 years ago
1
Testing for python3

#36 sgarciapdx opened 8 years ago
1
implemented store-by-domain, refactored a bit

#35 sgarciapdx closed 8 years ago
1
Making us PEP8 conformant & got rid of included "pprint" library for pretty printing

#34 ericalmadovapsu closed 8 years ago
0
Store list of targets in database (currently created but not stored)

#33 mjsiegfried opened 8 years ago
1
fixed up pep8 issues

#32 mjsiegfried closed 8 years ago
1
added hack for making tox work in vagrant

#31 sgarciapdx closed 8 years ago
0
storing original response length in db

#30 sgarciapdx closed 8 years ago
0
Integrate Domain into source history tree

#29 mjsiegfried opened 8 years ago
1
Topic/source list

#28 ericalmadovapsu closed 8 years ago
0
Extract domain from a url

#27 sgarciapdx opened 8 years ago
0
Decide whether or not to store one big cache, or by spider

#26 mjsiegfried opened 8 years ago
0
Begin communication with upstream

#25 mjsiegfried opened 8 years ago
0
Run tests for PEP8 Compliance

#24 mjsiegfried opened 8 years ago
0
Replaced JSON with cPickle for serialization

#23 trevoreyre closed 8 years ago
1
xdelta and level db cache storage class

#22 trevoreyre closed 8 years ago
0
ERROR: Compression_Enabled needs to be set to False for LeveldbDeltaCacheStorage backend to work

#21 mjsiegfried opened 8 years ago
2
Delta + Level DB cache storage class

#20 trevoreyre closed 8 years ago
1
Merging topic/fswrapper with develop

#19 sgarciapdx closed 8 years ago
0
Look into tests necessary for pull requests

#18 mjsiegfried opened 8 years ago
0
Look into openvcdiff

#17 mjsiegfried closed 8 years ago
0
XDelta Encoding / Buffer Size

#16 mjsiegfried opened 8 years ago
1
Handle out of date deltas getting replaced when they are out of date

#15 mjsiegfried opened 8 years ago
0
Reconstruct the raw http response from scrapy's objects

#14 mjsiegfried closed 8 years ago
0
Create a system for deciding which deltas to use as sources

#13 mjsiegfried opened 8 years ago
0
Create a system for storing delta references

#12 mjsiegfried opened 8 years ago
0
Topic/fswrapper

#11 sgarciapdx closed 8 years ago
0