search

taganaka / polipus

Polipus: distributed and scalable web-crawler framework
MIT License
92 stars 32 forks source link

Kill s3 entirely, use Fog, yo! #47

Open taganaka opened 10 years ago

taganaka commented 10 years ago

Current s3 implementation is partially broken and it doesn't work well under heavy load

Providing a fog adapter looks a way better to me

http://fog.io/storage/