search

taganaka / polipus

Polipus: distributed and scalable web-crawler framework
MIT License
92 stars 32 forks source link

added the ability to use list user agents #62

Closed parallel588 closed 9 years ago

taganaka commented 9 years ago

nice :+1: