search

pgaref / HTTP_Request_Randomizer

Proxying Python Requests
http://pgaref.com/blog/python-proxy/
MIT License
151 stars 59 forks source link

trace time so proxies may be ordered by response time #11

Open ieguiguren opened 8 years ago

ieguiguren commented 8 years ago

When a proxy is checked, if request time is traced and kept, it may be used later to order the list of proxies per response speed, being able to return the fastest proxy in the list in stead of a random one. As checking the whole proxies list takes a while, it could return the fastest proxy as a fastest proxy appears during checks.

I can't tag this as 'enhancement'

pgaref commented 8 years ago

Sounds reasonable. For that purpose we could create an extra ProxyManager class which would store the proxies in a dictionary of the form Proxy:Response_time(ms) - Response time would be None at initialization time and at each iteration we would store the new time. Then we would query the ProxyManager for next_random or next_faster proxy.