claffin / cloudproxy

Hide your scrapers IP behind the cloud. Provision proxy servers across different cloud providers to improve your scraping success.
https://cloudproxy.io/
MIT License
1.4k stars 80 forks source link

Proxy without usr/psw authentication and IP-based auntenthication #73

Closed gitrid closed 1 year ago

gitrid commented 1 year ago

Chromium based solutions do not support user/password proxy authentication and given the fact that it is outdated/unsecure option it will rather not change in future. To bypass this, this request:

No breaking changes, although I don't know why in case of proxies without username and password this check_alive directly reaching to proxy fails: result = requests.get("http://" + ip_address + ":8899", timeout=3) and returns this error (after removing timeout restriction): Unable to connect Tinyproxy was unable to connect to the remote web server. Generated by [tinyproxy](https://tinyproxy.github.io/) version 1.11.0. However, using the very same IP as a proxy works without issues, hence I updated the line to: result = requests.get("http://ipecho.net/plain", proxies={'http': "http://" + ip_address + ":8899"}, timeout=3).

claffin commented 1 year ago

Thank you for the contribution @gitrid