search

alirezamika / autoscraper

A Smart, Automatic, Fast and Lightweight Web Scraper for Python
MIT License
6.16k stars 648 forks source link

how to use this behind a proxy #52

Closed mikelty closed 2 years ago

mikelty commented 3 years ago

this is a great project but when i open a proxy, nothing worked. help please.

alirezamika commented 3 years ago

can you use the requests module with your proxy?

mikelty commented 3 years ago

yeah, with a proxies variable in requests like such:

https://stackoverflow.com/questions/8287628/proxies-with-python-requests-module

(sorry for the late response)

alfarias commented 3 years ago

@mikelty Do you have solved your problem using proxies? I got some problems when using a proxy, but it was a bug on urllib module, because there's a bug when you pass https proxy. I don't know if you got a similar problem.

alirezamika commented 3 years ago

@mikelty is your problem solved?