use paid proxy and get error while tunneling

[tjengbudi]

hello, i want to ask about some question about proxy, i have a paid proxy that used authentication i trying with pure "requests" in blank project to fetch google and it works fine. but after i added it at yagooglesearch it have error like this Traceback (most recent call last): File "C:\Users\budi\.virtualenvs\article-30Om8FEk\lib\site-packages\urllib3\connectionpool.py", line 700, in urlopen self._prepare_proxy(conn) File "C:\Users\budi\.virtualenvs\article-30Om8FEk\lib\site-packages\urllib3\connectionpool.py", line 996, in _prepare_proxy conn.connect() File "C:\Users\budi\.virtualenvs\article-30Om8FEk\lib\site-packages\urllib3\connection.py", line 369, in connect self._tunnel() File "C:\Users\budi\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 924, in _tunnel raise OSError(f"Tunnel connection failed: {code} {message.strip()}") OSError: Tunnel connection failed: 407 Proxy Authentication Required

i check several time and it can used in pure requests code. is there any clue for this?

[opsdisk]

Take a look at https://requests.readthedocs.io/en/latest/user/advanced/#proxies

You can...

1) set proxy using this format when instantiating the class proxy=http://user:password@host:port

2) hardcode it in the library here: https://github.com/opsdisk/yagooglesearch/blob/master/yagooglesearch/__init__.py#L183

3) or trying wrapping it with proxychains4 - https://github.com/opsdisk/pagodo#proxychains4-support

[tjengbudi]

hello, as i state before i try with simple request and it is ok and can get the page through my paid proxy. so the number 1 is used but working with my test sample only, the sample is only request.get("https://www.google.com", proxies=proxy, verify=False) and it is ok without error. but with yagooglesearch it get's error. i already check the code there is no something strange in your code. or i might missing something? for clue number 2 i already check and passing proxy already good. it create http and https dict proxies. so it is ok. however when create tunneling in proxy ( i believe it is used by code in request lib) it is error authentication. i still did not trying about clue number 3.

btw the solution before is nice to run. with recaptcha solver. however it slow down crawling much. so i try with your clue about proxy. create proxy by myself, might be not best solution for me. cause i need cheap and fast as possible

the strange with test proxy is. it can run with simple run get method. andi see that you used the same approach. but i said it again i might missing something :(

[opsdisk]

@tjengbudi Without access to the paid proxy, it's hard for me to assist in troubleshooting this. I never got a response from you in our Twitter DM discussion. I'll keep this issue open a bit longer.

[opsdisk]

Closing this issue out @tjengbudi. If you have additional or new information, we can reopen this one.