[Feature] Support socks5 proxy with authentication

viponedream commented 3 years ago

Playwright version

lastest

Operating system

No response

What browsers are you seeing the problem on?

No response

Other information

No response

What happened? / Describe the bug

  _proxy = {
        "server": f"{proxy.scheme}://{proxy.ip}:{proxy.port}",
        "username": f"{proxy.username}",
        "password": f"{proxy.password}"
    }

this proxy is correct, i can use it via requests. but in playwright, it doesnt work . the doc says it support http or socks, how to use socks5 proxy ? thank yoy.

Code snippet to reproduce your bug

_proxy = {
            "server": f"{proxy.scheme}://{proxy.ip}:{proxy.port}",
            "username": f"{proxy.username}",
            "password": f"{proxy.password}"
        }

this proxy is correct, i can use it via requests. but in playwright, it doesnt work .
the doc says it support http or socks, 
how to use socks5 proxy ?
thank yoy.

Relevant log output

No response

mxschmitt commented 3 years ago

What kind of socks5 proxy are you using? Could you provide us e.g. a squid proxy configuration so we can repro?

Looking at the ecosystem:

Chromium doesn't support it: https://issues.chromium.org/issues/40323993
Firefox doesn't support it: https://bugzilla.mozilla.org/show_bug.cgi?id=122752
WebKit networking stack for linux: libsoup seems to support it
WebKit networking stack for Windows: libcurl seems to support it
WebKit networking stack for macOS: cfnetwork seems to support it: kCFStreamPropertySOCKSUser

I think the dependency on this issue is Firefox and Chromium support. Without that its hard for us to drive this forward. I recommend having maybe a proxy between you and the actual proxy in the meantime.

pavelfeldman commented 3 years ago

Socks5 proxy w/ auth is not supported but major browsers or infrastructures. Could you share your use case?

viponedream commented 3 years ago

import asyncio import requests from playwright.async_api import async_playwright

username = 'usertestme888' password = 'usertestme888' ip = '47.241.238.22' port = '2333'

async def mytest():

url = 'http://httpbin.org/ip'

proxies = {
    'http': f'socks5://{username}:{password}@{ip}:{port}',
    'https': f'socks5://{username}:{password}@{ip}:{port}'
}

res = requests.get(url, proxies=proxies,  verify=False)
print(res.text)

async def main(): proxy = { "server": f'https://{ip}:{port}', "username": f"{username}", "password": f"{password}" }

user_data_dir = ''

playwright = await async_playwright().start()
browser = await playwright.chromium.launch_persistent_context(
    user_data_dir,
    proxy=proxy,
    timeout=300000,
    headless=False,
)
page = await browser.new_page()
await page.goto('https://baidu.com')
pass

if name == 'main':

asyncio.run(mytest())
asyncio.run(main())
pass

pavelfeldman commented 3 years ago

Let's see if others upvote this.

dstampher commented 2 years ago

Any update on adding this functionality?

cyberfuhrer commented 2 years ago

Really need socks5 proxy authentication

colinsarah commented 2 years ago

socks5 with auth is ok？

TurboKach commented 1 year ago

still need socks5 proxy with credentials auth

igorank commented 1 year ago

still need socks5 with auth

AbelLin1214 commented 1 year ago

still need socks5 with auth

tecbr commented 1 year ago

still need socks5 with auth

georgycus commented 1 year ago

still need socks5 with auth

Lucas-Bruschi commented 1 year ago

still need socks5 with auth

dgtlmoon commented 1 year ago

to those who keep posting the same comment - please just upvote the post at the top

for something productive here - I tried to bump the 'proxy-authorization' header, but ofcourse this triggers net::ERR_INVALID_ARGUMENT from inside puppeteer because overriding some headers is not supported

            if self.proxy and self.proxy.get('server', '').lower().startswith('socks5://'):
                import base64
                auth_str = base64.b64encode(f"{self.proxy['username']}:{self.proxy['password']}".encode('utf8')).decode('ascii')
                self.proxy['username'] = ''
                self.proxy['password'] = ''
                request_headers['proxy-authorization']= f"Basic {auth_str}"

https://github.com/puppeteer/puppeteer/issues/676 https://cri.dev/posts/2020-03-30-How-to-solve-Puppeteer-Chrome-Error-ERR_INVALID_ARGUMENT

However an interesting work-around is to use a local SOCKS proxy chain, so you bounce a local request to localhost socks server off to the real server, because SOCKS chains very well due to its low-level TCP socket type arrangement

https://medium.com/apifier-blog/how-to-make-headless-chrome-and-puppeteer-use-a-proxy-server-with-authentication-249a21a79212

I also tried self.page.evaluate('const aHandle = page.authenticate({"xxxx", "pppp"});') but that didnt work (unsure why) SyntaxError: Unexpected string at eval (<anonymous>) at UtilityScript.evaluate (<anonymous>:201:30) at UtilityScript.<anonymous> (<anonymous>:1:44)