SeleniumHQ / selenium

A browser automation framework and ecosystem.
https://selenium.dev
Apache License 2.0
30.64k stars 8.18k forks source link

[🐛 Bug]: Trying to use socks5 proxy #12857

Closed TheGreatSimo closed 11 months ago

TheGreatSimo commented 1 year ago

What happened?

I am encountering an issue while using Selenium with Java to connect to a 4G proxy via SOCKS5 The browser opens successfully but there seems to be no actual internet connection within the browser I have reviewed my code and configurations, but I'm unable to identify the root cause of this problem

How can we reproduce the issue?

public static void main(String[] args) {
        String host = "hostprodiverhere (and yes it's correct )";
        String port = "(a correct port here)";

        ChromeOptions chromeOptions = new ChromeOptions();
        chromeOptions.addArguments("--proxy-server=socks5://" + host + ":" + port);

        WebDriver driver = new ChromeDriver(chromeOptions);

        driver.get("https://whoer.net/");
        WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10));
}

Relevant log output

Normal output no errors

Operating System

Linux (Ubuntu)

Selenium version

4.11.0

What are the browser(s) and version(s) where you see this issue?

chrome

What are the browser driver(s) and version(s) where you see this issue?

chromedriver

Are you using Selenium Grid?

no webdriver

github-actions[bot] commented 1 year ago

@NotJuba, thank you for creating this issue. We will troubleshoot it as soon as we can.


Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

titusfortner commented 1 year ago

Selenium does not manage Chrome arguments, so if --proxy-server is broken, it will need to be reported to Google.

The Selenium supported way of adding a proxy is to use the options.setProxy(myProxy). The Selenium Proxy object allows setting socksProxy value. This may still be an issue that Google needs to solve, but see if it works with this approach.

TheGreatSimo commented 1 year ago

@titusfortner Thank you bro I just subscribed to your youtube channel

TheGreatSimo commented 1 year ago

I'm noobie can you please give me an example of how to do

titusfortner commented 1 year ago

Hopefully, it's just:

        ChromeOptions options = new ChromeOptions();
        Proxy proxy = new Proxy();
        proxy.setSocksProxy(host + ":" + port);
        options.setProxy(proxy);
        driver = new ChromeDriver(options);
TheGreatSimo commented 1 year ago

My chromedriver is : 117.0.5938.92 and my Google chrome : 117.0.5938.132 and selenium is : 4.13.0 and webdrivermanager is : 5.5.3

I'm still getting this output and I have no clue how to fix it or what's causing the problem The proxy seems to be working perfectly fine but on selenium java

    > Task :run FAILED
    Oct 03, 2023 6:45:21 PM org.openqa.selenium.manager.SeleniumManager lambda$runCommand$1
    INFO: Driver path: /usr/bin/chromedriver
    Oct 03, 2023 6:45:21 PM org.openqa.selenium.manager.SeleniumManager lambda$runCommand$1
    INFO: Browser path: /opt/google/chrome/google-chrome
    SLF4J: No SLF4J providers were found.
    SLF4J: Defaulting to no-operation (NOP) logger implementation
    SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
    Exception in thread "main" org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Response code 400. Message: invalid argument: entry 0 of 'firstMatch' is invalid
    from invalid argument: cannot parse capability: proxy
    from invalid argument: 'socksVersion' must be between 0 and 255 
    Host info: host: 'MagicMirror', ip: '127.0.1.1'
    Build info: version: '4.12.0', revision: '249f2a7d1b*'
    System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.2.0-33-generic', java.version: '19.0.2'
    Driver info: org.openqa.selenium.chrome.ChromeDriver
    Command: [null, newSession {capabilities=[Capabilities {browserName: chrome, goog:chromeOptions: {args: [--remote-allow-origins=*], binary: /opt/google/chrome/google-c..., extensions: []}, proxy: Proxy(manual, socks=benj:ya...}]}]
            at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:140)
            at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:96)
            at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:68)
            at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:163)
            at org.openqa.selenium.remote.service.DriverCommandExecutor.invokeExecute(DriverCommandExecutor.java:196)
            at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:171)
            at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:602)
            at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:236)
            at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:163)
            at org.openqa.selenium.chromium.ChromiumDriver.<init>(ChromiumDriver.java:108)
            at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:88)
            at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:83)
            at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:72)
            at Tinder.main(Tinder.java:20)

    FAILURE: Build failed with an exception.

    * What went wrong:
    Execution failed for task ':run'.
    > Process 'command '/usr/lib/jvm/java-19-openjdk-amd64/bin/java'' finished with non-zero exit value 1

    * Try:
    > Run with --stacktrace option to get the stack trace.
    > Run with --info or --debug option to get more log output.
    > Run with --scan to get full insights.

    * Get more help at https://help.gradle.org

    BUILD FAILED in 1s
    4 actionable tasks: 3 executed, 1 up-to-date
TheGreatSimo commented 1 year ago

and this

    SLF4J: No SLF4J providers were found.
    SLF4J: Defaulting to no-operation (NOP) logger implementation
    SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
    Exception in thread "main" org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Response code 400. Message: invalid argument: entry 0 of 'firstMatch' is invalid
    from invalid argument: cannot parse capability: proxy
    from invalid argument: 'socksVersion' must be between 0 and 255 
    Host info: host: 'MagicMirror', ip: '127.0.1.1'
    Build info: version: '4.13.0', revision: 'ba948ece5b*'
    System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.2.0-33-generic', java.version: '19.0.2'
    Driver info: org.openqa.selenium.chrome.ChromeDriver
    Command: [null, newSession {capabilities=[Capabilities {browserName: chrome, goog:chromeOptions: {args: [--remote-allow-origins=*], binary: /opt/google/chrome/google-c..., extensions: []}, proxy: Proxy(manual, socks=benj:ya...}]}]
            at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:140)
            at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:96)
            at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:68)
            at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:163)
            at org.openqa.selenium.remote.service.DriverCommandExecutor.invokeExecute(DriverCommandExecutor.java:196)
            at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:171)
            at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:602)
            at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:236)
            at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:163)
            at org.openqa.selenium.chromium.ChromiumDriver.<init>(ChromiumDriver.java:108)
            at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:88)
            at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:83)
            at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:72)
            at Tinder.main(Tinder.java:18)
titusfortner commented 1 year ago

Looks like it wants you to specify socks version as well

TheGreatSimo commented 1 year ago

I tried proxy.setSocksVersion(5); options.setCapability("proxy", proxy);

but I still get this error

    > Task :run
    SLF4J: No SLF4J providers were found.
    SLF4J: Defaulting to no-operation (NOP) logger implementation
    SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
    SLF4J: Class path contains SLF4J bindings targeting slf4j-api versions 1.7.x or earlier.
    SLF4J: Ignoring binding found at [jar:file:/home/juba/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.7/25d160723ea37a6cb84e87cd70773ff02997e857/slf4j-jdk14-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See https://www.slf4j.org/codes.html#ignoredBindings for an explanation.
    Exception in thread "main" org.openqa.selenium.WebDriverException: unknown error: net::ERR_SOCKS_CONNECTION_FAILED
      (Session info: chrome=117.0.5938.132)
    Build info: version: '4.13.0', revision: 'ba948ece5b*'
    System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.2.0-33-generic', java.version: '19.0.2'
    Driver info: org.openqa.selenium.chrome.ChromeDriver
    Command: [f3bb7eeae4a8f8c7e9b710c5b81d76ac, get {url=https://whoer.net/}]
    Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 117.0.5938.132, chrome: {chromedriverVersion: 117.0.5938.92 (67649b10b92b..., userDataDir: /tmp/.org.chromium.Chromium...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:37833}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: linux, proxy: Proxy(manual, socks=proxy-f..., se:cdp: ws://localhost:37833/devtoo..., se:cdpVersion: 117.0.5938.132, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
    Session ID: f3bb7eeae4a8f8c7e9b710c5b81d76ac
            at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67)
            at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
            at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:484)
            at org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.createException(W3CHttpResponseCodec.java:200)
            at org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:133)
            at org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:52)
            at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:191)
            at org.openqa.selenium.remote.service.DriverCommandExecutor.invokeExecute(DriverCommandExecutor.java:196)
            at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:171)
            at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:602)
            at org.openqa.selenium.remote.RemoteWebDriver.get(RemoteWebDriver.java:304)
            at Tinder.main(Tinder.java:21)

    > Task :run FAILED

    FAILURE: Build failed with an exception.

Even chagpt and StackOverflow can't help

Is there documentation I can read

TheGreatSimo commented 1 year ago

I forget to say it's a 4k proxy in the fist place so it's a 4g socks5

titusfortner commented 1 year ago

I do not know proxies very well. You can try turning on logging and see if there's something getting sent that doesn't look right: https://www.selenium.dev/documentation/webdriver/troubleshooting/logging/

diemol commented 11 months ago

I will close this as we did not get more information from the OP.

TheGreatSimo commented 11 months ago

Thank you but I think I did get all the info

titusfortner commented 11 months ago

Does that mean you got it working, or not? If not, please submit the logs obtained based on the above documentation.

TheGreatSimo commented 11 months ago

Yes I got it working but it was a very specific case I don't think other people will encounter the same issue I'm still a beginner so if you notice anything missing please let me and I will address it

github-actions[bot] commented 10 months ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.