Since 3.14 I can't use proxy anymore

[aluneau]

I use proxy (on windows and Linux) to contact my selenium grid. Before 3.14 I used environnement variable http_proxy but with 3.14 it does not work anymore. It seems to be the swap from httplib to urllib3.

For now I downgraded Selenium 3.14 to 3.13 but do you think it is possible to include proxy in 3.14?

[AutomatedTester]

THis bug is in no way actionable...

For issues please provide a concise reproducible test case and describe what results you are seeing and what results you expect.

See CONTRIBUTING.md

[aluneau]

Meta -

OS:
Windows and Linux Selenium Version:
3.12 Browser:
GRID

Browser Version:
Latest

Client: Selenium Python 3.14

Expected Behavior -

export http_proxy=http://myproxy:9000
expot HTTP_PROXY=http://myproxy:9000
robot test.txt

RobotFramework and SeleniumLibrary should use proxy and it is not the case with Selenium3.14 client for python. It's the case with 3.13

Actual Behavior -

export http_proxy=http://myproxy:9000
expot HTTP_PROXY=http://myproxy:9000
robot test.txt

RobotFramework and SeleniumLibrary does not use proxy settings. I'm sure because I've tested with a proxy on my machine that was logging everything.

Steps to reproduce -

export http_proxy=http://myproxy:9000
expot HTTP_PROXY=http://myproxy:9000
robot test.txt

Content of test.txt:

*** Settings ***

Library  SeleniumLibrary
Library  RequestsLibrary
Library  OperatingSystem
Library  Collections

Test Setup  Open test browser
Test Teardown  Close test browser

*** Variables ***
${BROWSER}  firefox
${VERSION}  latest
${SELENIUM}   http://selenium-grid/wd/hub
*** Test Cases ***
test google
    Go to  http://google.fr
    Capture Page Screenshot

*** Keywords ***
Open test browser
    ${desired capabilities}=  Evaluate  {'browserName': '${BROWSER}', 'version': '${VERSION}'}
    ${executor}  Evaluate  sys.modules['selenium.webdriver'].remote.remote_connection.RemoteConnection("${SELENIUM}", resolve_ip=False)  sys, selenium.webdriver
    Create Webdriver  Remote  command_executor=${executor}  desired_capabilities=${desired capabilities}

Close test browser
    Close all browsers

[pekkaklarck]

@bloudman I would assume Selenium maintainers would prefer an example using Selenium only, not external tools that use Selenium internally.

[lmtierney]

Yes, please provide an example with selenium library code. As it is right now, it would be way too much work for us to setup your scenario

[lmtierney]

Keep in mind SeleniumLibrary is not from the selenium org

[cgoldberg]

more info:

urllib from standard library transparently supports [unauthenticated] proxies by setting the http_proxy environment variable (Unix or Windows environments only, not Mac). see: https://docs.python.org/2/library/urllib.html#urllib.urlopen

Apparantly, Urllib3 doesn't consider this environment variable by default.

[pekkaklarck]

Not supporting the de-facto standard http_proxy/https_proxy/no_proxy environment variables would be a big backwards incompatible change. AFAIK requests that uses urllib3 supports them, so it ought to be possible to add this support on top of it.

[aluneau]

Here is an example with only python: main.py:

from selenium import webdriver
from selenium.webdriver.remote.remote_connection import RemoteConnection

selenium_grid_url = "http://selenium-hub/wd/hub"
# Create a desired capabilities object as a starting point.
capabilities = capabilities = {'browserName': 'chrome', 'version': 'latest'}
executor = RemoteConnection(selenium_grid_url, resolve_ip=False)
driver = webdriver.Remote(desired_capabilities=capabilities,
                          command_executor=executor)

driver.get("https://google.fr")
print(driver.current_url)

For the proxy I used this: https://github.com/willthames/python-logging-proxy that will log everything.

[cgoldberg]

so it ought to be possible to add this support on top of it

it's certainly possible, just not implemented

[james-simpson-ST]

We are experiencing the same issue where we used to set the environment variable in python and the selenium remote webdriver would honor those settings. Since the 3.14 change to urllib3 these are no longer used.

import os os.environ["HTTP_PROXY"] = PROXY os.environ["HTTPS_PROXY"] = PROXY

Is there an alternative way of specifying a proxy to use for the new urllib3 implementation? Ideally the environment variables should be used but as a work around is there an option within the capabilities?

[ghost]

I faced this problem too. I had to downgrade selenium version from 3.14.0 to 3.13.0.

[james-simpson-ST]

I have seen that this has been fixed in 4.0.0.a6.post1 https://github.com/SeleniumHQ/selenium/pull/8297

Are there any plans to back-port this fix to 3.141?