search

SergeyPirogov / webdriver_manager

Apache License 2.0
2.07k stars 461 forks source link

Webdrivers crash in multi process parallel test runs #631

Open derdualist opened 1 year ago

derdualist commented 1 year ago

at first I opened this issue with the Selenium project, but it seems to be related with the webdriver-manager module. https://github.com/SeleniumHQ/selenium/issues/12751

As far as I can tell there are 2 problems here, that come together:

Maybe after confirmation, this can be split into a bug and a feature ticket.

I created a gist, that can reproduce the issue:

parallel_selenium_tests.py

output:

$ python parallel_selenium_tests.py -n2 -m webdriver-manager -b chrome
[WDM LOG] ====== WebDriver manager ======
[WDM LOG] ====== WebDriver manager ======
[WDM LOG] Get LATEST chromedriver version for google-chrome
[WDM LOG] Get LATEST chromedriver version for google-chrome
[WDM LOG] Get LATEST chromedriver version for google-chrome
[WDM LOG] There is no [mac64] chromedriver "117.0.5938.92" for browser google-chrome "117.0.5938.92" in cache
[WDM LOG] Get LATEST chromedriver version for google-chrome
[WDM LOG] There is no [mac64] chromedriver "117.0.5938.92" for browser google-chrome "117.0.5938.92" in cache
[WDM LOG] Get LATEST chromedriver version for google-chrome
[WDM LOG] Get LATEST chromedriver version for google-chrome
[WDM LOG] WebDriver version 117.0.5938.92 selected
[WDM LOG] Modern chrome version https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/117.0.5938.92/mac-arm64/chromedriver-mac-arm64.zip
[WDM LOG] About to download new driver from https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/117.0.5938.92/mac-arm64/chromedriver-mac-arm64.zip
[WDM LOG] WebDriver version 117.0.5938.92 selected
[WDM LOG] Modern chrome version https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/117.0.5938.92/mac-arm64/chromedriver-mac-arm64.zip
[WDM LOG] About to download new driver from https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/117.0.5938.92/mac-arm64/chromedriver-mac-arm64.zip
[WDM LOG] Driver downloading response is 200
[WDM LOG] Driver downloading response is 200
[WDM LOG] Get LATEST chromedriver version for google-chrome
[WDM LOG] Driver has been saved in cache [/Users/tobias.lehmann/.wdm/drivers/chromedriver/mac64/117.0.5938.92]
[WDM LOG] Get LATEST chromedriver version for google-chrome
[WDM LOG] Driver has been saved in cache [/Users/tobias.lehmann/.wdm/drivers/chromedriver/mac64/117.0.5938.92]
Process Process-2:
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.10/3.10.12_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/opt/homebrew/Cellar/python@3.10/3.10.12_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/tobias.lehmann/projects/ta/driver_management/parallel_driver.py", line 88, in test
    driver = create_driver(browser, executable_path, options)
  File "/Users/tobias.lehmann/projects/ta/driver_management/parallel_driver.py", line 75, in create_driver
    return {
  File "/Users/tobias.lehmann/projects/ta/driver_management/parallel_driver.py", line 76, in <lambda>
    'chrome': lambda: Chrome(service=ChromeService(executable_path=executable_path), options=options),
  File "/Users/tobias.lehmann/Library/Python/3.10/lib/python/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
    super().__init__(
  File "/Users/tobias.lehmann/Library/Python/3.10/lib/python/site-packages/selenium/webdriver/chromium/webdriver.py", line 53, in __init__
    self.service.start()
  File "/Users/tobias.lehmann/Library/Python/3.10/lib/python/site-packages/selenium/webdriver/common/service.py", line 109, in start
    self.assert_process_still_running()
  File "/Users/tobias.lehmann/Library/Python/3.10/lib/python/site-packages/selenium/webdriver/common/service.py", line 122, in assert_process_still_running
    raise WebDriverException(f"Service {self._path} unexpectedly exited. Status code was: {return_code}")
selenium.common.exceptions.WebDriverException: Message: Service /Users/tobias.lehmann/.wdm/drivers/chromedriver/mac64/117.0.5938.92/chromedriver-mac-arm64/chromedriver unexpectedly exited. Status code was: -9

The Internet
Broken Images

The output The Internet Broken Images comes from the test case in the one succeeding process.

Used versions

Content of cache

The correct driver key is inside the cache:

{
...
"mac64_chromedriver_117.0.5938.92_for_117.0.5938.92": {
        "timestamp": "29/09/2023",
        "binary_path": "/Users/USER/.wdm/drivers/chromedriver/mac64/117.0.5938.92/chromedriver-mac-arm64/chromedriver"
    }
}

Output of browser version command from os_manager.py:

$ /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --version
Google Chrome 117.0.5938.92
david-engelmann commented 1 year ago

In my experience, sharing chrome binaries across threads was a huge pain. I found it easiest to set a maximum amount of chrome binaries to download, based on the expected load and storage limitations. I cache the paths when an install is actually used. when a thread gets locked out of one chrome binary path it just tries the next until it hits one that isn't being used with a thread.

  1. Check if chrome binary path is set in the cache, if a path is cached, check if its executable. If path isnt set call the typical install() and cache the path returned.
  2. Try to utilize the chrome binaries in the path, if it fails repeat step 1 with the next cache key until you can create a driver
  3. Proceed with your stuff
david-engelmann commented 1 year ago

Trying to reproduce this issue with this repository- https://github.com/david-engelmann/selenium_sadness.

frichtarik commented 1 year ago

i'm using pytest session fixture to download driver before the tests will start so there is no issue with multiple tests writing into the same file, because it's already there when they run 

@pytest.fixture(scope="session", autouse=True)
def suite(request):
    ChromeDriverManager().install()