seleniumbase / SeleniumBase

📊 Python's all-in-one framework for web crawling, scraping, testing, and reporting. Supports pytest. UC Mode provides stealth. Includes many tools.
https://seleniumbase.io
MIT License
4.46k stars 910 forks source link

Failing Docker tests when running on M2 mac #2690

Closed evan1108 closed 2 months ago

evan1108 commented 2 months ago

I cloned the SeleniumBase repo, built the docker image docker build --platform linux/amd64 -t seleniumbase ., and ran the test docker run --platform linux/amd64 seleniumbase ./run_docker_test_in_chrome.sh I get an error saying Chrome has failed to start/crashed. I haven't made any changes to the Dockerfile. I've tried running the test without the --platform tag in the docker run command and that fails as well. Running on an M2 mac.

***** SeleniumBase Docker Machine *****
Running example SeleniumBase test from Docker with headless Chrome...
============================= test session starts ==============================
platform linux -- Python 3.8.0, pytest-8.1.1, pluggy-1.4.0
rootdir: /SeleniumBase/examples
configfile: pytest.ini
plugins: seleniumbase-4.25.3, rerunfailures-14.0, html-2.0.1, metadata-3.1.1, cov-5.0.0, ordering-0.6, xdist-3.5.0
collected 1 item

my_first_test.py 
Warning: chromedriver update needed. Getting it now:

*** chromedriver to download = 123.0.6312.122 (Latest Stable) 

Downloading chromedriver-linux64.zip from:
https://storage.googleapis.com/chrome-for-testing-public/123.0.6312.122/linux64/chromedriver-linux64.zip ...
Download Complete!

Extracting ['chromedriver'] from chromedriver-linux64.zip ...
Unzip Complete!

The file [chromedriver] was saved to:
/usr/local/lib/python3.8/dist-packages/seleniumbase/drivers/chromedriver

Making [chromedriver 123.0.6312.122] executable ...
[chromedriver 123.0.6312.122] is now ready for use!

F```

```>       raise exception_class(message, screen, stacktrace)
E       selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome failed to start: exited normally.
E         (session not created: DevToolsActivePort file doesn't exist)
E         (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
E       Stacktrace:
E       #0 0x00400077a863 <unknown>
E       #1 0x0040004708c6 <unknown>
E       #2 0x0040004a4d34 <unknown>
E       #3 0x0040004a0d3d <unknown>
E       #4 0x0040004e9aed <unknown>
E       #5 0x0040004dd343 <unknown>
E       #6 0x0040004ae593 <unknown>
E       #7 0x0040004aef5e <unknown>
E       #8 0x00400073e84b <unknown>
E       #9 0x0040007427a5 <unknown>
E       #10 0x00400072c571 <unknown>
E       #11 0x004000743332 <unknown>
E       #12 0x00400071187f <unknown>
E       #13 0x004000769728 <unknown>
E       #14 0x0040007698fb <unknown>
E       #15 0x0040007799b4 <unknown>
E       #16 0x004002bc96db start_thread

/usr/local/lib/python3.8/dist-packages/selenium/webdriver/remote/errorhandler.py:229: SessionNotCreatedException

During handling of the above exception, another exception occurred:
/usr/local/lib/python3.8/dist-packages/seleniumbase/fixtures/base_case.py:14645: in setUp
    self.driver = self.get_new_driver(
/usr/local/lib/python3.8/dist-packages/seleniumbase/fixtures/base_case.py:4018: in get_new_driver
    new_driver = browser_launcher.get_driver(
/usr/local/lib/python3.8/dist-packages/seleniumbase/core/browser_launcher.py:1632: in get_driver
    return get_local_driver(
/usr/local/lib/python3.8/dist-packages/seleniumbase/core/browser_launcher.py:3879: in get_local_driver
    driver = webdriver.Chrome(
/usr/local/lib/python3.8/dist-packages/selenium/webdriver/chrome/webdriver.py:45: in __init__
    super().__init__(
/usr/local/lib/python3.8/dist-packages/selenium/webdriver/chromium/webdriver.py:61: in __init__
    super().__init__(command_executor=executor, options=options)
/usr/local/lib/python3.8/dist-packages/selenium/webdriver/remote/webdriver.py:208: in __init__
    self.start_session(capabilities)
/usr/local/lib/python3.8/dist-packages/selenium/webdriver/remote/webdriver.py:292: in start_session
    response = self.execute(Command.NEW_SESSION, caps)["value"]
/usr/local/lib/python3.8/dist-packages/selenium/webdriver/remote/webdriver.py:347: in execute
    self.error_handler.check_response(response)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <selenium.webdriver.remote.errorhandler.ErrorHandler object at 0x4009f0c610>
response = {'status': 500, 'value': '{"value":{"error":"session not created","message":"session not created: DevToolsActivePort f...wn>\\n#15 0x0040007698fb \\u003Cunknown>\\n#16 0x0040007799b4 \\u003Cunknown>\\n#17 0x004002bc96db start_thread\\n"}}'}
evan1108 commented 2 months ago

I'm attempting to create my own Docker image with SeleniumBase but I'm getting the following error:

File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/common/service.py", line 115, in assert_process_still_running
raise WebDriverException(f"Service {self._path} unexpectedly exited. Status code was: {return_code}")
selenium.common.exceptions.WebDriverException: Message: Service /root/.cache/selenium/chromedriver/linux64/123.0.6312.122/chromedriver unexpectedly exited. Status code was: 127

This is after copying most parts of the standard Dockerfile (minus the chrome and webdriver installs as I understand Seleniumbase / Selenium Manager handles that)

Dockerfile:

FROM ubuntu:20.04

ADD . /src

RUN apt update
RUN apt-get update && apt-get install -y --no-install-recommends apt-utils

#=======================================
# Install Python and Basic Python Tools
#=======================================
RUN apt-get -o Acquire::Check-Valid-Until=false -o Acquire::Check-Date=false update
RUN apt-get install -y python3 python3-pip python3-setuptools python3-dev python-distribute
RUN alias python=python3
RUN echo "alias python=python3" >> ~/.bashrc

#=================================
# Install Bash Command Line Tools
#=================================
RUN apt-get -qy --no-install-recommends install \
    sudo \
    unzip \
    wget \
    curl \
    libxi6 \
    libgconf-2-4 \
    vim \
    xvfb \
    libpq-dev \
  && rm -rf /var/lib/apt/lists/*

#===========================
# Configure Virtual Display
#===========================
RUN set -e
RUN echo "Starting X virtual framebuffer (Xvfb) in background..."
RUN Xvfb -ac :99 -screen 0 1280x1024x16 > /dev/null 2>&1 &
RUN export DISPLAY=:99
RUN exec "$@"
#
#=======================
# Update Python Version
#=======================
RUN apt-get update -y
RUN apt-get -qy --no-install-recommends install python3.8
RUN rm /usr/bin/python3
RUN ln -s python3.8 /usr/bin/python3

RUN pip install -r ./src/requirements.txt

CMD python3 ./src/manage.py migrate && python3 ./src/manage.py selenium_worker

The entrypoint command is executing this:

class Command(BaseCommand):
    def handle(self, *args, **options):
        with SB("dummy") as sb:
            print("Initializing global web driver")
            settings.GLOBAL_WEB_DRIVER = sb

Which is what I see failing in the stack trace:

Operations to perform:
  Apply all migrations: admin, auth, contenttypes, django_rq, sessions, socials
Running migrations:
  No migrations to apply.
=========================== {manage.py:22:SB} starts ===========================

Warning: chromedriver not found. Getting it now:

*** chromedriver to download = 123.0.6312.122 (Latest Stable) 

Downloading chromedriver-linux64.zip from:
https://storage.googleapis.com/chrome-for-testing-public/123.0.6312.122/linux64/chromedriver-linux64.zip ...
Download Complete!

Extracting ['chromedriver'] from chromedriver-linux64.zip ...
Unzip Complete!

The file [chromedriver] was saved to:
/usr/local/lib/python3.8/dist-packages/seleniumbase/drivers/chromedriver

Making [chromedriver 123.0.6312.122] executable ...
[chromedriver 123.0.6312.122] is now ready for use!

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/seleniumbase/core/browser_launcher.py", line 3797, in get_local_driver
    driver = webdriver.Chrome(options=chrome_options)
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
    super().__init__(
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/chromium/webdriver.py", line 50, in __init__
    self.service.start()
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/common/service.py", line 102, in start
    self.assert_process_still_running()
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/common/service.py", line 115, in assert_process_still_running
    raise WebDriverException(f"Service {self._path} unexpectedly exited. Status code was: {return_code}")
selenium.common.exceptions.WebDriverException: Message: Service /root/.cache/selenium/chromedriver/linux64/123.0.6312.122/chromedriver unexpectedly exited. Status code was: 127

Appreciate any direction you can provide!

mdmintz commented 2 months ago

I'm still working on updating the included Dockerfile. Maybe you'll get it working before I do.

mdmintz commented 2 months ago

This was resolved in 4.25.4 - https://github.com/seleniumbase/SeleniumBase/releases/tag/v4.25.4

evan1108 commented 2 months ago

You are incredible @mdmintz! Confirmed it's working on my end. Thanks for the quick attention and fix.