Voyz / ibeam

IBeam is an authentication and maintenance tool used for the Interactive Brokers Client Portal Web API Gateway.
Apache License 2.0
543 stars 111 forks source link

Selenium crash with rebuilt docker image #197

Closed pmcenery closed 3 months ago

pmcenery commented 3 months ago

Describe the bug I was hoping to update all the packages to the latest and test them out before submitting a PR with the changes. This ran aground when I couldnt even rebuild the master branch using the existing Dockerfile. I appear to have run into this issue, as I'm building using docker on macOS. I've tried all sorts of things from specifying a bigger default-dev-shm in the docker settings, but for some or other reason, I cannot run the rebuilt image. The dockerhub one however seems to run just fine 🀷

To Reproduce Steps to reproduce the behavior:

  1. git clone
  2. cd ibeam and make sure env.list is populated...
  3. docker build -t voyz/ibeam .
  4. docker compose up

I get this output

docker compose up
[+] Running 1/0
 βœ” Container ibeam  Created                                                                                                                                                                    0.1s
Attaching to ibeam
ibeam  | 2024-06-25 08:38:33,100|I| ############ Starting IBeam version 0.5.3 ############
ibeam  | 2024-06-25 08:38:33,112|I| Secrets source: env
ibeam  | 2024-06-25 08:38:33,117|I| Health server started at port=5001
ibeam  | 2024-06-25 08:38:33,120|I| Configuration:
ibeam  | {'INPUTS_DIR': '/srv/inputs/', 'OUTPUTS_DIR': '/srv/outputs', 'GATEWAY_DIR': '/srv/clientportal.gw', 'CHROME_DRIVER_PATH': '/usr/bin/chromedriver', 'GATEWAY_STARTUP': 20, 'GATEWAY_PROCESS_MATCH': 'ibgroup.web.core.clientportal.gw.GatewayStart', 'MAINTENANCE_INTERVAL': 60, 'SPAWN_NEW_PROCESSES': False, 'LOG_LEVEL': 'INFO', 'LOG_TO_FILE': True, 'LOG_FORMAT': '%(asctime)s|%(levelname)-.1s| %(message)s', 'REQUEST_RETRIES': 2, 'REQUEST_TIMEOUT': 15, 'RESTART_FAILED_SESSIONS': True, 'RESTART_WAIT': 15, 'REAUTHENTICATE_WAIT': 15, 'HEALTH_SERVER_PORT': 5001, 'SECRETS_SOURCE': 'env', 'GCP_SECRETS_URL': None, 'START_ACTIVE': True, 'GATEWAY_BASE_URL': 'https://localhost:5000', 'ROUTE_AUTH': '/sso/Login?forwardTo=22&RL=1&ip2loc=on', 'ROUTE_VALIDATE': '/v1/portal/sso/validate', 'ROUTE_REAUTHENTICATE': '/v1/portal/iserver/reauthenticate?force=true', 'ROUTE_INITIALISE': '/v1/api/iserver/auth/ssodh/init', 'ROUTE_AUTH_STATUS': '/v1/api/iserver/auth/status', 'ROUTE_TICKLE': '/v1/api/tickle', 'ROUTE_LOGOUT': '/v1/api/logout', 'USER_NAME_EL': None, 'PASSWORD_EL': 'NAME@@password', 'SUBMIT_EL': 'CSS_SELECTOR@@.btn.btn-lg.btn-primary', 'ERROR_EL': None, 'SUCCESS_EL_TEXT': 'TAG_NAME@@Client login succeeds', 'OAUTH_TIMEOUT': 15, 'PAGE_LOAD_TIMEOUT': 15, 'ERROR_SCREENSHOTS': False, 'MAX_FAILED_AUTH': 5, 'MIN_PRESUBMIT_BUFFER': 5, 'MAX_PRESUBMIT_BUFFER': 30, 'MAX_IMMEDIATE_ATTEMPTS': 10, 'IBKEY_PROMO_EL_CLASS': 'CLASS_NAME@@ibkey-promo-skip', 'AUTHENTICATION_STRATEGY': 'B', 'MAX_STATUS_CHECK_RETRIES': 120, 'MAX_REAUTHENTICATE_RETRIES': 3, 'UI_SCALING': 1.0, 'TWO_FA_EL_ID': 'ID@@twofactbase', 'TWO_FA_NOTIFICATION_EL': 'CLASS_NAME@@login-step-notification', 'TWO_FA_INPUT_EL_ID': 'ID@@chlginput', 'TWO_FA_HANDLER': None, 'STRICT_TWO_FA_CODE': True, 'TWO_FA_SELECT_EL_ID': 'ID@@sf_select', 'TWO_FA_SELECT_TARGET': 'IB Key', 'CUSTOM_TWO_FA_HANDLER': 'custom_two_fa_handler.CustomTwoFaHandler'}
ibeam  | 2024-06-25 08:38:33,124|I| Gateway not found, starting new one...
ibeam  | 2024-06-25 08:38:33,131|I| Note that the Gateway log below may display "Open https://localhost:[PORT] to login" - ignore this command.
ibeam  | 2024-06-25 08:38:33,132|I| Starting Gateway as Linux process with params: ['bash', 'bin/run.sh', 'root/conf.yaml']
ibeam  | running
ibeam  |  runtime path : root:dist/ibgroup.web.core.iblink.router.clientportal.gw.jar:build/lib/runtime/*
ibeam  |  config file  : root/conf.yaml
ibeam  | 2024-06-25 08:38:33,278|I| Gateway started with pids: [13]
ibeam  | 2024-06-25 08:38:33,289|I| Cannot ping Gateway. Retrying for another 20 seconds
ibeam  | 2024-06-25 08:38:34,293|I| Cannot ping Gateway. Retrying for another 19 seconds
ibeam  | WARNING: An illegal reflective access operation has occurred
ibeam  | WARNING: Illegal reflective access by io.netty.util.internal.ReflectionUtil (file:/srv/clientportal.gw/build/lib/runtime/netty-common-4.1.15.Final.jar) to constructor java.nio.DirectByteBuffer(long,int)
ibeam  | WARNING: Please consider reporting this to the maintainers of io.netty.util.internal.ReflectionUtil
ibeam  | WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
ibeam  | WARNING: All illegal access operations will be denied in a future release
ibeam  | 2024-06-25 08:38:35,295|I| Cannot ping Gateway. Retrying for another 18 seconds
ibeam  |  -> mount demo on /demo
ibeam  | Java Version: 11.0.23
ibeam  | ****************************************************
ibeam  | version: a27ed42161ad96c53e715ca5c5e3e3fa4cff5262 Mon, 24 Apr 2023 15:41:53 -0400
ibeam  | ****************************************************
ibeam  | This is the Client Portal Gateway
ibeam  | for any issues, please contact api@ibkr.com
ibeam  | and include a copy of your logs
ibeam  | ****************************************************
ibeam  | https://www.interactivebrokers.com/api/doc.html
ibeam  |
ibeam  | ****************************************************
ibeam  | Open https://localhost:5000 to login
ibeam  |
ibeam  | App demo is available after you login under: https://localhost:5000/demo#/
ibeam  |
ibeam  |
ibeam  | 2024-06-25 08:38:37,484|I| Gateway connection established
ibeam  | 2024-06-25 08:38:37,629|I| NO SESSION Status(running=True, session=False, connected=False, authenticated=False, competing=False, collision=False, session_id=None, server_name=None, server_version=None, expires=None)
ibeam  | 2024-06-25 08:38:37,629|I| Authentication strategy: "B"
ibeam  | 2024-06-25 08:38:37,630|I| No active sessions, logging in...
ibeam  | 2024-06-25 08:38:37,630|I| Loading auth webpage at https://localhost:5000/sso/Login?forwardTo=22&RL=1&ip2loc=on
ibeam  | 2024-06-25 08:38:39,329|E| Error encountered during authentication
ibeam  | Exception:
ibeam  |   File "/srv/ibeam/ibeam_starter.py", line 172, in <module>
ibeam  |     success, shutdown, status = client.start_and_authenticate()
ibeam  |   File "/srv/ibeam/src/gateway_client.py", line 62, in start_and_authenticate
ibeam  |     success, shutdown, status = self.strategy_handler.try_authenticating(request_retries=request_retries)
ibeam  |   File "/srv/ibeam/src/handlers/strategy_handler.py", line 85, in try_authenticating
ibeam  |     return self._authentication_strategy_B(status, request_retries)
ibeam  |   File "/srv/ibeam/src/handlers/strategy_handler.py", line 140, in _authentication_strategy_B
ibeam  |     return self._log_in(status)
ibeam  |   File "/srv/ibeam/src/handlers/strategy_handler.py", line 151, in _log_in
ibeam  |     success, shutdown = self.login_handler.login()
ibeam  |   File "/srv/ibeam/src/handlers/login_handler.py", line 430, in login
ibeam  |     driver, display = start_up_browser(self.driver_factory)
ibeam  |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ibeam  |   File "/srv/ibeam/src/login/driver.py", line 152, in start_up_browser
ibeam  |     driver = driver_factory.new_driver()
ibeam  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ibeam  |   File "/srv/ibeam/src/login/driver.py", line 143, in new_driver
ibeam  |     return start_driver(driver_path=driver_path, name=name, headless=headless, incognito=incognito, ui_scaling=ui_scaling, page_load_timeout=page_load_timeout)
ibeam  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ibeam  |   File "/srv/ibeam/src/login/driver.py", line 78, in start_driver
ibeam  |     raise e
ibeam  |   File "/srv/ibeam/src/login/driver.py", line 66, in start_driver
ibeam  |     driver = _new_chrome_driver(driver_path=driver_path, name=name, headless=headless, incognito=incognito, ui_scaling=ui_scaling)
ibeam  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ibeam  |   File "/srv/ibeam/src/login/driver.py", line 47, in _new_chrome_driver
ibeam  |     driver = webdriver.Chrome(options=options, service=service)
ibeam  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ibeam  |   File "/opt/venv/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
ibeam  |     super().__init__(
ibeam  |   File "/opt/venv/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 56, in __init__
ibeam  |     super().__init__(
ibeam  |   File "/opt/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 205, in __init__
ibeam  |     self.start_session(capabilities)
ibeam  |   File "/opt/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 289, in start_session
ibeam  |     response = self.execute(Command.NEW_SESSION, caps)["value"]
ibeam  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ibeam  |   File "/opt/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 344, in execute
ibeam  |     self.error_handler.check_response(response)
ibeam  |   File "/opt/venv/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
ibeam  |     raise exception_class(message, screen, stacktrace)
ibeam  |
ibeam  |   <class 'selenium.common.exceptions.SessionNotCreatedException'> Message: session not created: Chrome failed to start: exited normally.
ibeam  |   (chrome not reachable)
ibeam  |   (The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
ibeam  | Stacktrace:
ibeam  | #0 0x55d33c495233 <unknown>
ibeam  | #1 0x55d33c15f0fe <unknown>
ibeam  | #2 0x55d33c193d7b <unknown>
ibeam  | #3 0x55d33c18ee89 <unknown>
ibeam  | #4 0x55d33c1e1370 <unknown>
ibeam  | #5 0x55d33c1e0957 <unknown>
ibeam  | #6 0x55d33c1d61a3 <unknown>
ibeam  | #7 0x55d33c19b13e <unknown>
ibeam  | #8 0x55d33c19c712 <unknown>
ibeam  | #9 0x55d33c46a534 <unknown>
ibeam  | #10 0x55d33c46d759 <unknown>
ibeam  | #11 0x55d33c46d238 <unknown>
ibeam  | #12 0x55d33c46dc05 <unknown>
ibeam  | #13 0x55d33c45cabf <unknown>
ibeam  | #14 0x55d33c46df8e <unknown>
ibeam  | #15 0x55d33c446c86 <unknown>
ibeam  | #16 0x55d33c4863e5 <unknown>
ibeam  | #17 0x55d33c4865cb <unknown>
ibeam  | #18 0x55d33c4947bf <unknown>
ibeam  | #19 0x7f1dc3c67ea7 start_thread
ibeam  |
ibeam  |
ibeam  | 2024-06-25 08:38:39,330|I| Cleaning up the resources. Display: None | Driver: None
ibeam  | 2024-06-25 08:38:39,331|I| Logging in failed

Expected behavior Running as normal

Environment IBeam version: master Docker image or standalone: Built using master Python version (standalone users only): N/A OS: macOS Sonoma 14.5

Additional context I'm not sure if something has changed in the package build process - i.e. newer apt-installed packages that breaks it, or if its something local to my build process? I build docker images all the time with no issue, so wouldn't have thought it was build host specific.

Suggest a Fix Is a build using master branch still working?

Voyz commented 3 months ago

Hey @pmcenery many thanks for describing your situation in detail (and wanting to contribute to IBeam!) πŸ‘

Is a build using master branch still working?

Yes, I have built it just two days ago as of writing this: https://hub.docker.com/layers/voyz/ibeam/0.5.4-rc3/images/sha256-39cb65946adba6e1dd36db4b3e54bb927e6fda21673762d7a2e324368e0fdb28?context=repo

I'm not sure if something has changed in the package build process - i.e. newer apt-installed packages that breaks it, or if its something local to my build process?

I'm imagining this would be something related to MacOS - I briefly remember issues happening to some other users on MacOS that didn't exist on other OSs. I can't find the specific Issues at the moment though.


Since I don't have an Apple device I cannot help you debugging this directly from my end. If it is of any help - I can see that the build fails when we attempt to access the login page and create a new webdriver:

# File "/srv/ibeam/src/login/driver.py", line 47
driver = webdriver.Chrome(options=options, service=service)

I'd suggest trying to create a Chrome Webdriver using the code in _new_chrome_driver and see if that works for you on its own. I've simplified the code for you so it should be ready to execute on its own:

def _new_chrome_driver(driver_path, name: str = 'default', headless: bool = True, incognito: bool = True, ui_scaling: float = 1) -> webdriver.Chrome:
    """Creates a new chrome driver."""
    options = webdriver.ChromeOptions()
    if headless:
        options.add_argument('--headless')
        options.add_argument('--disable-gpu')
    options.add_argument('--no-sandbox')
    if incognito:
        options.add_argument("--incognito")  # this allows 2FA method to be selected every time
    options.add_argument('--ignore-ssl-errors=yes')
    options.add_argument('--ignore-certificate-errors')
    options.add_argument(f'--remote-debugging-port={9222}')
    options.add_argument('--useAutomationExtension=false')
    options.add_argument('--disable-extensions')
    options.add_argument('--dns-prefetch-disable')
    options.add_argument('--disable-features=VizDisplayCompositor')
    options.add_argument(f"--force-device-scale-factor={ui_scaling}")
    options.add_argument(f'--user-data-dir={tempfile.gettempdir()}/ibeam-chrome-{name}')
    service = Service(executable_path=driver_path)
    driver = webdriver.Chrome(options=options, service=service)
    if driver is None:
        print('Unable to create a new chrome driver.')

    return driver

Let me know how that goes

pmcenery commented 3 months ago

Thanks for the detailed and prompt response @Voyz

I battled with this one for quite some time and then found a Docker desktop update. Installed and restarted and now it works as I'd expect! I've never seen anything like it, but at least it explains the rather odd behaviour!

Voyz commented 3 months ago

@pmcenery great, happy to hear you've solved it! Thanks for leaving the detailed guide on how to deal with the issue you've found πŸ‘πŸ‘