Closed helpdeskdan closed 3 months ago
jdholtz
commented
3 months ago Hey @helpdeskdan, thanks for filing this issue. Users have run into this issue before, but it’s been in Docker. What OS are you running this on? It may be worth it to try Google Chrome instead of Chromium to see if it fixes the issue.
Is the issue repeatable or has it just happened this one time?
helpdeskdan
commented
3 months ago Ubuntu 22.04 in miniconda. I was going to try the docker next, instead I'll try real chrome. As it happens at check in, it is difficult to recreate. I'm not exactly a frequent flier, but I do have one flight coming up later this week.
jdholtz
commented
3 months ago As it happens at check in, it is difficult to recreate
This error is more general, so even though it happened right before check-in, the refreshing headers process (which is where it failed) is run right when the script is started (with reservations, not logins).
helpdeskdan
commented
3 months ago That ran fine, so perhaps the problem is intermittent? Maybe catch the exception and try again?
ne0ark
commented
3 months ago As it happens at check in, it is difficult to recreate
This error is more general, so even though it happened right before check-in, the refreshing headers process (which is where it failed) is run right when the script is started (with reservations, not logins).
How do I change the order so it starts with reservations in docker?
2024-08-12 19:10:15 DEBUG Process-2[reservation_monitor:60]: Lock acquired
2024-08-12 19:10:15 DEBUG Process-2[reservation_monitor:198]: Retrieving reservations for account
2024-08-12 19:10:15 DEBUG Process-2[webdriver:128]: Starting webdriver for current session
Traceback (most recent call last):
File "/usr/local/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/local/lib/python3.12/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/app/lib/reservation_monitor.py", line 44, in monitor
self._monitor()
File "/app/lib/reservation_monitor.py", line 62, in _monitor
should_exit = self._check()
^^^^^^^^^^^^^
File "/app/lib/reservation_monitor.py", line 180, in _check
reservations, skip_scheduling = self._get_reservations()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/lib/reservation_monitor.py", line 202, in _get_reservations
reservations = webdriver.get_reservations(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/lib/webdriver.py", line 99, in get_reservations
driver = self._get_driver()
^^^^^^^^^^^^^^^^^^
File "/app/lib/webdriver.py", line 137, in _get_driver
driver = Driver(
^^^^^^^
File "/app/.local/lib/python3.12/site-packages/seleniumbase/plugins/driver_manager.py", line 529, in Driver
driver = browser_launcher.get_driver(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.local/lib/python3.12/site-packages/seleniumbase/core/browser_launcher.py", line 1657, in get_driver
return get_local_driver(
^^^^^^^^^^^^^^^^^
File "/app/.local/lib/python3.12/site-packages/seleniumbase/core/browser_launcher.py", line 3600, in get_local_driver
driver = undetected.Chrome(
^^^^^^^^^^^^^^^^^^
File "/app/.local/lib/python3.12/site-packages/seleniumbase/undetected/__init__.py", line 312, in __init__
super().__init__(options=options, service=service_)
File "/app/.local/lib/python3.12/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
super().__init__(
File "/app/.local/lib/python3.12/site-packages/selenium/webdriver/chromium/webdriver.py", line 66, in __init__
super().__init__(command_executor=executor, options=options)
File "/app/.local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 208, in __init__
self.start_session(capabilities)
File "/app/.local/lib/python3.12/site-packages/seleniumbase/undetected/__init__.py", line 470, in start_session
super().start_session(capabilities)
File "/app/.local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 292, in start_session
response = self.execute(Command.NEW_SESSION, caps)["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 347, in execute
self.error_handler.check_response(response)
File "/app/.local/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: timeout: Timed out receiving message from renderer: 600.000
(Session info: chrome=127.0.6533.99)Tried again with real chrome, worked great. Thank you!
jdholtz
commented
3 months ago That ran fine, so perhaps the problem is intermittent? Maybe catch the exception and try again?
People used to have this issue more before SeleniumBase. I can add it if it still gives people trouble, but SeleniumBase has been very reliable, especially for Chrome.
Tried again with real chrome, worked great. Thank you!
Great, I guess that’ll be the solution for now. @ne0ark can you try with Google Chrome? If people are still running into the issue with Chrome or they want it to be reliable on Chromium, I can implement the retries again.
How do I change the order so it starts with reservations in docker?
@ne0ark currently the script starts each account/reservation in a new process, so the order is not really guaranteed. However, logging in versus using reservations still won’t make a difference as both need to use the browser.
ne0ark
commented
3 months ago I tried it with Google Chrome, and it works on my local machine. However, the issue still occurs in the Docker version. I had the same problem on the Stable version of the docker, and after switching to develop, the issue went away, but it came back after the last update.
jdholtz
commented
3 months ago However, the issue still occurs in the Docker version.
This is addressed in the FAQ in the last question. Can you run the Docker container with the --privileged flag and see if that fixes your issue?
ne0ark
commented
3 months ago However, the issue still occurs in the Docker version.
This is addressed in the FAQ in the last question. Can you run the Docker container with the
--privilegedflag and see if that fixes your issue?
It's the same error when running in privileged mode.
jdholtz
commented
3 months ago It's the same error when running in privileged mode.
Could you send the command you are using to run the Docker container and the logs + traceback (you may need to run the container with the --verbose argument passed to the script).
ne0ark
commented
3 months ago docker run
-d
--name='auto-southwest'
--net='bridge'
-e TZ="America/New_York"
-e HOST_OS="Unraid"
-e HOST_HOSTNAME="SW"
-e HOST_CONTAINERNAME="auto-southwest"
-e '/app/config.json'='/mnt/user/appdata/autosouthwest/config.json'
-e 'NVIDIA_DRIVER_CAPABILITIES'='all'
-e 'NVIDIA_VISIBLE_DEVICES'='all'
-e 'PUID'='99'
-e 'PGID'='100'
-e 'UMASK'='022'
-l net.unraid.docker.managed=dockerman
-l net.unraid.docker.icon='https://cdn-1.webcatalog.io/catalog/southwest/southwest-icon-filled-256.png'
--restart unless-stopped
--volume /mnt/user/appdata/autosouthwest/config.json:/app/config.json
--runtime=nvidia 'jdholtz/auto-southwest-check-in:develop'
--verboseTrace back was posted above. How do I get to logs on docker? Alpine has support for chrome?
jdholtz
commented
3 months ago Trace back was posted above. How do I get to logs on docker?
Ah, I see it now. That includes the logs so it’s sufficient.
Alpine has support for chrome?
No, Chrome doesn’t work with Alpine Linux (musl libc is used by Alpine and Chrome doesn’t have a build for musl AFAIK) which is why Chromium is used.
I wonder if the net=bridge flag is an issue. I’m not very familiar with the different networks for a Docker image, but the issue is that the driver can’t connect to the browser, so it’s possible that flag is the issue.
Not sure why privileged mode is not working. This comment (and a few others in the thread) show this issue being fixed for their Unraid environments. Hopefully we can find a better solution though.
ne0ark
commented
3 months ago The changes I sent for the pull request seem to work for me. I am not sure what's going on, but something is broken in the 3.12 version of alpine. I reverted to 3.12-rc, and it seems to be working.
[webdriver:146]: Using browser version: 119.0.6045.159 <-- seems to have fixed the issue.
jdholtz
commented
3 months ago I rebuilt the Docker image and am seeing this now as well. Using 3.12-rc-alpine fixed it for me too. Let me know when the PR is ready for review and I’ll try to review it as soon as I can.
ne0ark
commented
3 months ago I rebuilt the Docker image and am seeing this now as well. Using 3.12-rc-alpine fixed it for me too. Let me know when the PR is ready for review and I’ll try to review it as soon as I can.
Seems like Seleniumbase can auto download applicable drivers but check-in is not able to find the path.
2024-08-15 10:30:06 DEBUG Process-4[reservation_monitor:58]: Acquiring lock...
2024-08-15 10:30:06 DEBUG Process-3[reservation_monitor:58]: Acquiring lock...
/bin/sh: /usr/local/lib/python3.12/site-packages/seleniumbase/drivers/uc_driver: not found
Warning: uc_driver update needed. Getting it now:
*** chromedriver to download = 119.0.6045.105 (Legacy Version)
Downloading chromedriver-linux64.zip from:
https://storage.googleapis.com/chrome-for-testing-public/119.0.6045.105/linux64/chromedriver-linux64.zip ...
Download Complete!
Extracting ['chromedriver'] from chromedriver-linux64.zip ...
Unzip Complete!
The file [uc_driver] was saved to:
/usr/local/lib/python3.12/site-packages/seleniumbase/drivers/uc_driver
Making [uc_driver 119.0.6045.105] executable ...
[uc_driver 119.0.6045.105] is now ready for use!
2024-08-15 10:30:09 DEBUG Process-1[reservation_monitor:60]: Lock acquired
2024-08-15 10:30:09 DEBUG Process-1[reservation_monitor:198]: Retrieving reservations for account
2024-08-15 10:30:09 DEBUG Process-1[webdriver:128]: Starting webdriver for current session
Process Process-2:
Traceback (most recent call last):
File "/usr/local/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/local/lib/python3.12/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/app/lib/reservation_monitor.py", line 44, in monitor
self._monitor()
File "/app/lib/reservation_monitor.py", line 62, in _monitor
should_exit = self._check()
^^^^^^^^^^^^^
File "/app/lib/reservation_monitor.py", line 180, in _check
reservations, skip_scheduling = self._get_reservations()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/lib/reservation_monitor.py", line 202, in _get_reservations
reservations = webdriver.get_reservations(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/lib/webdriver.py", line 99, in get_reservations
driver = self._get_driver()
^^^^^^^^^^^^^^^^^^
File "/app/lib/webdriver.py", line 137, in _get_driver
driver = Driver(
^^^^^^^
File "/usr/local/lib/python3.12/site-packages/seleniumbase/plugins/driver_manager.py", line 583, in Driver
driver = browser_launcher.get_driver(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/seleniumbase/core/browser_launcher.py", line 2339, in get_driver
return get_local_driver(
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/seleniumbase/core/browser_launcher.py", line 4286, in get_local_driver
driver = undetected.Chrome(
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/seleniumbase/undetected/__init__.py", line 316, in __init__
super().__init__(options=options, service=service_)
File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
super().__init__(
File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/chromium/webdriver.py", line 55, in __init__
self.service.start()
File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/common/service.py", line 98, in start
self._start_process(self._path)
File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/common/service.py", line 208, in _start_process
self.process = subprocess.Popen(
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/subprocess.py", line 1026, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/local/lib/python3.12/subprocess.py", line 1950, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.12/site-packages/seleniumbase/drivers/uc_driver'
/bin/sh: /usr/local/lib/python3.12/site-packages/seleniumbase/drivers/uc_driver: not foundFile exists, maybe its missing dependencies?
At this point, it would be easier to use https://github.com/seleniumbase/SeleniumBase/blob/master/Dockerfile as a template and move the docker to Ubuntu.
jdholtz
commented
3 months ago Seems like Seleniumbase can auto download applicable drivers but check-in is not able to find the path.
Are you seeing this on the develop branch or your PR? Also, do you happen to be using an ARM architecture? I removed a couple lines in the Docker file that related to this issue because I was no longer running into it, but that may need to be re-added.
ne0ark
commented
3 months ago Seems like Seleniumbase can auto download applicable drivers but check-in is not able to find the path.
Are you seeing this on the develop branch or your PR? Also, do you happen to be using an ARM architecture? I removed a couple lines in the Docker file that related to this issue because I was no longer running into it, but that may need to be re-added.
That was on my development branch. I am not on ARM using AMD CPU.
I removed chromium-chromedriver and let seleniumbase download the driver. It seems to download the driver but check-in has trouble with that driver.
https://github.com/jdholtz/auto-southwest-check-in/pull/290 <- Ready for review.
Version
Auto-Southwest Check-In v7.5
Browser Version
Chromium
Description
Warning: uc_driver update needed. Getting it now: *** chromedriver to download = 127.0.6533.88 (Latest Stable) Downloading chromedriver-linux64.zip from: https://storage.googleapis.com/chrome-for-testing-public/127.0.6533.88/linux64/chromedriver-linux64.zip ... Download Complete! Extracting ['chromedriver'] from chromedriver-linux64.zip ... Unzip Complete! The file [uc_driver] was saved to: /home/dan/.local/lib/python3.10/site-packages/seleniumbase/drivers/uc_driver Making [uc_driver 127.0.6533.88] executable ... [uc_driver 127.0.6533.88] is now ready for use! Process Process-1:1: Traceback (most recent call last): File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/dan/auto-southwest-check-in/lib/checkin_handler.py", line 78, in _set_check_in self._wait_for_check_in(checkin_time) File "/home/dan/auto-southwest-check-in/lib/checkin_handler.py", line 103, in _wait_for_check_in self.checkin_scheduler.refresh_headers() File "/home/dan/auto-southwest-check-in/lib/checkin_scheduler.py", line 53, in refresh_headers webdriver.set_headers() File "/home/dan/auto-southwest-check-in/lib/webdriver.py", line 86, in set_headers driver = self._get_driver() File "/home/dan/auto-southwest-check-in/lib/webdriver.py", line 142, in _get_driver driver = Driver( File "/home/dan/.local/lib/python3.10/site-packages/seleniumbase/plugins/driver_manager.py", line 529, in Driver driver = browser_launcher.get_driver( File "/home/dan/.local/lib/python3.10/site-packages/seleniumbase/core/browser_launcher.py", line 1657, in get_driver return get_local_driver( File "/home/dan/.local/lib/python3.10/site-packages/seleniumbase/core/browser_launcher.py", line 3600, in get_local_driver driver = undetected.Chrome( File "/home/dan/.local/lib/python3.10/site-packages/seleniumbase/undetected/__init__.py", line 312, in __init__ super().__init__(options=options, service=service_) File "/home/dan/.local/lib/python3.10/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__ super().__init__( File "/home/dan/.local/lib/python3.10/site-packages/selenium/webdriver/chromium/webdriver.py", line 66, in __init__ super().__init__(command_executor=executor, options=options) File "/home/dan/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 208, in __init__ self.start_session(capabilities) File "/home/dan/.local/lib/python3.10/site-packages/seleniumbase/undetected/__init__.py", line 470, in start_session super().start_session(capabilities) File "/home/dan/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 292, in start_session response = self.execute(Command.NEW_SESSION, caps)["value"] File "/home/dan/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 347, in execute self.error_handler.check_response(response) File "/home/dan/.local/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message: timeout: Timed out receiving message from renderer: 600.000 (Session info: chrome=127.0.6533.88) Stacktrace: #0 0x562e847c071a
#1 0x562e84491640
#2 0x562e844795a3
#3 0x562e84479292
And so on
To Reproduce
The error showed up instead of checking me in.
Expected Behavior
No response
Relevant logs and program output
Additional context
No response