Open jvasa opened 5 months ago
Same error also happens running in docker
Workaround is to upgrade Chrome (google-chrome-stable). Here's how I did it in docker:
$ docker exec -it safeway-coupons bash
root@safeway-coupons:/# apt update && apt upgrade -y
BEFORE:
root@safeway-coupons:/# safeway-coupons -c accounts_file -d -n
Clipping coupons for Safeway account redacted@example.com
Would send email to redacted@example.com
>>>>>>
Safeway account: redacted@example.com
Error: Authentication Failure (cannot access local variable 'driver' where it is not associated with a value)
<<<<<<
.....
safeway_coupons.errors.AuthenticationFailure: Authentication Failure (cannot access local variable 'driver' where it is not associated with a value)
AFTER:
root@safeway-coupons:/# safeway-coupons -c accounts_file -d -n
Clipping coupons for Safeway account redacted@example.com
Connect to safeway.com
Decline cookie prompt
Return to safeway.com after declining cookie prompt
Open Sign In sidebar
Open Sign In form
Populate Sign In form
Skipping Keep Me Signed In checkbox which is not present
Click Sign In button
Wait for signed in landing page to load
Retrieve session information
.....
Clipped 121 coupons
Would send email to redacted@example.com
Since the dockerfile doesn't pin package versions, presumably the docker image just needs to be rebuilt and the apt command will pull the later version of the package...
Edit: This will rebuild you a new working docker container and tag it:
docker build -t localsafeway:latest https://github.com/smkent/safeway-coupons.git#main
Then you can run the freshly built localsafeway:latest image you just created.
Edit2: for future reference for those not as familiar with docker, to cleanup once the main package is updated and you want to delete the local version you just built you can run this to remove and untag the image you created:
docker rmi localsafeway:latest
and cleanup any other unused images:
docker image prune
Thanks guys for the local fix, it does work and I updated to use the localsafeway image.
Not working for me. Current driver version: ChromeDriver 123.0.6312.86
Try in your docker-compose.yaml to change the line:
from: image: ghcr.io/smkent/safeway-coupons:latest
to:build: https://github.com/smkent/safeway-coupons.git#main
This way it will build the latest version when you start the docker, although it may take some time.
Should have been more verbose. I've been cloning the repo, main branch, and building and running it from the locally built version.
I do get some build errors when I do the process as you've described but I don't think that matters so much since cloning the repo and building it is effectively the same thing, unless I'm mistaken.
For those of you using Docker Compose and using the hosted image you can temporarily add the following to get the image updated before /entrypoint
runs
entrypoint: ["bash", "-c", "apt update && apt upgrade -y && /entrypoint"]
I also the method suggested by @tuxthepenguin84
As far as I can tell the chrome debugger doesn't open the listening port.
`netstat -lnp (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 127.0.0.1:37439 0.0.0.0: LISTEN 95/undetected_chrom tcp 0 0 127.0.0.11:35507 0.0.0.0: LISTEN - udp 0 0 127.0.0.11:52714 0.0.0.0:* - Active UNIX domain sockets (only servers) Proto RefCnt Flags Type State I-Node PID/Program name Path
`Clipping coupons for Safeway account redacted@redacted.com send: b'GET /chrome-for-testing/last-known-good-versions-with-downloads.json HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: googlechromelabs.github.io\r\nUser-Agent: Python-urllib/3.12\r\nConnection: close\r\n\r\n' reply: 'HTTP/1.1 200 OK\r\n' header: Connection: close header: Content-Length: 8664 header: Server: GitHub.com header: Content-Type: application/json; charset=utf-8 header: permissions-policy: interest-cohort=() header: x-origin-cache: HIT header: Last-Modified: Sun, 31 Mar 2024 15:09:15 GMT header: Access-Control-Allow-Origin: * header: Strict-Transport-Security: max-age=31556952 header: ETag: "66097c9b-21d8" header: expires: Sun, 31 Mar 2024 15:19:19 GMT header: Cache-Control: max-age=600 header: x-proxy-cache: HIT header: X-GitHub-Request-Id: 519E:8BBA0:2BA0FB:336C98:66097D51 header: Accept-Ranges: bytes header: Date: Sun, 31 Mar 2024 15:36:18 GMT header: Via: 1.1 varnish header: Age: 235 header: X-Served-By: cache-bfi-kbfi7400089-BFI header: X-Cache: HIT header: X-Cache-Hits: 1 header: X-Timer: S1711899379.614038,VS0,VE2 header: Vary: Accept-Encoding header: X-Fastly-Request-ID: 4a8460ae8a0072a4cc75644dac2671c548deda8c send: b'GET /chrome-for-testing-public/123.0.6312.86/linux64/chromedriver-linux64.zip HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: storage.googleapis.com\r\nUser-Agent: Python-urllib/3.12\r\nConnection: close\r\n\r\n' reply: 'HTTP/1.1 200 OK\r\n' header: X-GUploader-UploadID: ABPtcPrllGsIelcFXA6EyU4t9JjvXt_mF0djk3SzDeGjQHPFq4oqZAll1toRN1SbCm42eP8HvWyXS4teqQ header: x-goog-generation: 1711403106340092 header: x-goog-metageneration: 1 header: x-goog-stored-content-encoding: identity header: x-goog-stored-content-length: 8636477 header: x-goog-hash: crc32c=DBrxPQ== header: x-goog-hash: md5=PTHBsl8A53AWgJAkTjpQqQ== header: x-goog-storage-class: STANDARD header: Accept-Ranges: bytes header: Content-Length: 8636477 header: Server: UploadServer header: Date: Sun, 31 Mar 2024 15:09:19 GMT header: Expires: Sun, 31 Mar 2024 16:09:19 GMT header: Cache-Control: public, max-age=3600 header: Age: 1619 header: Last-Modified: Mon, 25 Mar 2024 21:45:06 GMT header: ETag: "3d31c1b25f00e770168090244e3a50a9" header: Content-Type: application/zip header: Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 header: Connection: close send: b'POST /session HTTP/1.1\r\nHost: localhost:37439\r\nAccept-Encoding: identity\r\nContent-Length: 686\r\nAccept: application/json\r\nContent-Type: application/json;charset=UTF-8\r\nUser-Agent: selenium/4.19.0 (python linux)\r\nConnection: keep-alive\r\n\r\n' send: b'{"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome", "pageLoadStrategy": "normal", "goog:chromeOptions": {"extensions": [], "binary": "/bin/google-chrome", "args": ["--incognito", "--no-sandbox", "--disable-extensions", "--disable-application-cache", "--disable-gpu", "--disable-setuid-sandbox", "--disable-dev-shm-usage", "--remote-debugging-host=127.0.0.1", "--remote-debugging-port=47491", "--user-data-dir=/tmp/tmpnvza07jp", "--lang=C", "--no-default-browser-check", "--no-first-run", "--no-sandbox", "--test-type", "--headless=new", "--window-size=1920,1080", "--start-maximized", "--no-sandbox", "--log-level=0"], "debuggerAddress": "127.0.0.1:47491"}}}}' reply: 'HTTP/1.1 500 Internal Server Error\r\n' header: Content-Length: 769 header: Content-Type: application/json; charset=utf-8 header: cache-control: no-cache Sending email to redacted@redacted.com
Safeway account: redacted@redacted.com Error: Authentication Failure (cannot access local variable 'driver' where it is not associated with a value) <<<<<< sendmail: bad address '' Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/safeway_coupons/session.py", line 69, in _chrome_driver with chrome_driver(headless=headless) as driver: File "/usr/local/lib/python3.12/contextlib.py", line 137, in enter return next(self.gen) ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/safeway_coupons/chrome_driver.py", line 38, in chrome_driver driver = uc.Chrome(options=options) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/undetected_chromedriver/init.py", line 466, in init super(Chrome, self).init( File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in init super().init( File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/chromium/webdriver.py", line 61, in init super().init(command_executor=executor, options=options) File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 208, in init self.start_session(capabilities) File "/usr/local/lib/python3.12/site-packages/undetected_chromedriver/init.py", line 724, in start_session super(selenium.webdriver.chrome.webdriver.WebDriver, self).start_session( File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 292, in start_session response = self.execute(Command.NEW_SESSION, caps)["value"] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 347, in execute self.error_handler.check_response(response) File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at 127.0.0.1:47491 from chrome not reachable Stacktrace:
0 0x56225f18a8a3
1 0x56225ee80717
2 0x56225ee6b7c3
3 0x56225eeb9c91
4 0x56225eeb07bb
5 0x56225eef9aed
6 0x56225eeed343
7 0x56225eebe593
8 0x56225eebef5e
9 0x56225f14e88b
10 0x56225f1527e5
11 0x56225f13c5b1
12 0x56225f153372
13 0x56225f1218bf
14 0x56225f179768
15 0x56225f17993b
16 0x56225f1899f4
17 0x7fcf976b6134
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/safeway_coupons/session.py", line 58, in init self._login(account) File "/usr/local/lib/python3.12/site-packages/safeway_coupons/session.py", line 95, in _login with self._chrome_driver() as driver: File "/usr/local/lib/python3.12/contextlib.py", line 137, in enter return next(self.gen) ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/safeway_coupons/session.py", line 76, in _chrome_driver driver.save_screenshot(path) ^^^^^^ UnboundLocalError: cannot access local variable 'driver' where it is not associated with a value
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/safeway-coupons", line 8, in
I had multiple issues. One of which was PID limits causing the process to not be able to spawn. The recommended solution to build the project locally and deploy that image did end up resolve the base issue.
Rebuilding like @jvasa mentioned https://github.com/smkent/safeway-coupons/issues/116#issuecomment-2028558820 fixed the issue.
following @jvasa suggestion in https://github.com/smkent/safeway-coupons/issues/116#issuecomment-2028558820 also fixed it for me (it should also fix #115). originally i was using v0.2.12 and getting this error when i exec'd into the container to run it manually:
root@f1f45bae1ff4:/# safeway-coupons
Clipping coupons for Safeway account XXXX
sendmail: bad address ''
Error: Authentication Failure (cannot access local variable 'driver' where it is not associated with a value)
when you build from scratch instead of using the prebuilt container, you get a new version of ChromeDriver. since this seems to be a recurring issue, would it make sense to factor out the ChromeDriver install and make it so that it can be upgraded without touching the rest of the safeway-coupon's code? Would it be possible to make a container that keeps ChromeDriver up to date, saving it into a volume that can also be mounted/shared into safeway-coupons container? That way we wouldn't have to rebuild everything from scratch every time.
apparently there's some auto updaters like these: https://stackoverflow.com/questions/66018451/how-to-get-the-chromedriver-automatically-updated-through-python-selenium-after
but i dont know if these will fit our use case. let me know if i'm on the right track, the next time this breaks, i'll try to automate it.
This is how I fixed it and setup a connection to the SMTP email account I use:
admin@machine:~/containers/safeway$ cat docker-compose.yaml
version: "3.7"
services:
safeway-coupons:
image: ghcr.io/smkent/safeway-coupons:latest
entrypoint: >
bash -c "
apt update &&
apt upgrade -y &&
apt install -y msmtp &&
update-alternatives --install /usr/sbin/sendmail sendmail /usr/sbin/msmtp_wrapper 100 &&
/entrypoint"
volumes:
- ./sendmail.sh:/usr/sbin/msmtp_wrapper:ro
- ./msmtprc:/etc/msmtprc:ro
- ./accounts.ini:/accounts_file:ro
environment:
CRON_SCHEDULE: "0 2 * * *"
TZ: America/Los_Angeles
SMTPHOST: mail.privateemail.com
SMTP_PORT: 465
SMTP_SSL: true
SMTP_USERNAME: <EMAIL_SMTP_ACCOUNT>
SMTP_PASSWORD: <PASSWORD>
#SAFEWAY_ACCOUNT_USERNAME: <EMAIL_SAFEWAY>
#SAFEWAY_ACCOUNT_PASSWORD: <PASSWORD_SAFEWAY>
#SAFEWAY_ACCOUNT_MAIL_FROM: <EMAIL_SMTP_ACCOUNT>
#SAFEWAY_ACCOUNT_MAIL_TO: <EMAIL_SAFEWAY>
SAFEWAY_ACCOUNTS_FILE: /accounts_file
restart: unless-stopped
admin@machine:~/containers/safeway$ cat msmtprc
defaults
auth on
tls on
tls_starttls off
tls_trust_file /etc/ssl/certs/ca-certificates.crt
logfile ~/.msmtp.log
account default
host mail.privateemail.com
port 465
from <EMAIL_SMTP_ACCOUNT>
user <EMAIL_SMTP_ACCOUNT>
password <PASSWORD>
admin@machine:~/containers/safeway$ cat sendmail.sh
#!/bin/bash
# Read email content from stdin
EMAIL_CONTENT=$(cat)
# Use msmtp to send the email
echo "${EMAIL_CONTENT}" | msmtp -a default -t
I have a wrapper script to use msmtp
instead, and a config file for msmtp
and of course the updates needed for the container upon start.
Getting this error:
Error: Authentication Failure (cannot access local variable 'driver' where it is not associated with a value) <<<<<< Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/safeway_coupons/session.py", line 69, in _chrome_driver with chrome_driver(headless=headless) as driver: File "/usr/local/lib/python3.12/contextlib.py", line 137, in enter return next(self.gen) ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/safeway_coupons/chrome_driver.py", line 38, in chrome_driver driver = uc.Chrome(options=options) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/undetected_chromedriver/init.py", line 466, in init super(Chrome, self).init( File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in init super().init( File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/chromium/webdriver.py", line 61, in init super().init(command_executor=executor, options=options) File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 208, in init self.start_session(capabilities) File "/usr/local/lib/python3.12/site-packages/undetected_chromedriver/init.py", line 724, in start_session super(selenium.webdriver.chrome.webdriver.WebDriver, self).start_session( File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 292, in start_session response = self.execute(Command.NEW_SESSION, caps)["value"] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 347, in execute self.error_handler.check_response(response) File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at 127.0.0.1:60161 from session not created: This version of ChromeDriver only supports Chrome version 123 Current browser version is 122.0.6261.94 Stacktrace:
0 0x556b10a59993
1 0x556b10754136
2 0x556b1078e3bb
3 0x556b1078d585
4 0x556b10783ebb
5 0x556b107cd239
6 0x556b107c0a73
7 0x556b10791c93
8 0x556b1079265e
9 0x556b10a1e08b
10 0x556b10a22005
11 0x556b10a0c491
12 0x556b10a22b92
13 0x556b109f19ef
14 0x556b10a48df8
15 0x556b10a48fcb
16 0x556b10a58ae4
17 0x7fafa5a8a134
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/safeway_coupons/session.py", line 58, in init self._login(account) File "/usr/local/lib/python3.12/site-packages/safeway_coupons/session.py", line 95, in _login with self._chrome_driver() as driver: File "/usr/local/lib/python3.12/contextlib.py", line 137, in enter return next(self.gen) ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/safeway_coupons/session.py", line 76, in _chrome_driver driver.save_screenshot(path) ^^^^^^ UnboundLocalError: cannot access local variable 'driver' where it is not associated with a value
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/local/bin/safeway-coupons", line 8, in
sys.exit(main())
^^^^^^
File "/usr/local/lib/python3.12/site-packages/safeway_coupons/app.py", line 125, in main
sc.clip_for_account(account)
File "/usr/local/lib/python3.12/site-packages/safeway_coupons/safeway.py", line 38, in clip_for_account
swy = SafewayClient(account, self.debug_dir)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/safeway_coupons/client.py", line 17, in init
self.session = LoginSession(account, debug_dir)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/safeway_coupons/session.py", line 64, in init
raise AuthenticationFailure(e, account) from e
safeway_coupons.errors.AuthenticationFailure: Authentication Failure (cannot access local variable 'driver' where it is not associated with a value)