speedyg0nz / MagInkCal

E-Ink Magic Calendar that automatically syncs to Google Calendar and runs off a battery powered Raspberry Pi Zero
Apache License 2.0
3.08k stars 105 forks source link

Chromedriver Issues - Status codes (-4, or -11, or 127) #19

Closed ooglyboogly closed 1 year ago

ooglyboogly commented 1 year ago

Running the code as direct clone, i get: chromedriver unexpectedly exited. Status code was: -4

Which occurs in Render.py at line: driver = webdriver.Chrome(,options=opts)

I have scoured the internet trying to find a solution. Have changed versions, forced downgrades, written in explicit executable paths, set binary locations, etc etc etc that were found in countless Stackoverflow articles. Nothing seems to work. I even started this project from scratch with hopes to trigger a better result. However, the Chromium driver part simply will not work.

Would LOVE to know what versions of Chromium-browser, Selenium, Chromedriver, etc you are using in your successful build to see if I could replicate. Or if you could try your own project again from scratch to see if I am just crazy... Feels like today's packages just are not lining up with your instructions :(

speedyg0nz commented 1 year ago

Hi @ooglyboogly , I just tried to run my script it on a mostly fresh RPi, and installing the Python libraries from scratch. Sharing the library/package versions I'm seeing.

Selenium version: 4.8.0 > I got this by running: pip3 show selenium Chrome driver version: 92.0.4515.98 > I got this by adding the following in render.py: print(driver.capabilities['browserVersion'])

Upon running my code on the new libraries, I received some errors regarding a deprecated function in render.py: html = driver.find_element_by_tag_name('html')

This can be easily replaced by adding the following import: from selenium.webdriver.common.by import By

And changing the deprecated code to the following: html = driver.find_element(By.TAG_NAME,'html')

With those changes above, I'm able to run my script successfully and generate the calendar image. However, I'm not seeing the error status codes that you're seeing.

ooglyboogly commented 1 year ago

And you are still using this on your Raspberry Pi Zero W? I cannot understand why this does not work from a fresh build for me. When looking into the error messages, a lot of people are saying that they migrate from Pi Zero W with their same code/OS to a Pi 3 and it just magically works. Want to rule out that on your end.

I am going to try a new build again with fresh code from your repo. Appreciate your help here, feel like I am so close but so far. Thought the hardest part was going to be getting all the hardware! Even bought the starter kit for Pi Zero W since the base units are all sold out everywhere.

ooglyboogly commented 1 year ago

Did a fresh build from scratch with 9-22-2022 build of Raspberry Pi Lite 32Bit on the Pi Zero W. Followed your instructions ending up with Chrome-driver:

apt show chromium-chromedriver
Package: chromium-chromedriver
Version: 109.0.5414.112-rpt1
Priority: optional
Section: web
Source: chromium-browser
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Installed-Size: 15.5 MB
Provides: chromium-driver
Pre-Depends: dpkg (>= 1.15.6)
Depends: libc6 (>= 2.17), libgcc-s1 (>= 3.5), libglib2.0-0 (>= 2.22.0), libnspr4 (>= 2:4.9-2~), libnss3 (>= 2:3.13.4-2~), libxcb1 (>= 1.9.2), chromium-browser (>= 4.0.203.0~)
Homepage: https://chromium.googlesource.com/chromium/src/
Download-Size: 5,759 kB
APT-Manual-Installed: yes
APT-Sources: http://archive.raspberrypi.org/debian bullseye/main armhf Packages
Description: WebDriver driver for the Chromium Browser
 Chromedriver serves as a bridge between Chromium Browser and Selenium
 WebDriver.
 .
 See https://sites.google.com/a/chromium.org/chromedriver/ for details.

Selenium 4.8.0 using same method as you.

Unfortunately I still end up with:

Time synchronised to 2023-02-14 16:29:07.452108-06:00
Retrieving events between 2023-02-12T00:00:00-06:00 and 2023-03-18T23:59:59.999999-05:00...
No upcoming events found.
Calendar events retrieved in 0:00:00.684527
Message: Service chromedriver unexpectedly exited. Status code was: -4

Going to see if I can force down to chromedriver 92.0.4515.98 and test again.

speedyg0nz commented 1 year ago

Hi @ooglyboogly, wanted to reply earlier but my SD card got corrupted and it took me some time to get it replaced.

Anyway, I think I figured out the problem. It's probably because chromium-browser doesn't play well with the latest Raspberry Pi OS version (bullseye) for RPi0. This is why it was working with the other RPi versions but not RPi0.

Read here for more discussion on this: https://forums.raspberrypi.com/viewtopic.php?t=323478

I realised that my RPi0 setup is still running on the legacy OS (buster), which is still available to download and install. I did a completely fresh install of buster, installed the dependencies, and copied over the code. Runs well after the updating the deprecated code mentioned in my earlier reply. It complained of an outdated urllib3 library but the code still ran to completion otherwise. Do give it a shot and let me know if it helped.

Edit: It also complained of a missing libopenjp2.so.7 dependency, but it was easily fixed by running the following command. sudo apt-get install libopenjp2-7-dev

ooglyboogly commented 1 year ago

Ah!! Thanks. Trying it right now. That is quite a frustrating issue. Did not even think the OS would be the issue!

ooglyboogly commented 1 year ago

So, fresh install of Buster and going through the steps again seems to resolve the Chromium issues.

I had to follow the same steps you mentioned of Importing By and changing the HTML element call, also had to get libopenjp2-7-dev.

Good news is that the code executes without error (does have the warning about urllib3 but keeps going), and everything appears to work... however Bad News is that nothing actually renders. Maybe I should open a separate issue, but nothing even flickers on the screen.

My console is showing:

python3 maginkcal.py
/usr/lib/python3/dist-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.14) or chardet (3.0.4) doesn't match a supported version!
  RequestsDependencyWarning)
Starting daily calendar update
Battery level at start: 96.245
Time synchronised to 2023-03-02 16:18:52.384096+08:00
Retrieving events between 2023-02-26T00:00:00+08:00 and 2023-04-01T23:59:59.999999+08:00...
Calendar events retrieved in 0:00:00.756954
Screenshot captured and saved to file.
Image colours processed. Extracted grayscale and red images.
Battery level at end: 96.484
Completed daily calendar update
Checking if configured to shutdown safely - Current hour: 16

I ran the code at https://www.waveshare.com/wiki/12.48inch_e-Paper_Module to test the device and it seems to be rendering those test python files fine (although it uses WiringPi and other packages).

speedyg0nz commented 1 year ago

Let me guess, you've yet to set the isDisplayToScreen flag to 'true' in the config.json file (project root folder)?

And after you do so, it'll be wise to do two more steps. There has been some updates to the Waveshare driver since I first developed the code. So go to Waveshare's git repository, download the DEV_Config_32.so and epdconfig.py files. Copy them and replace the files in the "display" folder of the project.

Lastly, run the following command. sudo apt-get install wiringpi

With all that done, it should just work now. Let me know how it goes, and I'll probably update my instructions so others can benefit from the updated set of instructions as well.

ooglyboogly commented 1 year ago

You are correct, when I read: isDisplayToScreen = config['isDisplayToScreen'] # set to true when debugging rendering without displaying to screen

I thought that meant to set to true if you want to debug without displaying. Ugh, so simple! It does work now after doing that, replacing the display files, and having wiringpi installed.

FYI for wiringpi, at https://www.waveshare.com/wiki/12.48inch_e-Paper_Module they have instructions:

#Open the Raspberry Pi terminal and run the following command
sudo apt-get install wiringpi
#For Raspberry Pi systems after May 2019 (earlier than before, you may not need to execute), you may need to upgrade:
wget https://project-downloads.drogon.net/wiringpi-latest.deb
sudo dpkg -i wiringpi-latest.deb
gpio -v
# Run gpio -v and version 2.52 will appear. If it does not appear, the installation is wrong

I found I did need to do this to get to 2.52 for my system.

Another item for your instructions as an FYI is for PiSugar 3 setup, it required a little bit of extra steps from https://github.com/PiSugar/pisugar-power-manager-rs: running the bash as normal from your instructions, but I had to just select a random PiSugar model from the list since the 3 is not listed. I then ran the command found on that github sed -e "s|--model '.*' |--model '<model>' |" -i /etc/default/pisugar-server replacing <model> with <Pisugar 3> and then rerunning the bash for pisugar-power-manager.sh. This then allowed me to select the Pisugar 3 model and configure it.

Finally, I ended up having to add to add the sudo shutdown now command to my user's NOPASSWD abilities in order for the cron job to work. For those who are building the project under the pi user or some other fully root user maybe this isnt needed, but I made a new user with SUDO abilities, but it still requires PW for security. So I did sudo visudo and edited the line for %sudo to say: %sudo ALL=(ALL:ALL) ALL, NOPASSWD:/sbin/shutdown

This allows the user to run only the shutdown command with no password, and allow cron to run the command as my user when booting up.

Thanks for the awesome project! And I plan on eventually tweaking it to fit my needs as a fork. One of my first missions is to see if I can make PiSugar Wake up more than just 1 time a day... Would love to do it a few times a day with different screens to show.