thp / urlwatch

Watch (parts of) webpages and get notified when something changes via e-mail, on your phone or via other means. Highly configurable.
https://thp.io/2008/urlwatch/
Other
2.81k stars 352 forks source link

[pyppeteer] No module named 'pyppeteer' using Docker python3.10 bookworm #775

Closed SHU-red closed 9 months ago

SHU-red commented 9 months ago

I'm trying to get a (very heavy) docker-container to run.

My goal is to have a docker-container which runns urlwatch and playwright to use the Browse/Navigate Jobs of urlwatch

Here is a snippet of my Dockerfile where it can be seen that pyppeteer is installed:

FROM python:3.10.13-bookworm
...
# Install all packages
RUN apt-get -y update
RUN apt-get -y upgrade
#RUN apt-get -y install python3-venv python3-pip
RUN apt-get -y install build-essential libssl-dev libpoppler-cpp-dev pkg-config python3-dev gcc musl-dev python3-dev libffi-dev cargo cron pkg-config

# Pip-install all Python modules
RUN python3 -m pip install cryptography --break-system-packages
RUN python3 -m pip install pyyaml minidb requests keyring appdirs lxml cssselect --break-system-packages
RUN python3 -m pip install chump pyppeteer pushbullet.py matrix_client markdown2 colorama playwright beautifulsoup4 jsbeautifier cssbeautifier pdftotext pytesseract Pillow aioxmpp jq --break-system-packages

# Install applicatios and initialize cron file
RUN apt-get -y install urlwatch chromium vim
...

Urlwatch is runnung, but executing a browser job results in the following error message

raceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urlwatch/command.py", line 139, in test_filter
    raise job_state.exception
  File "/usr/lib/python3/dist-packages/urlwatch/handler.py", line 68, in __enter__
    self.job.main_thread_enter()
  File "/usr/lib/python3/dist-packages/urlwatch/jobs.py", line 406, in main_thread_enter
    from .browser import BrowserContext
  File "/usr/lib/python3/dist-packages/urlwatch/browser.py", line 33, in <module>
    import pyppeteer
ModuleNotFoundError: No module named 'pyppeteer'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urlwatch/handler.py", line 78, in __exit__
    self.job.main_thread_exit()
  File "/usr/lib/python3/dist-packages/urlwatch/jobs.py", line 410, in main_thread_exit
    self.ctx.close()
    ^^^^^^^^
AttributeError: 'BrowserJob' object has no attribute 'ctx'
Traceback (most recent call last):
  File "/usr/bin/urlwatch", line 33, in <module>
    sys.exit(load_entry_point('urlwatch==2.25', 'console_scripts', 'urlwatch')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urlwatch/cli.py", line 112, in main
    urlwatch_command.run()
  File "/usr/lib/python3/dist-packages/urlwatch/command.py", line 431, in run
    self.handle_actions()
  File "/usr/lib/python3/dist-packages/urlwatch/command.py", line 231, in handle_actions
    sys.exit(self.test_filter(self.urlwatch_config.test_filter))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urlwatch/command.py", line 139, in test_filter
    raise job_state.exception
  File "/usr/lib/python3/dist-packages/urlwatch/handler.py", line 68, in __enter__
    self.job.main_thread_enter()
  File "/usr/lib/python3/dist-packages/urlwatch/jobs.py", line 406, in main_thread_enter
    from .browser import BrowserContext
  File "/usr/lib/python3/dist-packages/urlwatch/browser.py", line 33, in <module>
    import pyppeteer
ModuleNotFoundError: No module named 'pyppeteer'

Loggin into the terminal of the docker container i can verify that pyppeteer is installed but urlwatch seems to not be able to find the module

Im not an expert in python nor dockerfiles and just want to have the browser job stuff available

Is there anything obvious im doing wrong?

SHU-red commented 9 months ago

Sorry my fault

I did not use pip to install urlwatch and got a depreceated version

With urlwatch 2.28 it seems to work and try to use playwright

--> Closed