openzim / gutenberg

Scraper for downloading the entire ebooks repository of project Gutenberg
https://download.kiwix.org/zim/gutenberg
GNU General Public License v3.0
130 stars 37 forks source link

Upgrade to Python3.11 + update dependencies versions #156

Closed benoit74 closed 1 year ago

benoit74 commented 1 year ago

While starting gutenberg2zim on my Mac for the first time in a fresh venv, I got the following error :

Traceback (most recent call last):
  File "/Users/benoit/Repos/gutenberg/gutenberg2zim", line 16, in <module>
    from gutenbergtozim.download import download_all_books
  File "/Users/benoit/Repos/gutenberg/gutenbergtozim/download.py", line 19, in <module>
    from gutenbergtozim.export import get_list_of_filtered_books, fname_for
  File "/Users/benoit/Repos/gutenberg/gutenbergtozim/export.py", line 18, in <module>
    from jinja2 import Environment, PackageLoader
  File "/Users/benoit/Repos/gutenberg/.venv/lib/python3.9/site-packages/jinja2/__init__.py", line 33, in <module>
    from jinja2.environment import Environment, Template
  File "/Users/benoit/Repos/gutenberg/.venv/lib/python3.9/site-packages/jinja2/environment.py", line 15, in <module>
    from jinja2 import nodes
  File "/Users/benoit/Repos/gutenberg/.venv/lib/python3.9/site-packages/jinja2/nodes.py", line 19, in <module>
    from jinja2.utils import Markup
  File "/Users/benoit/Repos/gutenberg/.venv/lib/python3.9/site-packages/jinja2/utils.py", line 624, in <module>
    from markupsafe import Markup, escape, soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/Users/benoit/Repos/gutenberg/.venv/lib/python3.9/site-packages/markupsafe/__init__.py)

Searching the Internet indicates that downgrading markupsage to 2.0.1 solves the issue

pip install markupsafe==2.0.1

I confirm it solves the issue on my side. Probably linked to the fact that Jinja2 version is pretty old (2017).

Some details :

Python 3.9.12
pip 22.3.1
MacOS Monterey 12.6
rgaudin commented 1 year ago

Yes, we had this on all Jinja-using scrapers. gutenberg hasn't received an update in a long time.

benoit74 commented 1 year ago

I would suggest to change the title and update all deps, does it makes sense for you? I can work on it and submit a PR.

kelson42 commented 1 year ago

@benoit74 Definitly, see https://github.com/openzim/overview/issues/13 as well.