search

JimmXinu / FanFicFare

FanFicFare is a tool for making eBooks from stories on fanfiction and other web sites.
Other
738 stars 156 forks source link

Import error when system python's urllib3 is version 2 #987

Closed ascendant512 closed 11 months ago

ascendant512 commented 11 months ago

My system had upgraded dev-python/urllib3-2.0.3 and Calibre failed to start the FanFicFare plugin in _compat.py 

from urllib3.contrib import appengine as gaecontrib
ImportError: cannot import name 'appengine' from 'urllib3.contrib' (/usr/lib/python3.11/site-packages/urllib3/contrib/__init__.py)

It looks like Google App Engine is absent in urllib3 version 2, and it also looks like FanFicFare ships with included urllib3 version 1.

In my PC, I worked around it by unzipping and deleting FanFicFare.zip, then creating a symlink to the included urllib3 in requests_toolbelt, then changing line 50 to read like: from .urllib3.contrib import appengine as gaecontrib

JimmXinu commented 11 months ago

I only support FanFicFare on the official Calibre releases, and Calibre in turn bundles its dependencies--including python--for exactly this sort of reason. (Most dependencies bundled in FanFicFare are for older versions of Calibre.)

kaithar commented 7 months ago

I'm sorry but I have to strongly object to that reply.

  1. The commit removing appengine from urllib3 claims it was only available for python 2.7, calibre now ships with 3.11 as a minimum. https://github.com/urllib3/urllib3/commit/6af53ac41757399607382bf10a92b288fdf4951d
  2. Actually, the offending code was only on GAE's python implementation, so it probably wouldn't work on any of the shipped calibre versions unless they were doing something screwy.
  3. Calibre's doesn't seem to bundle urllib3 and only mentions it in the Arch CI setup
  4. The offending code is in your bundled version of requests-toolbelt which removed appengine support in May of this year https://github.com/requests/toolbelt/blob/master/HISTORY.rst
  5. toolbelt is only included because it's a dep of cloudscraper... which you have bundled at a 2 year out of date and mangled version of 1.2.58... mangled because that project dropped py2 support in 2020, but it doesn't work anyway because cloudscaper literally only works if you update it when the CF updates break it... which is presumably why there are so many issues about it not working and the wiki says to use cfprox
  6. cloudscraper is only used by default for a single site now... quotev.com https://github.com/JimmXinu/FanFicFare/commit/53dd0073f139069eee6df0e084905662eefa8ab5
  7. ... which turns out to be a site that doesn't even use cloudflare: https://checkforcloudflare.selesti.com/?q=https://quotev.com

All of this easily found by just checking what versions are being bundled by Calibre and FFF.

So, short version: your bundled version of toolbelt is trying to import something that won't work on modern Calibre (py2 was dropped in calibre 5.0) and appears to be broken specifically because you aren't forcing the use of the version of urllib3 you bundled, to power a feature that doesn't work on any version, for a site that doesn't use that feature.

All of which also raises the question: Are you tracking and backporting security bugs for all the packages you have bundled? You seem to have some really old code in there.