ImportError: cannot import name 'dummy' from 'redis.utils

edsu commented 7 months ago

When running wayback with Python v3.11.9 I'm seeing this error:

$ poetry run wayback
Traceback (most recent call last):
  File "/Users/edsu/Library/Caches/pypoetry/virtualenvs/was-pywb-277i1Mfi-py3.11/bin/wayback", line 8, in <module>
    sys.exit(wayback())
             ^^^^^^^^^
  File "/Users/edsu/Library/Caches/pypoetry/virtualenvs/was-pywb-277i1Mfi-py3.11/lib/python3.11/site-packages/pywb/apps/cli.py", line 25, in wayback
    return WaybackCli(args=args,
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/edsu/Library/Caches/pypoetry/virtualenvs/was-pywb-277i1Mfi-py3.11/lib/python3.11/site-packages/pywb/apps/cli.py", line 98, in __init__
    self.application = self.load()
                       ^^^^^^^^^^^
  File "/Users/edsu/Library/Caches/pypoetry/virtualenvs/was-pywb-277i1Mfi-py3.11/lib/python3.11/site-packages/pywb/apps/cli.py", line 187, in load
    from pywb.apps.frontendapp import FrontEndApp
  File "/Users/edsu/Library/Caches/pypoetry/virtualenvs/was-pywb-277i1Mfi-py3.11/lib/python3.11/site-packages/pywb/apps/frontendapp.py", line 26, in <module>
    from pywb.apps.rewriterapp import RewriterApp
  File "/Users/edsu/Library/Caches/pypoetry/virtualenvs/was-pywb-277i1Mfi-py3.11/lib/python3.11/site-packages/pywb/apps/rewriterapp.py", line 4, in <module>
    from fakeredis import FakeStrictRedis
  File "/Users/edsu/Library/Caches/pypoetry/virtualenvs/was-pywb-277i1Mfi-py3.11/lib/python3.11/site-packages/fakeredis.py", line 27, in <module>
    from redis.utils import dummy
ImportError: cannot import name 'dummy' from 'redis.utils' (/Users/edsu/Library/Caches/pypoetry/virtualenvs/was-pywb-277i1Mfi-py3.11/lib/python3.11/site-packages/redis/utils.py)

I am using poetry to manage my environment, with this configuration.

If you want to replicate you can:

$ git clone https://github.com/sul-dlss/was-pywb.git
$ cd was-pywb/pywb
$ poetry install
$ poetry run wayback

However, I noticed that using a stock pyenv with pip works fine:

$ python -mvenv env
$ source env/bin/activate
$ pip install pywb
$ wayback

For whatever reason poetry seems to be pulling in the most recent version of redis (v5.0.4), whereas pip is installing v2.10.6. Perhaps the redis version should be pinned to support the old version of fakeredis that is pinned?

https://github.com/webrecorder/pywb/blob/b9f1609df920ee718237a4426d447111a850fa8e/requirements.txt#L4

ato commented 7 months ago

This occurs with older versions of pip too. It seems the way dependency conflicts are handled has changed.

# python3.9 -m pip --version
pip 20.2.4 from /usr/lib/python3.9/site-packages/pip (python 3.9)
# python3.9 -m pip install pywb
...
ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

fakeredis 0.16.0 requires redis<3, but you'll have redis 5.0.4 which is incompatible
# pywb
...
ImportError: cannot import name 'dummy' from 'redis.utils'

Upgrading pip resolves it:

# python3.9 -m pip install --upgrade pip
Successfully installed pip-24.0
# python3.9 -m pip install pywb
Installing collected packages: redis
  Attempting uninstall: redis
    Found existing installation: redis 5.0.4
    Uninstalling redis-5.0.4:
      Successfully uninstalled redis-5.0.4
Successfully installed redis-2.10.6

edsu commented 7 months ago

I don't think upgrading poetry resolves it though.

tw4l commented 7 months ago

Thanks for the patch! Released in 2.8.1: https://github.com/webrecorder/pywb/releases/tag/v-2.8.1

edit: now 2.8.3, had a few issues with the PyPI release action 😅

webrecorder / pywb

ImportError: cannot import name 'dummy' from 'redis.utils #903