search

ArchiveTeam / seesaw-kit

Making a reusable toolkit for writing seesaw scripts
Other
69 stars 30 forks source link

Current seesaw-kit is not compatible with Python 3.10.x #127

Open T31M opened 2 years ago

T31M commented 2 years ago

As requested by @Arkiver2

Cause:

Seems like MutableMapping got removed in Python 3.10 (https://docs.python.org/3/library/collections.html) though I can not find a changelog entry for it in the official docs... It still exists in module collections.abc

Traceback (most recent call last):
  File "/mnt/seesaw-kit/./run-pipeline3", line 4, in <module>
    from seesaw.script.run_pipeline import main
  File "/mnt/seesaw-kit/seesaw/script/run_pipeline.py", line 11, in <module>
    from seesaw.runner import SimpleRunner
  File "/mnt/seesaw-kit/seesaw/runner.py", line 11, in <module>
    from seesaw.item import Item
  File "/mnt/seesaw-kit/seesaw/item.py", line 13, in <module>
    class ItemData(collections.MutableMapping):
AttributeError: module 'collections' has no attribute 'MutableMapping'

Reproduce:

git clone https://github.com/ArchiveTeam/seesaw-kit.git
λ docker run --rm --name alpine-testing -v $(pwd):/mnt/seesaw-kit -it --entrypoint=/bin/ash python:3-alpine

/ # python --version
Python 3.10.1

/ # cd /mnt/seesaw-kit/ && ls -alh
total 184K   
drwxrwxr-x    7 1000     1000        4.0K Jan 10 10:00 .
drwxr-xr-x    1 root     root        4.0K Jan 10 10:16 ..
drwxrwxr-x    6 1000     1000        4.0K Jan 10 10:00 .git
-rw-rw-r--    1 1000     1000         132 Jan 10 10:00 .gitignore
-rw-rw-r--    1 1000     1000         392 Jan 10 10:00 .travis.yml
-rw-rw-r--    1 1000     1000        1.9K Jan 10 10:00 LICENSE.txt
-rw-rw-r--    1 1000     1000         346 Jan 10 10:00 MANIFEST.in
-rw-rw-r--    1 1000     1000        2.5K Jan 10 10:00 README.md
drwxrwxr-x    3 1000     1000        4.0K Jan 10 10:00 doc
drwxrwxr-x    2 1000     1000        4.0K Jan 10 10:00 examples
-rw-rw-r--    1 1000     1000          43 Jan 10 10:00 requirements.txt
-rwxrwxr-x    1 1000     1000         106 Jan 10 10:00 run-pipeline
-rwxrwxr-x    1 1000     1000         280 Jan 10 10:00 run-pipeline2
-rwxrwxr-x    1 1000     1000         280 Jan 10 10:00 run-pipeline3
-rwxrwxr-x    1 1000     1000         105 Jan 10 10:00 run-warrior
-rwxrwxr-x    1 1000     1000         279 Jan 10 10:00 run-warrior2
-rwxrwxr-x    1 1000     1000         279 Jan 10 10:00 run-warrior3
drwxrwxr-x    6 1000     1000        4.0K Jan 10 10:06 seesaw
-rw-rw-r--    1 1000     1000        1.3K Jan 10 10:00 setup.py
drwxrwxr-x    2 1000     1000        4.0K Jan 10 10:00 tests

/mnt/seesaw-kit # pip install -r requirements.txt 
Collecting Tornado<4.99999.99999,>=4.0
  Downloading tornado-4.5.3.tar.gz (484 kB)
     |████████████████████████████████| 484 kB 2.0 MB/s            
  Preparing metadata (setup.py) ... done
Collecting sockjs-tornado
  Downloading sockjs-tornado-1.0.7.tar.gz (21 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: Tornado, sockjs-tornado
  Building wheel for Tornado (setup.py) ... done
  Created wheel for Tornado: filename=tornado-4.5.3-cp310-cp310-linux_x86_64.whl size=420786 sha256=7802a90fb4d5606be8a589909e9953ba5554794c729fbc647b7a304357c65793
  Stored in directory: /root/.cache/pip/wheels/7a/fc/0f/8294151792cdc6cd7041df1467dbb536b4cff20d161d58501c
  Building wheel for sockjs-tornado (setup.py) ... done
  Created wheel for sockjs-tornado: filename=sockjs_tornado-1.0.7-py3-none-any.whl size=31709 sha256=3c6cad4f59103ff5d6979fdea1dba652dda08dccf39ef08a30232cc3b82bd6d9
  Stored in directory: /root/.cache/pip/wheels/e9/dd/6c/8645515dd3622f1a375f9d8b7c2387f20d006d62001d7bf0ce
Successfully built Tornado sockjs-tornado
Installing collected packages: Tornado, sockjs-tornado
Successfully installed Tornado-4.5.3 sockjs-tornado-1.0.7
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

# Just running run-pipeline3
/mnt/seesaw-kit # ./run-pipeline3 --help
Traceback (most recent call last):
  File "/mnt/seesaw-kit/./run-pipeline3", line 4, in <module>
    from seesaw.script.run_pipeline import main
  File "/mnt/seesaw-kit/seesaw/script/run_pipeline.py", line 11, in <module>
    from seesaw.runner import SimpleRunner
  File "/mnt/seesaw-kit/seesaw/runner.py", line 11, in <module>
    from seesaw.item import Item
  File "/mnt/seesaw-kit/seesaw/item.py", line 13, in <module>
    class ItemData(collections.MutableMapping):
AttributeError: module 'collections' has no attribute 'MutableMapping'

# Running example pipeline (obv. fails same error)
/mnt/seesaw-kit # ./run-pipeline3 examples/example-pipeline.py example
Traceback (most recent call last):
  File "/mnt/seesaw-kit/./run-pipeline3", line 4, in <module>
    from seesaw.script.run_pipeline import main
  File "/mnt/seesaw-kit/seesaw/script/run_pipeline.py", line 11, in <module>
    from seesaw.runner import SimpleRunner
  File "/mnt/seesaw-kit/seesaw/runner.py", line 11, in <module>
    from seesaw.item import Item
  File "/mnt/seesaw-kit/seesaw/item.py", line 13, in <module>
    class ItemData(collections.MutableMapping):
AttributeError: module 'collections' has no attribute 'MutableMapping'

I am open to digging deeper into the code and figuring out a solution, maybe I can be of support. :)

JustAnotherArchivist commented 2 years ago

This just blew up gamejolt-grab because grab-base-df got rebuilt with Python 3.10...

So apparently these ABC aliases were deprecated back in Python 3.3 and their removal delayed several times. Relevant CPython commit: https://github.com/python/cpython/commit/c47c78b878ff617164b2b94ff711a6103e781753

The fix is trivial (collections.abc.MutableMapping instead of collections.MutableMapping). This would break on Python 2.7, but Python 2 has been EOL for over two years now, so we should drop support for that anyway. It would also break on 3.0 to 3.2, but if anyone still runs that in 2022, they deserve an exception blowing up in their face.

We do need to fix the test suite though. Travis CI is no longer operational.