search

webrecorder / warcit

Convert Directories, Files and ZIP Files to Web Archives (WARC)
https://pypi.python.org/pypi/warcit
Apache License 2.0
81 stars 13 forks source link

Fails to install on Python 3.11 because cchardet can't build #27

Open aquatix opened 1 year ago

aquatix commented 1 year ago

On installing warcit in a python 3.11 venv, I get a compilation error:

      creating build/temp.linux-x86_64-cpython-311/src/ext/uchardet/src/LangModels
      x86_64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -Isrc/ext/uchardet/src -I/home/mbscholt/.virtualenvs/divault-pyramid/include -I/usr/include/python3.11 -c src/cchardet/_cchardet.cpp -o build/temp.linux-x86_64-cpython-311/src/cchardet/_cchardet.o
      src/cchardet/_cchardet.cpp:196:12: fatal error: longintrepr.h: No such file or directory
        196 |   #include "longintrepr.h"
            |            ^~~~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]

This is a known issue with cChardet: https://github.com/PyYoshi/cChardet/issues/81 and has not been fixed since last year. It's unlikely that it will be fixed soon.

There are options to for example move to a fork at https://github.com/faust-streaming/cChardet but also updating to a recent Cython helps (tested): pip install --upgrade Cython. Maybe add the latter as a dependency for Python 3.11 and up?

lasztoth commented 7 months ago

The same issue occurs with Python 3.10.x. However, on Python 3.9 you can simply run python setup.py install to set everything up just fine.

You can use a venv/conda to set up a 3.9 environment easily.

aquatix commented 7 months ago

Python 3.9 is not an option for us, as warcit has to work inside a bigger project that targets Python 3.10+