search

whipper-team / whipper

Python CD-DA ripper preferring accuracy over speed
GNU General Public License v3.0
1.14k stars 90 forks source link

Dockerfile: Images does not build with Debian Buster (any more?) because of an outdated python #603

Open k11i opened 9 months ago

k11i commented 9 months ago

Hello, trying to build the image with podman and docker fails because of an outdated python used in Debian Buster. Since Buster is out of support, it shouldn't be used anymore anyway.

$> optical_gid=$(getent group optical | cut -d: -f3) uid=$(id -u) buildah bud --no-cache --pull --build-arg optical_gid --build-arg uid -t charly/whipper .

Step 15/22 : RUN cd /whipper && python3 setup.py install     && rm -rf /whipper     && whipper -v
 ---> Running in feebb1b46665
Traceback (most recent call last):
  File "setup.py", line 31, in <module>
    'scripts/accuraterip-checksum',
  File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.7/distutils/core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 447, in __init__
    k: v for k, v in attrs.items()
  File "/usr/lib/python3.7/distutils/dist.py", line 292, in __init__
    self.finalize_options()
  File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 739, in finalize_options
    ep.load()(self, ep.name, value)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2411, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2417, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/whipper/.eggs/setuptools_scm-8.0.4-py3.7.egg/setuptools_scm/__init__.py", line 7, in <module>
    from ._config import Configuration
  File "/whipper/.eggs/setuptools_scm-8.0.4-py3.7.egg/setuptools_scm/_config.py", line 11, in <module>
    from typing import Protocol
ImportError: cannot import name 'Protocol' from 'typing' (/usr/lib/python3.7/typing.py)
The command '/bin/sh -c cd /whipper && python3 setup.py install     && rm -rf /whipper     && whipper -v' returned a non-zero code: 1

Trying to build the Image with Docker fails in the same way.

In another project I found an explanation for the error message:

[Bug]: ImportError: cannot import name 'Protocol' from 'typing' · Issue #6515 · AUTOMATIC1111/stable-diffusion-webui · GitHub

As far as I understood, the problem only occurs on Python 3.7 and lower, since Protocol was only added into typing from Python 3.8+, however it is present in typing_extensions

Buster is still using Python 3.7, from the build log:

Selecting previously unselected package python3.7-minimal.
Preparing to unpack .../8-python3.7-minimal_3.7.3-2+deb10u6_amd64.deb ...
Unpacking python3.7-minimal (3.7.3-2+deb10u6) ...
Setting up python3.7-minimal (3.7.3-2+deb10u6) ...
Selecting previously unselected package python3.7.
Preparing to unpack .../7-python3.7_3.7.3-2+deb10u6_amd64.deb ...
Unpacking python3.7 (3.7.3-2+deb10u6) ...
Selecting previously unselected package python3.7-dev.
Preparing to unpack .../113-python3.7-dev_3.7.3-2+deb10u6_amd64.deb ...
Unpacking python3.7-dev (3.7.3-2+deb10u6) ...
Setting up python3.7 (3.7.3-2+deb10u6) ...
running python rtupdate hooks for python3.7...
running python post-rtupdate hooks for python3.7...

Setting the Base-Image to bullseye works: FROM debian:bullseye since bullseye uses Python3.9 (from the log): Setting up python3.9 (3.9.2-1)

STEP 15/22: RUN cd /whipper && python3 setup.py install     && rm -rf /whipper     && whipper -v
/whipper/.eggs/setuptools_scm-8.0.4-py3.9.egg/setuptools_scm/_integration/setuptools.py:30: RuntimeWarning: 
ERROR: setuptools==52.0.0 is used in combination with setuptools_scm>=8.x

Your build configuration is incomplete and previously worked by accident!
setuptools_scm requires setuptools>=61

Suggested workaround if applicable:
 - migrating from the deprecated setup_requires mechanism to pep517/518
   and using a pyproject.toml to declare build dependencies
   which are reliably pre-installed before running the build tools

  warnings.warn(
WARNING setuptools_scm.pyproject_reading toml section missing 'pyproject.toml does not contain a tool.setuptools_scm section'
running install
.....
Adding whipper 0.10.1.dev23+g6ad681a.d20231223 to easy-install.pth file
Installing accuraterip-checksum script to /usr/local/bin
Installing whipper script to /usr/local/bin

Installed /usr/local/lib/python3.9/dist-packages/whipper-0.10.1.dev23+g6ad681a.d20231223-py3.9-linux-x86_64.egg
Processing dependencies for whipper==0.10.1.dev23+g6ad681a.d20231223
Finished processing dependencies for whipper==0.10.1.dev23+g6ad681a.d20231223
whipper 0.10.1.dev23+g6ad681a.d20231223
STEP 16/22: ENV LC_ALL=en_US.UTF-8
.....
STEP 22/22: ENTRYPOINT ["whipper"]
COMMIT charly/whipper
--> 19cb6f8913b2
Successfully tagged localhost/charly/whipper:latest

What the error regarding setuptools means I don't know at the moment. Perhaps somebody should take a look at that. :)

Trying to build with podman using the current Debian Bookworm as a base image fails with another problem in step 4

STEP 4/22: RUN apt-get update && apt-get install --no-install-recommends -y     autoconf     automak
e     cdrdao     bzip2     curl     eject     flac     git     libdiscid0     libiso9660-dev     lib
sndfile1-dev     libtool     locales     make     pkgconf     python3-dev     python3-musicbrainzngs
     python3-mutagen     python3-pil     python3-pip     python3-ruamel.yaml     python3-setuptools 
    sox     swig     && apt-get clean && rm -rf /var/lib/apt/lists/*     && pip3 --no-cache-dir inst
all pycdio==2.1.0 discid
....
Processing triggers for libc-bin (2.36-9+deb12u3) ...
Processing triggers for ca-certificates (20230311) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
Error: building at STEP "RUN apt-get update && apt-get install --no-install-recommends -y     autoconf     automake     cdrdao     bzip2     curl     eject     flac     git     libdiscid0     libiso9660-dev     libsndfile1-dev     libtool     locales     make     pkgconf     python3-dev     python3-musicbrainzngs     python3-mutagen     python3-pil     python3-pip     python3-ruamel.yaml     python3-setuptools     sox     swig     && apt-get clean && rm -rf /var/lib/apt/lists/*     && pip3 --no-cache-dir install pycdio==2.1.0 discid": while running runtime: exit status 1

This also escapes my current knowledge.

And trying to build the image with docker using Debian Bookworm also fails but with another error message:

optical_gid=$(getent group optical | cut -d: -f3) uid=$(id -u) sudo docker build --build-arg optical_gid --build-arg uid -t whipperteam/whipper .

Step 4/22 : RUN apt-get update && apt-get install --no-install-recommends -y     autoconf     automake     cdrdao     bzip2     curl     eject     flac     git     libdiscid0     libiso9660-dev     libsndfile1-dev     libtool     locales     make     pkgconf     python3-dev     python3-musicbrainzngs     python3-mutagen     python3-pil     python3-pip     python3-ruamel.yaml     python3-setuptools     sox     swig     && apt-get clean && rm -rf /var/lib/apt/lists/*     && pip3 --no-cache-dir install pycdio==2.1.0 discid
 ---> Running in e210cc971f3d
Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
Get:2 http://deb.debian.org/debian bookworm-updates InRelease [52.1 kB]
Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8787 kB]
Get:5 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [11.3 kB]
Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [130 kB]
Fetched 9180 kB in 1s (6403 kB/s)
Reading package lists...
E: Problem executing scripts APT::Update::Post-Invoke 'rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true'
E: Sub-process returned an error code
The command '/bin/sh -c apt-get update && apt-get install --no-install-recommends -y     autoconf     automake     cdrdao     bzip2     curl     eject     flac     git     libdiscid0     libiso9660-dev     libsndfile1-dev     libtool     locales     make     pkgconf     python3-dev     python3-musicbrainzngs     python3-mutagen     python3-pil     python3-pip     python3-ruamel.yaml     python3-setuptools     sox     swig     && apt-get clean && rm -rf /var/lib/apt/lists/*     && pip3 --no-cache-dir install pycdio==2.1.0 discid' returned a non-zero code: 100

So at least the base image referenced in the Dockerfile should be set to Bulleye: FROM debian:bullseye

Greetings, KS!

github-actions[bot] commented 9 months ago

👋 Thanks for opening your first issue here! If you're reporting a 🐞 bug, please make sure you include steps to reproduce it. We get a lot of issues on this repo, so please be patient and we will get back to you as soon as we can.

To help make it easier for us to investigate your issue, please follow the contributing instructions.

tmowbray commented 8 months ago

It escapes my current knowledge too, but if I checkout the latest version tag, "v0.10.0", and then edit the Dockerfile to use Bullseye instead of Buster, the build completes.

sjrct commented 4 months ago

Same issue here. Switching to bullseye resolved it