search

AppImageCrafters / appimage-builder

GNU/Linux packaging solution using the AppImage format
MIT License
316 stars 58 forks source link

Running appimage-builder in docker and having RuntimeError: "/usr/bin/apt-cache show dpkg debconf apt" #174

Closed alevikpes closed 2 years ago

alevikpes commented 2 years ago

Hi! I am trying to setup a docker image for running appimage-builder for my app. I use the Dockerfile from the appimage-builder repo and add there my own stuff, because I was not able to make the original working.

So, the building docker image works fine, but when I try running it I get error. Here is the last part of the log:

INFO:main:Running apt deploy
INFO:apt:apt-get update
Reading package lists...
E: No packages found
ERROR:root:"/usr/bin/apt-cache show dpkg debconf apt" execution failed
Traceback (most recent call last):
  File "/usr/local/bin/appimage-builder", line 33, in <module>
    sys.exit(load_entry_point('appimage-builder==0.9.2', 'console_scripts', 'appimage-builder')())
  File "/usr/local/lib/python3.8/site-packages/appimage_builder-0.9.2-py3.8.egg/appimagebuilder/__main__.py", line 58, in __main__
    invoker.execute(commands)
  File "/usr/local/lib/python3.8/site-packages/appimage_builder-0.9.2-py3.8.egg/appimagebuilder/invoker.py", line 41, in execute
    command()
  File "/usr/local/lib/python3.8/site-packages/appimage_builder-0.9.2-py3.8.egg/appimagebuilder/commands/apt_deploy.py", line 46, in __call__
    deployed_packages = apt_deploy.deploy(
  File "/usr/local/lib/python3.8/site-packages/appimage_builder-0.9.2-py3.8.egg/appimagebuilder/modules/deploy/apt/deploy.py", line 41, in deploy
    self._prepare_apt_venv()
  File "/usr/local/lib/python3.8/site-packages/appimage_builder-0.9.2-py3.8.egg/appimagebuilder/modules/deploy/apt/deploy.py", line 56, in _prepare_apt_venv
    apt_core_packages = self.apt_venv.search_packages(listings.apt_core)
  File "/usr/local/lib/python3.8/site-packages/appimage_builder-0.9.2-py3.8.egg/appimagebuilder/modules/deploy/apt/venv.py", line 238, in search_packages
    output = self._run_apt_cache_show(names)
  File "/usr/local/lib/python3.8/site-packages/appimage_builder-0.9.2-py3.8.egg/appimagebuilder/modules/deploy/apt/venv.py", line 152, in _run_apt_cache_show
    shell.assert_successful_result(_proc)
  File "/usr/local/lib/python3.8/site-packages/appimage_builder-0.9.2-py3.8.egg/appimagebuilder/utils/shell.py", line 36, in assert_successful_result
    raise RuntimeError(
RuntimeError: "/usr/bin/apt-cache show dpkg debconf apt" execution failed with code 100

I do not understand this error and can't find any useful information about it.

It fails on the script execution of my recipe:

# appimage-builder recipe see https://appimage-builder.readthedocs.io for details
version: 1

script:
  # Remove any previous build
  - rm -rf AppDir | true
  # Make usr and icons dirs
  - mkdir -p AppDir/usr/src
  # Copy the python application code into the AppDir
  - cp src AppDir/usr/ -r
  # Install application dependencies
  - python3 -m pip install --upgrade --ignore-installed --prefix=/usr --root=AppDir -r ./requirements.txt

AppDir:
  path: /app/AppDir
  app_info:
    id: org.alevikpes.media-player
    name: media-player
    icon: utilities-terminal
    version: latest
    exec: "usr/bin/python3"
    exec_args: "${APPDIR}usr/src/main.py $@"
  apt:
    arch: amd64
    allow_unauthenticated: true
    sources: []
    #    - sourceline: deb http://nl.archive.ubuntu.com/ubuntu/ focal main restricted
    #    - sourceline: deb http://nl.archive.ubuntu.com/ubuntu/ focal-updates main restricted
    #    - sourceline: deb http://nl.archive.ubuntu.com/ubuntu/ focal universe
    #    - sourceline: deb http://nl.archive.ubuntu.com/ubuntu/ focal-updates universe
    #    - sourceline: deb http://nl.archive.ubuntu.com/ubuntu/ focal multiverse
    #    - sourceline: deb http://nl.archive.ubuntu.com/ubuntu/ focal-updates multiverse
    #    - sourceline: deb http://nl.archive.ubuntu.com/ubuntu/ focal-backports main restricted
    #        universe multiverse
    #    - sourceline: deb http://security.ubuntu.com/ubuntu focal-security main restricted
    #    - sourceline: deb http://security.ubuntu.com/ubuntu focal-security universe
    #    - sourceline: deb http://security.ubuntu.com/ubuntu focal-security multiverse
    include:
      - libgl-dev
      - python3
        #      - python3-pyqt5
        #      - qt5ct
      - vlc
        #      - wmctrl
        #      - xdotool
        #      #QT5 related packages
        #      - libcanberra-gtk3-module
        #      - qt5-style-plugins
        #      - qt5-gtk-platformtheme
        #      - qt5-style-kvantum
        #      - qt5-style-kvantum-themes
    exclude: []
        #      - launchpadlib
        #      - libfontconfig1
        #      - libfreetype6
        #      - libharfbuzz0b
  files:
    include:
      - usr/src/core.py
      - usr/src/generate_headers.py
      - usr/src/gui.py
      - usr/src/main.py
      - usr/src/requirements.txt
      - usr/src/vimeo_manager.py
      - usr/src/youtube_manager.py
    exclude:
      - usr/share/man
      - usr/share/doc/*/README.*
      - usr/share/doc/*/changelog.*
      - usr/share/doc/*/NEWS.*
      - usr/share/doc/*/TODO.*
  runtime:
    env:
      PATH: "${APPDIR}/usr/bin:${PATH}"
      # Set python home
      # See https://docs.python.org/3/using/cmdline.html#envvar-PYTHONHOME
      PYTHONHOME: "${APPDIR}/usr"
      # Path to the site-packages dir or other modules dirs
      # See https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPATH
      #PYTHONPATH: '${APPDIR}/usr:${PYTHONPATH}'
      PYTHONPATH: "${APPDIR}/usr/lib/python3.8/site-packages:${APPDIR}/usr:${PYTHONPATH}"
      QT_QPA_PLATFORMTHEME: "${QT_QPA_PLATFORMTHEME}"
  test:
    fedora-30:
      image: appimagecrafters/tests-env:fedora-30
      command: ./AppRun
      use_host_x: true
    debian-stable:
      image: appimagecrafters/tests-env:debian-stable
      command: ./AppRun
      use_host_x: true
    archlinux-latest:
      image: appimagecrafters/tests-env:archlinux-latest
      command: ./AppRun
      use_host_x: true
    centos-7:
      image: appimagecrafters/tests-env:centos-7
      command: ./AppRun
      use_host_x: true
    ubuntu-xenial:
      image: appimagecrafters/tests-env:ubuntu-xenial
      command: ./AppRun
      use_host_x: true

AppImage:
  arch: x86_64
  update-information: guess

And here is the Dockerfile I am using:

FROM python:3.8

RUN apt-get update \
    && apt-get -y upgrade \
    && apt-get -y install \
        desktop-file-utils \
        elfutils \
        fakeroot \
        file \
        g++ \
        gnupg2 \
        gtk-update-icon-cache \
        libgdk-pixbuf2.0-dev \
        libgl-dev \
        libglib2.0-bin \
        librsvg2-dev \
        libyaml-dev \
        p7zip-full \
        python3-dev \
        python3-pip \
        python3-setuptools \
        strace \
        tree \
        vim \
        wget \
        zsync

WORKDIR /opt/
RUN git clone https://github.com/AppImageCrafters/appimage-builder.git 
RUN cd ./appimage-builder \
    && python3 ./setup.py install \
    && cd \
    && rm -rf /opt/appimage-builder

WORKDIR /tmp
RUN wget https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-x86_64.AppImage \
    && chmod +x /tmp/appimagetool-x86_64.AppImage \
    && cd /opt \
    && /tmp/appimagetool-x86_64.AppImage --appimage-extract \
    && mv /opt/squashfs-root /opt/appimage-tool.AppDir \
    && ln -s /opt/appimage-tool.AppDir/AppRun /usr/bin/appimagetool \
    && rm /tmp/appimagetool-x86_64.AppImage

RUN wget https://github.com/NixOS/patchelf/releases/download/0.12/patchelf-0.12.tar.bz2; \
    tar -xvf patchelf-0.12.tar.bz2; \
    cd patchelf-0.12.20200827.8d3a16e; \
    ./configure && make && make install; \
    rm -rf patchelf-*

WORKDIR /
RUN apt-get -y autoclean

ADD . /app/
WORKDIR /app/

CMD ["appimage-builder", "--recipe", "AppImageBuilder.yml"]
azubieta commented 2 years ago

To make apt work you need to add at least one source in your recipe. Otherwise it will not know where to get the packages from. It's recommended to use the same sources that are configured in the host system.

  apt:
    arch: amd64
    allow_unauthenticated: true
    sources: []
alevikpes commented 2 years ago

Thanks for the tip. I would actually expect, this would be filled in automatically, since it is the required argument. Where is this part in the code? Is it possible to make reading the list file from /etc/apt/ automatically?

alevikpes commented 2 years ago

After fixing the sources issue, I got the following docker error:

WARNING:root:Unable to locate the application desktop entry: org.alevikpes.media-player.desktop
INFO:main:Running deploy record generation
INFO:root:Writing deploy record to: .bundle.yml
INFO:main:Running AppDir tests
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3-1.26.8-py3.8.egg/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3-1.26.8-py3.8.egg/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.8/http/client.py", line 1256, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.8/http/client.py", line 1302, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.8/http/client.py", line 1251, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.8/http/client.py", line 1011, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.8/http/client.py", line 951, in send
    self.connect()
  File "/usr/local/lib/python3.8/site-packages/docker-5.0.3-py3.8.egg/docker/transport/unixconn.py", line 30, in connect
    sock.connect(self.unix_socket)
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/requests-2.27.1-py3.8.egg/requests/adapters.py", line 440, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/site-packages/urllib3-1.26.8-py3.8.egg/urllib3/connectionpool.py", line 785, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/site-packages/urllib3-1.26.8-py3.8.egg/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/site-packages/urllib3-1.26.8-py3.8.egg/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.8/site-packages/urllib3-1.26.8-py3.8.egg/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3-1.26.8-py3.8.egg/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.8/http/client.py", line 1256, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.8/http/client.py", line 1302, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.8/http/client.py", line 1251, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.8/http/client.py", line 1011, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.8/http/client.py", line 951, in send
    self.connect()
  File "/usr/local/lib/python3.8/site-packages/docker-5.0.3-py3.8.egg/docker/transport/unixconn.py", line 30, in connect
    sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/docker-5.0.3-py3.8.egg/docker/api/client.py", line 214, in _retrieve_server_version
    return self.version(api_version=False)["ApiVersion"]
  File "/usr/local/lib/python3.8/site-packages/docker-5.0.3-py3.8.egg/docker/api/daemon.py", line 181, in version
    return self._result(self._get(url), json=True)
  File "/usr/local/lib/python3.8/site-packages/docker-5.0.3-py3.8.egg/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/docker-5.0.3-py3.8.egg/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.8/site-packages/requests-2.27.1-py3.8.egg/requests/sessions.py", line 542, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests-2.27.1-py3.8.egg/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests-2.27.1-py3.8.egg/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests-2.27.1-py3.8.egg/requests/adapters.py", line 501, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/appimage-builder", line 33, in <module>
    sys.exit(load_entry_point('appimage-builder==0.9.2', 'console_scripts', 'appimage-builder')())
  File "/usr/local/lib/python3.8/site-packages/appimage_builder-0.9.2-py3.8.egg/appimagebuilder/__main__.py", line 58, in __main__
    invoker.execute(commands)
  File "/usr/local/lib/python3.8/site-packages/appimage_builder-0.9.2-py3.8.egg/appimagebuilder/invoker.py", line 41, in execute
    command()
  File "/usr/local/lib/python3.8/site-packages/appimage_builder-0.9.2-py3.8.egg/appimagebuilder/commands/run_test.py", line 31, in __call__
    test_cases = self._load_tests(self.tests_settings())
  File "/usr/local/lib/python3.8/site-packages/appimage_builder-0.9.2-py3.8.egg/appimagebuilder/commands/run_test.py", line 50, in _load_tests
    test = ExecutionTest(
  File "/usr/local/lib/python3.8/site-packages/appimage_builder-0.9.2-py3.8.egg/appimagebuilder/modules/test/execution_test.py", line 37, in __init__
    self.client = docker.from_env()
  File "/usr/local/lib/python3.8/site-packages/docker-5.0.3-py3.8.egg/docker/client.py", line 96, in from_env
    return cls(
  File "/usr/local/lib/python3.8/site-packages/docker-5.0.3-py3.8.egg/docker/client.py", line 45, in __init__
    self.api = APIClient(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/docker-5.0.3-py3.8.egg/docker/api/client.py", line 197, in __init__
    self._version = self._retrieve_server_version()
  File "/usr/local/lib/python3.8/site-packages/docker-5.0.3-py3.8.egg/docker/api/client.py", line 221, in _retrieve_server_version
    raise DockerException(
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
azubieta commented 2 years ago

Tests doesn't work inside Docker as they require a docker container to run the test. Try running it using --skip-test

azubieta commented 2 years ago

Closing this as solved. Please feel free to reopen if required