daq-tools / kotori

A flexible data historian based on InfluxDB, Grafana, MQTT, and more. Free, open, simple.
https://getkotori.org/
GNU Affero General Public License v3.0
108 stars 17 forks source link

Packaging: `make package-baseline-images` croaks when building arm64v8 images #73

Closed amotl closed 2 years ago

amotl commented 2 years ago

Hi there,

while working on #64, by running the packaging machinery on a Linux system, we discovered that the Building baseline image for Debian "bullseye" on arm64v8 step croaks when installing libc-bin, coming from

RUN apt-get install --yes --no-install-recommends inetutils-ping nano git build-essential pkg-config libffi-dev ruby ruby-dev

The error manifests itself as

Processing triggers for libc-bin (2.31-13+deb11u2) ...
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault
dpkg: error processing package libc-bin (--configure):
 installed libc-bin package post-installation script subprocess returned error exit status 139
Errors were encountered while processing:
 libc-bin
E: Sub-process /usr/bin/dpkg returned an error code (1)
The command '/bin/sh -c apt-get install --yes --no-install-recommends     inetutils-ping nano git     build-essential pkg-config libffi-dev     ruby ruby-dev' returned a non-zero code: 100
Command failed
make: *** [packaging/tasks.mk:13: package-baseline-images] Error 1

With kind regards, Andreas.

amotl commented 2 years ago

Currently I have no idea why this happens. Some observations and thoughts:

[1] cat /proc/cpuinfo | grep "model name" | uniq

amotl commented 2 years ago

Hi again,

in this post we outline a minimal example how to reproduce the problem.

  1. Follow the first few sections of https://getkotori.org/docs/development/releasing/packaging.html, specifically 1.1. Install Docker 1.2. Install more packages to support multiarch builds: apt-get install qemu-user-static binfmt-support

  2. Invoke this docker run command, targeting arm64v8:

    docker run --pull --rm -it --env DEBIAN_FRONTEND=noninteractive arm64v8/debian:bullseye-slim dpkg-reconfigure libc-bin
    qemu: uncaught target signal 11 (Segmentation fault) - core dumped
    Segmentation fault
    qemu: uncaught target signal 11 (Segmentation fault) - core dumped
    Segmentation fault
  3. As mentioned above, targeting arm32v7 has no issues:

    docker run --pull --rm -it --env DEBIAN_FRONTEND=noninteractive arm32v7/debian:bullseye-slim dpkg-reconfigure libc-bin

With kind regards, Andreas.

amotl commented 2 years ago

Hi again,

at https://github.com/moby/moby/issues/43039#issuecomment-974830858, we found a solution to this problem. Apparently, we needed a more recent version of qemu-aarch64-static. While it didn't work with version 4.2.1, version 5.0.0 from [1] works like a charm.

$ docker run --pull --rm -it arm64v8/debian:bullseye-slim /bin/sh -c "ldconfig --version"

ldconfig (Debian GLIBC 2.31-13+deb11u2) 2.31
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Andreas Jaeger.

With kind regards, Andreas.

[1] http://security.ubuntu.com/ubuntu/pool/universe/q/qemu/qemu-user-static_5.0-5ubuntu9.9_amd64.deb

amotl commented 2 years ago

With 3b029ca9, we just updated the documentation and added this to the troubleshooting section, see [2].

[2] https://getkotori.org/docs/development/releasing/packaging.html#troubleshooting

amotl commented 2 years ago

Others are also confirming that this has been resolved with Qemu 5, see [3].

[3] https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1928075/comments/15