Multi-Architecture Builds on Debian

mfeit-internet2 commented 3 years ago

Accommodations need to be made for building across multiple architectures on Debian.

This needs to be be resolved before rolling Unibuild into production code.

This is possible on RPM but is a hassle. We will likely need to build for aarch64 (which appears to be the same thing as armhf) since Alma supports it.

Determining Architecture

Two possible methods:

Explicit Configuration

Add a top-level file, maybe called unibuild-architectures, that dictates what architectures are cross-built:

# Packaging   Architectures (* = Adds the native architecture if not already explicitly specified)
deb           amd64 armel armhf *
rpm           *

Note that Debian's architecture name for Intel doesn't match uname -m, so the determination will have to be packing-specific.

Automagic

Let let which architectures are installed on the system be the driver of what's built, making system prep responsible for what architectures are supported.

# dpkg --add-architecture armel
# dpkg --add-architecture armhf
# dpkg --add-architecutre amd64
# apt-get update

# (dpkg --print-architecture && dpkg --print-foreign-architectures) | sort | uniq
amd64
armel
armhf

[MF] I like this method better because it doesn't require enumerating architectures in every repository.

Multi-Arch Build

Could use input from @tonin on how this is accomplished.

Possibly-useful reference material:

laeti-tia commented 2 years ago

The way I'm currently doing the multiarch builds is through the scripts in this repository: https://github.com/perfsonar/debian-docker-buildmachines More specifically these scripts:

https://github.com/perfsonar/debian-docker-buildmachines/blob/master/build-in-docker#L34
https://github.com/perfsonar/debian-docker-buildmachines/blob/master/build-in-docker#L141
https://github.com/perfsonar/debian-docker-buildmachines/blob/master/ps-binary-builder#L58
and the Docker setup to have the multiarch containers.

mfeit-internet2 commented 2 years ago

Notes from a conversation with @tonin:

Problem

The concern about not duplicating effort stems from the fact that builds on non-Intel architectures are run under QEMU and run at about 20% of the speed of a native system.

This will probably become an issue with EL systems as well if we start building for ARM.

Solution

Unibuild will be modified to produce a copy of the build order as an artifact. See #10.

The native (x86_64) build will do a full build of everything (binaries and sources).

The result of that build will trigger and be handed to separate Jenkins jobs that do builds for each architecture under QEMU.

That job will be consist of a non-Unibuild script that is given the entire set of artifacts from a native build and does roughly this inside the QEMU environment:

Create an artifact directory where the built packages will be stored.

For each package in the build order:

  If the package in the native build is architecture-independent:

    Copy the pre-built binary package from the native build into the artifact directory
    Install the pre-built package

  Else:

    Build the package from the source package (absent any involvement from Unibuild)
    Copy the resulting binary package into the artifact directory
    Install the just-build package

laeti-tia commented 2 years ago

Additionally to that, I think we'd want to keep a few more build artifacts, at the very least, the source package to build the other binaries. For a package named pkgname it would be nice to have unibuild produce the following artifacts:

pkgname_version-revision.debian.tar.xz
pkgname_version-revision.dsc
pkgname_version-revision_amd64.build
pkgname_version-revision_amd64.buildinfo
pkgname_version-revision_amd64.changes
pkgname_version-revision_amd64.deb
pkgname_version-revision_source.buildinfo
pkgname_version-revision_source.changes
pkgname_version.orig.tar.bz2

And then the binary build would produce the artifacts for the other architectures, something like:

pkgname_version-revision_arm64.buildinfo
pkgname_version-revision_arm64.changes
pkgname_version-revision_arm64.deb
pkgname_version-revision_armhf.buildinfo
pkgname_version-revision_armhf.changes
pkgname_version-revision_armhf.deb
pkgname_version-revision_ppc64el.buildinfo
pkgname_version-revision_ppc64el.changes
pkgname_version-revision_ppc64el.deb

And additionally the dbgsym packages as well.

mfeit-internet2 commented 2 years ago

There doesn't seem to be much more that we can do on this from the Unibuild end. Closing.

perfsonar / unibuild