Pre-compile packages for faster install/upgrade

[selfhoster1312]

Discussed during the last Yunocamp

It would be great, especially for lower end systems where compilation is super slow or fails due to low memory available, to have pre-compiled packages for Yunohost applications. For example, invidious can take several minutes to install even on a powerful machine.

There's two sides for pre-compilation:

• actual binary compilation for backend: requires per-architecture targets (eg. raspberry pi, amd64...)
• frontend build for web apps: architecture-independent

There's also the question of how to distribute these pre-packaged assets:

• custom Debian repository? @alexAubin seemed horrified at the idea
• simple web server with checksums? then we have question of who gets rights to upload there
• using Github releases artifacts? requires using Github and Github Actions but does not introduce further maintenance for Yunohost project sysadmins (and would be compatible with Gitea actions if we ever migrate off Github)

For backend programs, there's also the question of linking against system libraries. Most programs will require using libc, openssl, or libpcre. The binary layout of the ABI changes with many releases, which is why some folks do static linking against a specific version. Here's the pros and cons:

• static linking: by linking against a specific version of system libraries that are distributed with the binary, updates don't break the program (pro) but updates don't fix the program for example for security vulnerabilities in TLS stack (con)
• dynamic linking: Debian packages should not break library ABIs between major version upgrades (eg. bullseye to bookworm) which is good, but doesn't solve the problem of dependencies not packages by debian (or when the debian version is too old)

Some flavors to explore for binary distribution:

• AppImage format for self-extracting packages ; great but requires CAP_SYS_ADMIN capability for mounting the image (or maybe just CAP_SYS_USER_NS ??)
• nix/guix packaging: best of both (static/dynamic) linking, but requires learning an entirely new paradigm for packaging things

Another option is to pre-compile binary objects to be linked dynamically on the Yunohost instance, because the linking step is usually fast and low on resource usage. I will post more information where i ran practical tests

[selfhoster1312]

Case study: invidious

Tried approaches:

• AppImage using nix-appimage: not selected because it requires giving too much capabilities to the systemd service, but works great!!!
• crystal cross-compilation: only works on same Debian release due to GLIBC ABI changes ; not selected because it requires bullseye for building, and invidious will crash after Yunohost migration to bookworm unless invidious package is upgraded too (that sucks!)
• crystal static compilation using musl libc : works great?!

I went with the last option because we can cross-compile to a different architecture/libc using a container/VM and it should not break across upgrades.

$ docker run --rm -it crystallang/crystal:latest-alpine sh
# apk add sqlite-static
# git clone https://github.com/iv-org/invidious
# cd invidious
# make STATIC=1
# # PUBLISH `invidious` somewhere

I've started work on a Github action to automate it but so far i find it "meh". See here. So instead i've used gh command line to publish a release with my own build artifacts. I'll revisit this soon.

There's the question of how to cross-compile to other architectures, but this should be easily done with qemu. There's many OCI images and github actions to address this i just haven't tried them so far, and i need to put my hands on an original Raspberry Pi to confirm if it works.

If you wanna try this pre-compiled package (amd64 only) it's here:

yunohost app install -a "domain=tube.DOMAIN.TLD&path=/&init_main_permission=visitors" -l youtube https://github.com/selfhoster1312/invidious_ynh

I still have a lot of stuff to do (fix upgrade script, make sure locales and prebuilt binary stay in sync, etc) it's just a demo don't use it for real folks. But to give an example, it took 44.676s to install on my server, while the "real" invidious package took 3m33,825s.

[alexAubin]

Related (example use case) : https://github.com/YunoHost-Apps/synapse_python_build