Alpine linux based container image

[vitoyucepi]

Is your feature request related to a problem? Please describe. I'd like to see very small container image. I think the image size will be less than 150MB. In comparison ubuntu based image size is about 450MB.

I have tried to create container image and it works. But then I failed because of segmentation faults here and there.

Describe the solution you'd like I think there are two possible ways.

1. Maintain liquidsoap and all ocaml dependencies in the alpine repo.
2. Create container that will build all dependencies from local APKBUILD files.

Additional context Some ocaml packages have been already added to alpine repository https://gitlab.alpinelinux.org/alpine/aports/-/tree/master/unmaintained. But now they are unmaintained. You can use my own APKBUILD files if you want to.

To maintain dependencies versions in my own repo I use renovate bot.

[toots]

That's a great idea, it would also be a good basis for building a static binary. I'll see if I can find time to look at it.

[vitoyucepi]

You can re-license APKBUILD files or any other files under any license compatible with liquidsoap if you have to.

[toots]

Hi!

I have added alpine builds for amd64 and arm64: https://github.com/savonet/liquidsoap/actions/runs/944792743

The base image is pretty big because it contains all the files required to build the binary. However, I believe that, from there, we should be able to build an alpine package based on the installed liquidsoap build.

Is that something that you could help us with? Basically, if you could write the alpine packaging part, I'd be happy to add them to the CI like we do for debian packages and export the alpine package as a build artifact and, eventually, an asset associated with the release.

Lastly, we could also make a minimal docker image for alpine by simply installing the extracted alpine package, just like we do with the debian testing package..

[vitoyucepi]

Hello

I'd be happy to add them to the CI like we do for debian packages and export the alpine package as a build artifact and, eventually, an asset associated with the release.

I'm not sure if it would be right to build everything from opam instead of building package-by-package. Also I think alpine:3.13 or alpine:3.14 should be used as a base image, because in alpine:edge something could be broken.

If I would create packages right, then I will try to create my own custom repo for liquidsoap. Originally I thought the best place for packages is official repo, but I have no idea how to maintain package properly. I have some experience creating APKBUILD files.

As for the docker image the solutions will look like

1. Create package repo first and then simply maintain packages there like aur in archlinux. So to create default image in this case you have simply to add custom repo and install packages. I think this way is used in mariadb image.
2. Create packages in official repos.
3. Build everything in container like in php without any intermediate states. Just copy all required files in their places by hand or by make install to /usr/local.
4. Build all packages each time building the container like in nginx. I also use this idea in my own way because there's no need to register domain, host repository and so on.

So I have to do some research on how liquidsoap ci works.

[toots]

Hi!

I believe creating packages with all the OCaml modules needed to build liquidsoap would be a lot of work for not much benefit: once built, liquidsoap does not need anything from them except some static files from camomile.

Instead, if you think about it, after we're done building and installing liquidsoap inside the docker image, we should pretty much be at the last stage of making a binary package. I don't know how apk packages work in detail but, in debian world, all you have to do from there is direct the install to a local directory, something like:

/path/to/chroot

Where the binary is installed as if it were at the root of your filesystem:

/path/to/chroot/usr/bin/liquidsoap
/path/to/chroot/usr/lib/liquidsoap/stdlib.liq
....

And you can finally package it.

Is that any different with apk?

[vitoyucepi]

In APKBUILD(alpine) and PKGBUILD(arch) you have to execute

1. prepare
   • To extract packages, if they were not extracted automatically from archive.
   • To apply patches, if they are not in standard format.
2. build
   • To configure and build.
3. package
   • To copy build artifacts and additional files to special path.

For example

package() {
  cp "$srcdir/dist/bin/my_program" "$pkdir/usr/bin/my_program"
}

It's possible to copy files from global scope without prepare and build. So no dependency management by packaging tools, no building, simply copy already built files to $pkgdir. In alpine packaging utility will also try to find *.so dependencies in binary files.

So for liquidsoap we have to copy

• liquidsoap binary
• camomile *.mar
• *.liq scripts

[toots]

Yes, that's exactly that! The build system actually has something designed just for this, if you set a DESTDIR environment variable, here's how it's done for the debian package:

    make install DESTDIR=$(DESTDIR) OCAMLFIND_DESTDIR=$(DESTDIR)/$(OCAML_STDLIB_DIR) \
               prefix=$(DESTDIR)/usr sysconfdir=$(DESTDIR)/etc \
               INSTALL_DAEMON=no OCAMLFIND_LDCONF=ignore
    mkdir -p $(DESTDIR)/usr/share/liquidsoap
    cp -rf `ocamlfind ocamlc -where`/../../share/camomile $(DESTDIR)/usr/share/liquidsoap

This should install every file you need under the path referred to in DESTDIR!

[toots]

I'm working in it! Check this out: https://hub.docker.com/layers/155848290/savonet/liquidsoap-ci-build/alpine-apk_amd64/images/sha256-3615f97d7244b7c79f6de6194886347f107081c3e10c7bccb98ea70cd7e511ac?context=explore

55 MB instead of 250 MB indeed..!

Still having some issues uploading/downloading assets in alpine/arm64 but I think we can make it work!

[vitoyucepi]

Some cleaning for APKBUILD.

--- a/APKBUILD
+++ b/APKBUILD
@@ -5,27 +5,22 @@ pkgdesc="Swiss-army knife for multimedia streaming"
 url="https://github.com/savonet/liquidsoap"
 arch="all"
 license="GPL-2.0-only"
-depends=""
-makedepends=""
-checkdepends=""
-install=""
-source=""
-
-build() {
-        # Replace with proper build command(s)
-        :
-}
-
-check() {
-        # Replace with proper check command(s)
-        :
-}
+depends="curl"
+install="liquidsoap.pre-install"
+options="!check"

 package() {
        eval $(opam env)
-        cd liquidsoap
-        make install DESTDIR="$pkgdir" OCAMLFIND_DESTDIR="$pkgdir"/"$OCAML_STDLIB_DIR" prefix="$pkgdir"/usr sysconfdir="$pkgdir"/etc INSTALL_DAEMON=no OCAMLFIND_LDCONF=ignore
-        mkdir -p "$pkgdir"/usr/share/liquidsoap
-        cp -rf `ocamlfind ocamlc -where`/../../share/camomile "$pkgdir"/usr/share/liquidsoap
-        :
+       cd liquidsoap
+
+       make install \
+               DESTDIR="$pkgdir" \
+               OCAMLFIND_DESTDIR="$pkgdir/$OCAML_STDLIB_DIR" \
+               prefix="$pkgdir/usr" \
+               sysconfdir="$pkgdir/etc" \
+               INSTALL_DAEMON=no \
+               OCAMLFIND_LDCONF=ignore
+
+       mkdir -p "$pkgdir/usr/share/liquidsoap"
+       cp -rf "$(ocamlfind ocamlc -where)/../../share/camomile" "$pkgdir/usr/share/liquidsoap"
 }

I pin curl as a dependency and disable check script by options.

Also we can use install scripts to do something with user and groups. So there's no need to do it in Dockerfile. liquidsoap.pre-install

#!/bin/sh

addgroup -S liquidsoap 2>/dev/null
adduser -S -D -h /var/liquidsoap -s /sbin/nologin -G liquidsoap -g liquidsoap liquidsoap 2>/dev/null
addgroup liquidsoap audio 2>/dev/null

exit 0

[vitoyucepi]

I still think that APKBUILD should use rules for generic *.sh files. So

cp -rf "$(ocamlfind ocamlc -where)/../../share/camomile" "$pkgdir/usr/share/liquidsoap"

Also there's no need to use : symbol in non-empty functions. By the way tab is the recommended symbol for indentation instead of spaces, but there's no problem if file will use only spaces or only tabs.

Setting curl as as dependency and creating pre-install script are must have if .apk file will be distributed by it's own.

depends="curl"
install="liquidsoap.pre-install"

To disable check this has to be a part of APKBUILD

options="!check"

Also I personally think that in busybox ash single [ should be used

if [ "${ARCH}" = "amd64" ]; then

[toots]

Word thanks. The tabs must have been lost in the copy/paste.

Thanks for the review, I've applied most changes. Why is curl required for package distribution?

[vitoyucepi]

In version 1.4 it was a dependency. As I see now it has been traced https://github.com/savonet/liquidsoap/runs/2925819287#step:10:178. Also I see some spaces here <tab><space><space>

      DESTDIR="$pkgdir" \
      OCAMLFIND_DESTDIR="$pkgdir/$OCAML_STDLIB_DIR" \
      prefix="$pkgdir/usr" \
      sysconfdir="$pkgdir/etc" \
      INSTALL_DAEMON=no \
      OCAMLFIND_LDCONF=ignore

Btw maybe build against alpine:3.14 without testing repo? It's not really outdated, because it has released 2 weeks ago. And in docker it's not problem to move from one version to another.

[toots]

In version 1.4 it was a dependency. As I see now it has been traced https://github.com/savonet/liquidsoap/runs/2925819287#step:10:178. Also I see some spaces here <tab><space><space>

    DESTDIR="$pkgdir" \
    OCAMLFIND_DESTDIR="$pkgdir/$OCAML_STDLIB_DIR" \
    prefix="$pkgdir/usr" \
    sysconfdir="$pkgdir/etc" \
    INSTALL_DAEMON=no \
    OCAMLFIND_LDCONF=ignore

Btw maybe build against alpine:3.14 without testing repo? It's not really outdated, because it has released 2 weeks ago. And in docker it's not problem to move from one version to another.

Yeah, I like to indent a little more after the line break and two tabs is too big.

Are you saying that alpine:3.14 can build the whole suite without the need of the additional repository? I had to add the testing repo to enable some features.

[vitoyucepi]

Oh, I see liblo and shine.

[toots]

Fixed in https://github.com/savonet/liquidsoap/commit/c6da3a88c9e41a336c2bb773904a0a7183972bd6