Align Debian package versions with GitHub releases (aka APT-Pinning)

[Berbe]

As a best-practice building Dockerfiles, people often recommend to pin the package versions the image needs to build. That way you can be sure that the build is deterministic. How do you feel about that?

We cannot do that right now, because images are built using our Debian repository.

Besides that point, the images use the pre-built packages in debian repository. I could not easily find were all debian packages are built. Can someone point me in the right direction?

We build tghem on a Jenkins machine. It's a process we trigger manually (for the stable repository) when we want to promote a release to stable.

Originally posted by @saghul in https://github.com/jitsi/docker-jitsi-meet/pull/560#issuecomment-626540317

At the moment, Debian packages pulled by Dockerfile recipes reference no specific version, merely pulling the latest package from added repositories. As discussed in the linked PR, a major improvement on the GitHub releases cycle would be to have deterministic releases, ie content of which do not change over time.

@saghul

1. Is there a way to guess Debian packages version from GitHub docker-jitsi-meet releases?
2. Or, better, would there be a way to specify each & every pulled Debian package version with each release, for example as build argument (ARG) inside each Dockerfile?

This way, it's only a matter of adding APT-pinning commands inside each Dockerfile to make each release self-sufficient.

[sapkra]

1. There are two ways to find out the package releases of the current version: a) e.g. jicofo is currently version 1.0-567-1. The 567 is a tag which you can find at the jicofo repository. The same commit which was tagged also has a tag called jitsi-meet_4548. There we have the current release number for stable-4548. b) Here you can see the current stable releases: https://download.jitsi.org/stable/ There is one package which is like a divider jitsi-meet_2.0.XXXX. The XXXX is the release id e.g. 4548. So basically all packages below a package named like this are part of the current release. They also have the correct version number. c) You can pull the image and just run apt-cache policy PACKAGE_NAME inside it.
2. This was also my thought and I really think that this is the right way to go. Maybe someone can write a nice script which automatically detects the current versions when doing a new release.

[saghul]

I hadn’t thought of apt policy, good call!

Now, in terms of practicality I’d like to understand what problem we’re solving here.

I get that you can’t build a specific version. So what? I create releases for each stable release, so you can pin your container images to the specific version you like.

Just to be clear, I’m not against doing this, but time is limited and I think there are more important things where time could be spent on. If either of you is up for sending a PR, I’ll happily review it.

[Berbe]

@saghul The problem we here solving here, as described in my 1st post, is time-consistent Docker recipes: ensure than built images grab a precise version of the package, whenever the image is built, ie whatever version the latest release of a package is in a pulled APT repository inside an image definition.

@sapkra If packages are tagged with the exact version of the release, I guess this version can be defined in a central location (Makefile?) and propagated throughout templates with the help of variables.

I make no promises, but a PR might be easily achievable. Time is limited for everyone. :)

[saghul]

What you described is what you want to achieve not a problem you have that needs solving.

Yes, the current system has limitations, we know that. But given the pace at which the project advances I don’t think being able to replicate an old build has much value, since one can use an old image.

Again, if this is something you think can be solved easily let’s look at it when you have a PR! No rush.

[sapkra]

What will may be a problem: if you change something and build the images by yourself while your changes are not based on the current master your containers might break because when you rebuild your working fork it might break because of breaking changes in the lastest debian packages (jitsi, prosody etc.). If they are locked to a specific version it will never happen.

[vkruoso]

Not having this makes it hard to rebuild the container in other environments. Tools like hadolint are good to catch all kinds of problems with Dockerfiles, like the package pinning: here.

Do you intend to have a CI tool here that builds and pushes those images automatically? By your statement seems that the release process is very manual right now.

[Berbe]

What will may be a problem: if you change something and build the images by yourself while your changes are not based on the current master your containers might break because when you rebuild your working fork it might break because of breaking changes in the lastest debian packages (jitsi, prosody etc.). If they are locked to a specific version it will never happen.

That is... precisely what I meant, worded otherwise.

Releases are there to ensure a coherent state of all building blocks, each in a very specific version. In the current state of the cooking recipes, trying to build an old release in the future will be impossible, as it is always the latest package from each APT repository which will be pulled.

Cue APT-Pinning or APT policies to ensure a specific package version is pulled from repositories in the Dockerfile recipes for each and every release version.

[saghul]

We don't keep all Debian packages around forever, so that's not future-proof either...