search

silkeh / docker-clang

Dockerfiles for Clang and LLVM
European Union Public License 1.2
97 stars 30 forks source link

Separate build jobs for each architecture #29

Open jdoubleu opened 1 year ago

jdoubleu commented 1 year ago

Wanted to continue the discussion from https://github.com/silkeh/docker-clang/issues/28#issuecomment-1780092404 in a dedicated issue.

So I've looked into how these new multi-arch images work.

The last time I did some work on this it seemed like the image needed to be pushed for all architectures at the same time.

This seems to be the recommended and easiest way to handle multi-arch images.

However, I found a way to build these images independently and stitch them together afterwards (see "The hard way with docker manifest" https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/). This would require us to upload each image for each architecture with its own tag to the registry, then creating a new "tag" or manifest which combines them. Afterwards the initial tags can be deleted, because their layers are referenced by the new manifest.

Interestingly, the docker manifest create command is marked experimental and not recommended for production. On the other hand, there's docker buildx imagetools create, which might work the same way.

This is the best I can come up with, yet. I don't like the fact, that we're required to push the "intermediate" images first, and I'm not sure if that's an acceptable solution. Furthermore it would increase the complexity of the task. What do you think?

I also couldn't find 1) how to improve the readability of the logs (only through --output=plain) nor 2) allow individual architectures to fail, while keeping the successfully build, as alternative solutions.

silkeh commented 11 months ago

Just FYI: it seems the packaging issues were resolved upstream, so all images should be available in all flavours (see this pipeline).

silkeh commented 11 months ago

This is the best I can come up with, yet. I don't like the fact, that we're required to push the "intermediate" images first, and I'm not sure if that's an acceptable solution. Furthermore it would increase the complexity of the task. What do you think?

If that's the only issue, I think it would be acceptable to push the images with a suffix in the tag (eg: :17-bookworm-arm64). That also allows those explicit images to be published when the others aren't. For example: if the 17-bookworm-s390x build fails, the 17-bookworm-amd64 image will be available, but the multi-architecture image (17-bookworm) will stay at the latest version for which all platforms succeeded.

excitoon commented 6 months ago

It would be awesome if you could also backport multiarch builds up to early versions. 🙏