multi-platform tested container build

[akostadinov]

This one enables arm64 build adds smoke tests for the built images adds dependency updates from dependabot container image badge to the readme

[akostadinov]

For accessing certificates, probably it would be necessary to mount a volume in podman/docker using the -v local/path:container/path:z option. As I said, I can assist getting it working.

[vi]

Is this pull request the cause of dependabot pull request spam?

[akostadinov]

Yes, see dependabot.yml. The cargo part can be removed if you don't want it. But for github actions, the frequency shouldn't be much. Otherwise one has to keep track of updates. And from time to time, the nodejs on github side is updated to a new version, some tasks stop working.

[vi]

Why the builder hangs in about half of attempts?

Example: https://github.com/vi/websocat/actions/runs/8906583058/job/24458977279

[akostadinov]

I imssed the question. But just noticed the builds fail. I think they hit some timeout at 6h :( I don't know if it is hanging or is it just super slow building.

Looking at the log last thing I see is

[1/2] STEP 9/9: RUN cargo build --release $CARGO_OPTS &&     strip target/release/websocat
   Compiling websocat v1.13.0 (/src/websocat)

Does it build properly for you locally?

If it builds locally, another thing that can be tried is (that I don't have permissions to), maybe try to run the workflow manually and specify a single platform only - once linux/amd64, once linux/arm64 which are the two built by default.

Another thing that comes to mind, in case build uses a lot of memory, that can work locally but overload the gh actions builder. Really not a clear idea what might be the case.

[vi]

Does it build properly for you locally?

Within Docker or in usual dev mode?

Obviously, when I run cargo build --release locally it works.

[akostadinov]

I mean the container build. Docker or podman.

[vi]

Running docker build . in Websocat's source root completed successfully and in reasonable time, taking about half a gig of memory.

 => [internal] load build context                                                                                                                                        0.1s
 => => transferring context: 179.13kB                                                                                                                                    0.0s
 => [stage-1 2/4] RUN apk add --no-cache libgcc                                                                                                                          9.8s
 => [cargo-build 2/7] RUN apk add --no-cache musl-dev pkgconfig openssl-dev                                                                                              7.0s
 => [cargo-build 3/7] WORKDIR /src/websocat                                                                                                                              0.7s
 => [cargo-build 4/7] COPY Cargo.toml Cargo.toml                                                                                                                         0.7s
 => [cargo-build 5/7] RUN mkdir src/ &&    echo "fn main() {println!("if you see this, the build broke")}" > src/main.rs &&     cargo build --release --features=worka  49.0s
 => [cargo-build 6/7] COPY src src                                                                                                                                       0.7s
 => [cargo-build 7/7] RUN cargo build --release --features=workaround1,seqpacket,prometheus_peer,prometheus/process,crypto_peer &&     strip target/release/websocat    67.6s
 => [stage-1 3/4] COPY --from=cargo-build /src/websocat/target/release/websocat /usr/local/bin/                                                                          0.8s
 => exporting to image                                                                                                                                                   0.1s
 => => exporting layers                                                                                                                                                  0.1s
 => => writing image sha256:1a301c1eeece061e764cdaa97de85ba82229c3c158de59a03cb3857aed0d5052                                                                             0.0s
 => => naming to docker.io/library/websocat                                                                                                                              0.0s

 1 warning found (use docker --debug to expand):
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 2)

I doubt it is multi-arch.

Or does that Github builder use some entirely different Dockerfile?

[akostadinov]

The build is multi-arch which means a separate image of each selected architecture is built from the same Dockerfile.

https://github.com/vi/websocat/blob/214de61843911f1e80286e37070700314ef859fa/.github/workflows/container-image-buildah.yml#L98

And the default archs can be seen here: https://github.com/vi/websocat/blob/214de61843911f1e80286e37070700314ef859fa/.github/workflows/container-image-buildah.yml#L55

When building image manually, there are parameters that can be set to override the default platforms and set a custom tag.

[vi]

I see mentions of "QEMU" in that file. Does it build everything within an additional virtual machine instead of directly?

---

Meanwhile removed LTO and other heavyweight compilation options from Cargo.toml. Let's see if it would help the CI.

[akostadinov]

I'm sorry for the delay. I couldn't get around to this. When only x86_64 is built, it works: https://github.com/akostadinov/websocat/actions/runs/11601291231/job/32303673256

When arm is built though, it seems to freeze or is extremely slow: https://github.com/akostadinov/websocat/actions/runs/11601415361/job/32304088695

I can't say why. You can replicate locally by building the arm target. Should be just having the necessary qemu-static package installed locally and adding --platform=linux/arm64 to the build command.

[vi]

Maybe it is trying to build code within qemu instead of cross-compiling from x86_64 and using qemu only for running tests?

[akostadinov]

It is indeed running the container build entirely under qemu. There is no mechanism to split these.

So unless compiling can be done less intensive, a short term solution is to only build x86_64. For the time being this can be applied #274

A nicer solution would be to cross compile outside the docker/buildah build. Then have a separate Dockerfile that will just install dependencies and only copy the pre-compiled binaries to the container. Which then will allow to have quick qemu cross-platform builds because the compiling step, which is the issue right now, will not be performed during the container build.

I see a guide about cross-compiling https://blog.logrocket.com/guide-cross-compilation-rust/ but have no idea about rust. So if you have steps that will cross compile on ubuntu (what github seems to be using), then I can look at how this can be integrated into the action.