arm/v7 and arm/v8 variants for runner (not controller) images

[mumoshu]

Extracted from https://github.com/actions-runner-controller/actions-runner-controller/pull/1120

Although I don't think it's common to run K8s and ARC on 32bit ARM, it would be still useful to allow runner pods on those arch, like when you run K8s and ARC in a managed K8s cluster and nodes and runner pods on your raspis.

This would require us to migrate our use of dumb-init to an init system that runs of 32bit ARM, e.g. tini.

[denisa]

this would be very helpful to validate projects that have to build on Apple’s silicon

[edenreich]

Hey any updates on that ? I still can't get it running on the latest version..

[edenreich]

How complex would it be to migrate from dumb-init to an init system ? Did you try to use docker export to export the filesystem as a tar archive and rebuild the controller with buildx for arm7 ?

[mumoshu]

Updating ARC's runner Dockerfile to install and configure e.g. tini would be the way to go :)

[mumoshu]

@denisa It should work on M1 mac as AFAIK M1 is 64bit ARM

[edenreich]

Updating ARC's runner Dockerfile to install and configure e.g. tini would be the way to go :)

So tini is like a wrapper around the manager process ? is it like an emulator ?

Are you referring to this one right ?

Installation looks fairly simple hope we can modify the Dockerfile to use it.

Looking forward.

[mumoshu]

@edenreich Yep that's the project I'm referring to.

I don't know all the details of tini and its differences vs dumb-init either. But, at least, you'd need to read https://github.com/Yelp/dumb-init#session-behavior and https://github.com/krallin/tini#process-group-killing and match the tini's behavior to dumb-init's so that we won't break anything.

It's no way an emulator if not I'm misreading what you're saying. "A wrapper" sounds more correct based on the roles of tini and ARC's controller-manager. tini is a compact init system that manages sub-processes, and it's needed when you want to run multiple root processes within a container in a Kubernetes pod.

[edenreich]

It's no way an emulator if not I'm misreading what you're saying. "A wrapper" sounds more correct based on the roles of tini and ARC's controller-manager. tini is a compact init system that manages sub-processes, and it's needed when you want to run multiple root processes within a container in a Kubernetes pod.

Looks interesting, so no performance overhead at runtime. I just thought it's doing what Rosetta is doing for apple M1 chips for compatibility with different CPU architectures.

Thanks for the clarification.

Proceeding to implementation details.. So I assume we need to add the statically linked version of tini(which is less than 1mb) to the distroless final container image, add an entrypoint tini -- and change the entrypoint of ./manager to a CMD. Then this manager should be able to run on arm7l 32bit ? To avoid breaking changes, should we do this only if we build for arm7l ? guess we don't need it for the other CPU architecture, because they are already working ?

Hope I understood the process for implementing it correctly, or do you have something else in mind ?

I can test it on my raspberry pi's if needed..

[edenreich]

I tried to run it on ubuntu workspace(not on my raspberry pi 4), cross-compiling it with buildx using the following command:

docker buildx build \ 
  -t test \ 
  --platform linux/arm/v7 \ 
  --build-arg TARGETOS=linux \ 
  --build-arg TARGETARCH=arm \
  --build-arg TARGETVARIANT=v7 \
  .

And I get the following error:

[+] Building 6.1s (12/14)
 => [internal] load build definition from Dockerfile                                                                                                                                   0.0s
 => => transferring dockerfile: 32B                                                                                                                                                    0.0s
 => [internal] load .dockerignore                                                                                                                                                      0.1s
 => => transferring context: 34B                                                                                                                                                       0.0s
 => [internal] load metadata for gcr.io/distroless/static:nonroot                                                                                                                      0.5s
 => [internal] load metadata for docker.io/library/golang:1.18.2                                                                                                                       5.8s
 => [builder 1/5] FROM docker.io/library/golang:1.18.2@sha256:04fab5aaf4fc18c40379924674491d988af3d9e97487472e674d0b5fd837dfac                                                         0.0s
 => CACHED [internal] settings cache mount permissions                                                                                                                                 0.0s
 => CACHED [stage-1 1/4] FROM gcr.io/distroless/static:nonroot@sha256:66cd130e90992bebb68b8735a72f8ad154d0cd4a6f3a8b76f1e372467818d1b4                                                 0.0s
 => [internal] load build context                                                                                                                                                      0.0s
 => => transferring context: 8.81kB                                                                                                                                                    0.0s
 => CACHED [builder 2/5] WORKDIR /workspace                                                                                                                                            0.0s
 => CACHED [builder 3/5] COPY go.mod go.sum ./                                                                                                                                         0.0s
 => CACHED [builder 4/5] RUN go mod download                                                                                                                                           0.0s
 => ERROR [builder 5/5] RUN --mount=target=.   --mount=type=cache,mode=0777,target=${GOCACHE}   export GOOS=${TARGETOS} GOARCH=${TARGETARCH} GOARM=${TARGETVARIANT#v} &&   go build -  0.1s
------
 > [builder 5/5] RUN --mount=target=.   --mount=type=cache,mode=0777,target=${GOCACHE}   export GOOS=${TARGETOS} GOARCH=${TARGETARCH} GOARM=${TARGETVARIANT#v} &&   go build -o /out/manager main.go &&   go build -o /out/github-webhook-server ./cmd/githubwebhookserver:
#0 0.084 runc run failed: unable to start container process: error during container init: error mounting "/var/lib/docker/overlay2/v4pnliqyrmty543yq3hk0cttt/merged/cache" to rootfs at "/workspace/${GOCACHE}": mkdir /var/lib/docker/buildkit/executor/apso9t6cg93fsu33dwrzp4kqg/rootfs/workspace/${GOCACHE}: read-only file system
------
error: failed to solve: executor failed running [/bin/sh -c export GOOS=${TARGETOS} GOARCH=${TARGETARCH} GOARM=${TARGETVARIANT#v} &&   go build -o /out/manager main.go &&   go build -o /out/github-webhook-server ./cmd/githubwebhookserver]: exit code: 1

So it fails even before the final container image :S is that an expected error ?

[edenreich]

I guess I'll need to upgrade my raspberry pi OS to 64bit, seems like they support it now. Before I do that, can someone please confirm that it works on arm64 ? Highly appreciated...

[edenreich]

FYI: I've upgraded my raspberry pi's OS to arm64 and everything seems to work, thanks for the advice - I guess running it on arm 32 bit OS is not a big priority - so consider me silent...Thanks again!

[corneil]

arm64v8 will definitely be useful for running software that needs to build containers for M1 or run other tools aimed at 64-bit ARM.

[mumoshu]

@corneil Hey. We already publish runner images for arm64. What are actually missing for your use-case? https://github.com/actions-runner-controller/actions-runner-controller/pkgs/container/actions-runner-controller%2Factions-runner/43251224?tag=v2.297.0-ubuntu-20.04-5fd6ec4

[malik-n]

@mumoshu so there is currently no armv7 support from this project?