Request: ARM container builds

[gclawes]

This is a request to provide ARM container builds on quay.io. Quay now supports multi-arch container manifests (anouncement here).

NOTE: This is related to my personal use of Teleport Community Edition, not my employer's use of Teleport Enterprise.

[gclawes]

Builds do not currently work on arm64 boxes:

ubuntu@pi01:~/teleport$ make docker
make -C build.assets
make[1]: Entering directory '/home/ubuntu/teleport/build.assets'
docker build \
        --build-arg UID=$(id -u) \
        --build-arg GID=$(id -g) \
        --build-arg RUNTIME=go1.13.2 \
        --tag teleport-buildbox:go1.13.2 .
Sending build context to Docker daemon  344.6kB
Step 1/17 : FROM quay.io/gravitational/buildbox-base:1.0
1.0: Pulling from gravitational/buildbox-base
[DEPRECATION NOTICE] registry v2 schema1 support will be removed in an upcoming release. Please contact admins of the quay.io registry NOW to avoid future disruption.
c73ab1c6897b: Pull complete
b598305699cb: Pull complete
f026d35e258c: Pull complete
e09a000c8b5d: Pull complete
b9107d7bc9e2: Pull complete
fbfeba66f94c: Pull complete
Digest: sha256:2dba0755407aebcf21ce377905e6552c4b3485afe010e38d130d0cecebd186a3
Status: Downloaded newer image for quay.io/gravitational/buildbox-base:1.0
 ---> a9e9f55b37e1
Step 2/17 : ARG UID
 ---> Running in d4796e9f2b5e
Removing intermediate container d4796e9f2b5e
 ---> eef27a400bed
Step 3/17 : ARG GID
 ---> Running in 5c499dd375f6
Removing intermediate container 5c499dd375f6
 ---> 92e577a6be66
Step 4/17 : COPY pam/pam_teleport.so /lib/x86_64-linux-gnu/security
 ---> 5c6a3db3f8d2
Step 5/17 : COPY pam/teleport-acct-echo /etc/pam.d
 ---> a9f7e80827c3
Step 6/17 : COPY pam/teleport-acct-failure /etc/pam.d
 ---> ff0f475a52dd
Step 7/17 : COPY pam/teleport-success /etc/pam.d
 ---> 199813c464cc
Step 8/17 : COPY pam/teleport-session-failure /etc/pam.d
 ---> ed0da36f5747
Step 9/17 : COPY pam/teleport-session-environment /etc/pam.d
 ---> cadf9c7d4ab2
Step 10/17 : RUN apt-get update; apt-get install -q -y libpam-dev libc6-dev-i386 net-tools tree
 ---> Running in 08eb9319d368
standard_init_linux.go:211: exec user process caused "exec format error"
The command '/bin/sh -c apt-get update; apt-get install -q -y libpam-dev libc6-dev-i386 net-tools tree' returned a non-zero code: 1
make[1]: *** [Makefile:54: bbox] Error 1
make[1]: Leaving directory '/home/ubuntu/teleport/build.assets'
make: *** [Makefile:263: docker] Error 2

[jon-can]

Just received a request for us to have arm64 Teleport community edition container images available. A group is deploying Teleport community edition across ~12,000 IOx Cisco IR1101 devices, which requires arm64 container images.

[gclawes]

Getting a different build error now on arm64 with the latest master branch:

Linux pi01.pico 5.4.0-1022-raspi #25-Ubuntu SMP PREEMPT Thu Oct 15 13:31:49 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
ubuntu@pi01:~/teleport$ make image
---> Cleaning up OSS build artifacts.
rm -rf build
go clean -cache
rm -rf `go env GOPATH`/pkg/`go env GOHOSTOS`_`go env GOARCH`/github.com/gravitational/teleport*
rm -rf teleport
rm -rf *.gz
rm -rf *.zip
rm -f gitref.go
make -C build.assets build-binaries
make[1]: Entering directory '/home/ubuntu/teleport/build.assets'
docker build \
        --build-arg UID=$(id -u) \
        --build-arg GID=$(id -g) \
        --build-arg RUNTIME=go1.15.5 \
        --cache-from quay.io/gravitational/teleport-buildbox:go1.15.5 \
        --tag quay.io/gravitational/teleport-buildbox:go1.15.5 .
Sending build context to Docker daemon  334.8kB
Step 1/17 : FROM ubuntu:18.04
 ---> 84282c80cdda
Step 2/17 : COPY locale.gen /etc/locale.gen
 ---> 2fa5aaeb789a
Step 3/17 : COPY profile /etc/profile
 ---> 7fa5d3b1a8eb
Step 4/17 : ENV LANGUAGE="en_US.UTF-8"     LANG="en_US.UTF-8"     LC_ALL="en_US.UTF-8"     LC_CTYPE="en_US.UTF-8"     DEBIAN_FRONTEND="noninteractive"
 ---> Running in 9b8dcda2a219
Removing intermediate container 9b8dcda2a219
 ---> 73d31423b0c4
Step 5/17 : RUN apt-get update -y --fix-missing &&     apt-get -q -y upgrade &&     apt-get install -q -y apt-utils curl gcc gcc-multilib git gzip libbpfcc-dev libc6-dev libpam-dev libsqlite3-0 locales make net-tools tar tree zip shellcheck &&     dpkg-reconfigure locales &&     apt-get -y autoclean && apt-get -y clean
 ---> Running in 442656073b91
Get:1 http://ports.ubuntu.com/ubuntu-ports bionic InRelease [242 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports bionic-updates InRelease [88.7 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports bionic-backports InRelease [74.6 kB]
Get:4 http://ports.ubuntu.com/ubuntu-ports bionic-security InRelease [88.7 kB]
Get:5 http://ports.ubuntu.com/ubuntu-ports bionic/main arm64 Packages [1285 kB]
Get:6 http://ports.ubuntu.com/ubuntu-ports bionic/restricted arm64 Packages [572 B]
Get:7 http://ports.ubuntu.com/ubuntu-ports bionic/multiverse arm64 Packages [153 kB]
Get:8 http://ports.ubuntu.com/ubuntu-ports bionic/universe arm64 Packages [11.0 MB]
Get:9 http://ports.ubuntu.com/ubuntu-ports bionic-updates/restricted arm64 Packages [2262 B]
Get:10 http://ports.ubuntu.com/ubuntu-ports bionic-updates/universe arm64 Packages [1865 kB]
Get:11 http://ports.ubuntu.com/ubuntu-ports bionic-updates/multiverse arm64 Packages [5758 B]
Get:12 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main arm64 Packages [1431 kB]
Get:13 http://ports.ubuntu.com/ubuntu-ports bionic-backports/main arm64 Packages [11.2 kB]
Get:14 http://ports.ubuntu.com/ubuntu-ports bionic-backports/universe arm64 Packages [11.0 kB]
Get:15 http://ports.ubuntu.com/ubuntu-ports bionic-security/universe arm64 Packages [1182 kB]
Get:16 http://ports.ubuntu.com/ubuntu-ports bionic-security/restricted arm64 Packages [1149 B]
Get:17 http://ports.ubuntu.com/ubuntu-ports bionic-security/multiverse arm64 Packages [2430 B]
Get:18 http://ports.ubuntu.com/ubuntu-ports bionic-security/main arm64 Packages [1048 kB]
Fetched 18.5 MB in 5s (3617 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
Calculating upgrade...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Reading package lists...
Building dependency tree...
Reading state information...
Package gcc-multilib is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'gcc-multilib' has no installation candidate
The command '/bin/sh -c apt-get update -y --fix-missing &&     apt-get -q -y upgrade &&     apt-get install -q -y apt-utils curl gcc gcc-multilib git gzip libbpfcc-dev libc6-dev libpam-dev libsqlite3-0 locales make net-tools tar tree zip shellcheck &&     dpkg-reconfigure locales &&     apt-get -y autoclean && apt-get -y clean' returned a non-zero code: 100
make[1]: *** [Makefile:83: buildbox] Error 100
make[1]: Leaving directory '/home/ubuntu/teleport/build.assets'
make: *** [Makefile:371: docker-binaries] Error 2

[gclawes]

These are the gcc-multilib-* packages available in the arm64 ubuntu:18.04 image:

root@674e87a3744e:/# apt search gcc-multilib
Sorting... Done
Full Text Search... Done
gcc-multilib-arm-linux-gnueabi/bionic-updates,bionic-security 4:7.4.0-1ubuntu2.3 arm64
  GNU C compiler for the armel architecture

gcc-multilib-arm-linux-gnueabihf/bionic-updates,bionic-security 4:7.4.0-1ubuntu2.3 arm64
  GNU C compiler for the armhf architecture

gcc-multilib-i686-linux-gnu/bionic-updates,bionic-security 4:7.4.0-1ubuntu2.3 arm64
  GNU C compiler for the i386 architecture

gcc-multilib-s390x-linux-gnu/bionic-updates,bionic-security 4:7.4.0-1ubuntu2.3 arm64
  GNU C compiler for the s390x architecture

gcc-multilib-x86-64-linux-gnu/bionic-updates,bionic-security 4:7.4.0-1ubuntu2.3 arm64
  GNU C compiler for the amd64 architecture

gcc-multilib-x86-64-linux-gnux32/bionic-updates,bionic-security 4:7.4.0-1ubuntu1.3 arm64
  GNU C compiler for the x32 architecture

[gclawes]

These changes to build.assets/Dockerfile produced a working image on an arm64 device (RPi 4):

diff --git a/build.assets/Dockerfile b/build.assets/Dockerfile
index 7e4e87184..bed5d2b1e 100644
--- a/build.assets/Dockerfile
+++ b/build.assets/Dockerfile
@@ -19,7 +19,7 @@ ENV LANGUAGE="en_US.UTF-8" \

 RUN apt-get update -y --fix-missing && \
     apt-get -q -y upgrade && \
-    apt-get install -q -y apt-utils curl gcc gcc-multilib git gzip libbpfcc-dev libc6-dev libpam-dev libsqlite3-0 locales make net-tools tar tree zip shellcheck && \
+    apt-get install -q -y apt-utils curl gcc gcc-multilib-arm-linux-gnueabi git gzip libbpfcc-dev libc6-dev libpam-dev libsqlite3-0 locales make net-tools tar tree zip shellcheck && \
     dpkg-reconfigure locales && \
     apt-get -y autoclean && apt-get -y clean

@@ -29,12 +29,12 @@ RUN (groupadd ci --gid=$GID -o && useradd ci --uid=$UID --gid=$GID --create-home
      mkdir -p -m0700 /var/lib/teleport && chown -R ci /var/lib/teleport)

 # Install etcd.
-RUN (curl -L https://github.com/coreos/etcd/releases/download/v3.3.9/etcd-v3.3.9-linux-amd64.tar.gz | tar -xz ;\
-     cp etcd-v3.3.9-linux-amd64/etcd* /bin/)
+RUN (curl -L https://github.com/coreos/etcd/releases/download/v3.3.9/etcd-v3.3.9-linux-arm64.tar.gz | tar -xz ;\
+     cp etcd-v3.3.9-linux-arm64/etcd* /bin/)

 # Install Go.
 ARG RUNTIME
-RUN mkdir -p /opt && cd /opt && curl https://storage.googleapis.com/golang/$RUNTIME.linux-amd64.tar.gz | tar xz;\
+RUN mkdir -p /opt && cd /opt && curl https://storage.googleapis.com/golang/$RUNTIME.linux-arm64.tar.gz | tar xz;\
     mkdir -p /go/src/github.com/gravitational/teleport;\
     chmod a+w /go;\
     chmod a+w /var/lib;\

I believe gcc-multilib-arm-linux-gnueabi is the armv8/arm64 package, some quick googling indicates gcc-multilib-arm-linux-gnueabihf is for armv7 32-bit.

[gclawes]

Any update on this now that #5610 is merged?

[webvictim]

@gclawes #5610 is still draft unfortunately, we've had a lot of other things come up which have put it on the back burner. It's still something we'd like to do, but I'm afraid I don't know when we'll be able to get to it.

[Clarky3]

@gclawes #5610 is still draft unfortunately, we've had a lot of other things come up which have put it on the back burner. It's still something we'd like to do, but I'm afraid I don't know when we'll be able to get to it.

Does gravitational have any affiliation with: https://hub.docker.com/r/draconrose/teleport/tags?page=1&ordering=last_updated

There appears to be ARM64 docker images available for teleport on the above hub for 6.1.2 and other versions. I found those on my quest to find ARM support for teleport, then i came across this thread to find it's not officially support yet

Edit: Let me know if you want the above link removed.

[webvictim]

@clarkycal We have no affiliation with that user or repository. As they don’t appear to have a Github link or any kind of public code which describes how their builds are created, I would be hesistant about trusting it.

[Beanow]

I'm also interested in v7 and v8 ARM docker builds.

So to help things along, I did a local test of this using docker buildx and the binaries from https://goteleport.com/teleport/download.

First, having a build context that looks like:

.
├── Dockerfile
└── linux
    ├── amd64
    │   ├── tctl
    │   ├── teleport
    │   └── tsh
    ├── arm
    │   └── v7
    │       ├── tctl
    │       ├── teleport
    │       └── tsh
    └── arm64
        ├── tctl
        ├── teleport
        └── tsh

With the Dockerfile updated:

FROM ubuntu:20.04
+ARG TARGETPLATFORM

RUN apt-get update && \
    DEBIAN_FRONTEND=noninteractive apt-get upgrade -y && \
    DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y ca-certificates dumb-init && \
    update-ca-certificates && \
    apt-get -y clean && \
    rm -rf /var/lib/apt/lists/*

-COPY teleport /usr/local/bin/teleport
+COPY ./$TARGETPLATFORM/teleport /usr/local/bin/teleport
-COPY tctl /usr/local/bin/tctl
+COPY ./$TARGETPLATFORM/tctl /usr/local/bin/tctl
-COPY tsh /usr/local/bin/tsh
+COPY ./$TARGETPLATFORM/tsh /usr/local/bin/tsh

ENTRYPOINT ["/usr/bin/dumb-init", "teleport", "start", "-c", "/etc/teleport/teleport.yaml"]

Tips for preparing a local build.

# If you're having issues with the qemu based builds not installing deb-packages properly through apt-get.
# Warning: uses `--privileged` to modify the host!
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

# Before you can use buildx for multi-arch.
docker buildx create --use

I haven't spotted something for drone, but these Github Actions could be used as reference for the setup steps? https://github.com/docker/setup-qemu-action https://github.com/docker/setup-buildx-action

Then to actually build (local command I used):

docker buildx build --platform linux/arm/v7,linux/arm64/v8,linux/amd64 <path_to_context>

For trying it out on actual different architecture hosts, I pushed it to a private registry:

docker buildx build --platform linux/arm/v7,linux/arm64/v8,linux/amd64 \
  --tag registry.example.com/beanow/teleport:buildx --push \
  <path_to_context>

And was able to run a basic test on an arm64 (pi4):

uname -p
# aarch64
docker run --rm --entrypoint="" registry.example.com/beanow/teleport:buildx teleport version
# Teleport v6.1.2 git:v6.1.2-0-g23ab88c00 go1.15.5

Edit: as a bit more realistic test, I was able to start a --roles=node,proxy,auth container on arm64 and set it up as a leaf cluster, with a quay.io/gravitational/teleport:6.1.2 amd64 based root cluster. And log into the node using the root's web UI shell, calling some tctl get commands to peruse the leaf's status.

[ollytheninja]

Just bumping this - wondering if there are any plans to add arm64 containers to quay.io?

[ronaldmiranda]

@ollytheninja i just have created mine version as well https://hub.docker.com/r/ronaldmiranda/teleport-arm it's also compatible with teleport-cluster helm chart

[gclawes]

Any updates on this?

[uGiFarukh]

No arm64 support yet? Any update on this?

[gclawes]

FYI, this is my current patchset to build arm64 containers locally, it's change a bit since my original. I'm not sure if all of these modifications are 100% necessary.

diff --git a/Makefile b/Makefile
index 6fa159ece..6dfd34b06 100644
--- a/Makefile
+++ b/Makefile
@@ -971,7 +971,7 @@ install: build
 .PHONY: image
 image: clean docker-binaries
    cp ./build.assets/charts/Dockerfile $(BUILDDIR)/
-   cd $(BUILDDIR) && docker build --no-cache . -t $(DOCKER_IMAGE):$(VERSION)
+   cd $(BUILDDIR) && docker buildx build --load --platform linux/arm64 --no-cache . -t $(DOCKER_IMAGE):$(VERSION)
    if [ -f e/Makefile ]; then $(MAKE) -C e image; fi

 .PHONY: publish
@@ -987,7 +987,7 @@ publish: image
 .PHONY: image-ci
 image-ci: clean docker-binaries
    cp ./build.assets/charts/Dockerfile $(BUILDDIR)/
-   cd $(BUILDDIR) && docker build --no-cache . -t $(DOCKER_IMAGE_CI):$(VERSION)
+   cd $(BUILDDIR) && docker buildx build --load --platform linux/arm64 --no-cache . -t $(DOCKER_IMAGE_CI):$(VERSION)
    if [ -f e/Makefile ]; then $(MAKE) -C e image-ci; fi

 .PHONY: publish-ci
diff --git a/build.assets/Makefile b/build.assets/Makefile
index 270a3081c..413bc6be9 100644
--- a/build.assets/Makefile
+++ b/build.assets/Makefile
@@ -82,7 +82,7 @@ export
 #
 .PHONY:build
 build: buildbox
-   docker run $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX) \
+   docker run --platform linux/arm64 $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX) \
        make -C $(SRCDIR) ADDFLAGS='$(ADDFLAGS)' release

 #
@@ -90,7 +90,7 @@ build: buildbox
 #
 .PHONY:build-binaries
 build-binaries: buildbox
-   docker run $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX) \
+   docker run --platform linux/arm64 $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX) \
        make -C $(SRCDIR) ADDFLAGS='$(ADDFLAGS)' full

 #
@@ -98,7 +98,7 @@ build-binaries: buildbox
 #
 .PHONY:build-enterprise-binaries
 build-enterprise-binaries: buildbox
-   docker run $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX) \
+   docker run --platform linux/arm64 $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX) \
        make -C $(SRCDIR)/e ADDFLAGS='$(ADDFLAGS)' VERSION=$(VERSION) GITTAG=v$(VERSION) clean full

 #
@@ -107,7 +107,7 @@ build-enterprise-binaries: buildbox
 #
 .PHONY:build-binaries-fips
 build-binaries-fips: buildbox-fips
-   docker run $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX_FIPS) \
+   docker run --platform linux/arm64 $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX_FIPS) \
        make -C $(SRCDIR)/e ADDFLAGS='$(ADDFLAGS)' VERSION=$(VERSION) GITTAG=v$(VERSION) FIPS=yes clean full

 #
@@ -119,9 +119,8 @@ build-binaries-fips: buildbox-fips
 #
 .PHONY:buildbox
 buildbox:
-   if [[ "$(BUILDBOX_NAME)" == "$(BUILDBOX)" ]]; then \
-       if [[ $${DRONE} == "true" ]] && ! docker inspect --type=image $(BUILDBOX) 2>&1 >/dev/null; then docker pull $(BUILDBOX) || true; fi; \
-       docker build --platform=linux/$(RUNTIME_ARCH) \
+       docker buildx build --platform=linux/$(RUNTIME_ARCH) \
+           --load \
            --build-arg UID=$(UID) \
            --build-arg GID=$(GID) \
            --build-arg BUILDARCH=$(RUNTIME_ARCH) \
@@ -132,7 +131,6 @@ buildbox:
            --build-arg LIBBPF_VERSION=$(LIBBPF_VERSION) \
            --cache-from $(BUILDBOX) \
            --tag $(BUILDBOX) . ; \
-   fi

 # Builds a Docker buildbox for FIPS
 #
@@ -140,7 +138,9 @@ buildbox:
 buildbox-fips:
    if [[ "$(BUILDBOX_FIPS_NAME)" == "$(BUILDBOX_FIPS)" ]]; then \
        if [[ $${DRONE} == "true" ]] && ! docker inspect --type=image $(BUILDBOX_FIPS) 2>&1 >/dev/null; then docker pull $(BUILDBOX_FIPS) || true; fi; \
-       docker build \
+       docker builx build \
+           --load \
+           --platform linux/$(RUNTIME_ARCH) \
            --build-arg UID=$(UID) \
            --build-arg GID=$(GID) \
            --build-arg BORINGCRYPTO_RUNTIME=$(BORINGCRYPTO_RUNTIME) \
@@ -155,7 +155,9 @@ buildbox-fips:
 .PHONY:buildbox-centos7
 buildbox-centos7:
    @if [[ $${DRONE} == "true" ]] && ! docker inspect --type=image $(BUILDBOX_CENTOS7) 2>&1 >/dev/null; then docker pull $(BUILDBOX_CENTOS7) || true; fi;
-   docker build \
+   docker builx build \
+       --load \
+       --platform linux/$(RUNTIME_ARCH) \
        --build-arg UID=$(UID) \
        --build-arg GID=$(GID) \
        --build-arg BUILDARCH=$(RUNTIME_ARCH) \
@@ -172,7 +174,9 @@ buildbox-centos7:
 .PHONY:buildbox-centos7-fips
 buildbox-centos7-fips:
    @if [[ $${DRONE} == "true" ]] && ! docker inspect --type=image $(BUILDBOX_CENTOS7_FIPS) 2>&1 >/dev/null; then docker pull $(BUILDBOX_CENTOS7_FIPS) || true; fi;
-   docker build \
+   docker builx build \
+       --load \
+       --platform linux/$(RUNTIME_ARCH) \
        --build-arg UID=$(UID) \
        --build-arg GID=$(GID) \
        --build-arg BORINGCRYPTO_RUNTIME=$(BORINGCRYPTO_RUNTIME) \
@@ -189,7 +193,9 @@ buildbox-centos7-fips:
 .PHONY:buildbox-arm
 buildbox-arm: buildbox
    @if [[ $${DRONE} == "true" ]] && ! docker inspect --type=image $(BUILDBOX_ARM) 2>&1 >/dev/null; then docker pull $(BUILDBOX_ARM) || true; fi;
-   docker build \
+   docker buildx build \
+       --load \
+       --platform linux/$(RUNTIME_ARCH) \
        --build-arg BUILDBOX_VERSION=$(BUILDBOX_VERSION) \
        --cache-from $(BUILDBOX) \
        --cache-from $(BUILDBOX_ARM) \
@@ -203,7 +209,9 @@ buildbox-arm: buildbox
 .PHONY:buildbox-arm-fips
 buildbox-arm-fips: buildbox-fips
    @if [[ $${DRONE} == "true" ]] && ! docker inspect --type=image $(BUILDBOX_ARM_FIPS) 2>&1 >/dev/null; then docker pull $(BUILDBOX_ARM_FIPS) || true; fi;
-   docker build \
+   docker buildx build \
+       --load \
+       --platform linux/$(RUNTIME_ARCH) \
        --build-arg BUILDBOX_VERSION=$(BUILDBOX_VERSION) \
        --cache-from $(BUILDBOX_FIPS) \
        --cache-from $(BUILDBOX_ARM_FIPS) \
@@ -229,6 +237,7 @@ buildbox-teleterm: buildbox
 .PHONY: grpc
 grpc: buildbox
    docker run \
+       --platform linux/arm64 \
        $(DOCKERFLAGS) -e CLANG_FORMAT=/usr/bin/clang-format-10 -t $(BUILDBOX) \
        make -C /go/src/github.com/gravitational/teleport buildbox-grpc

@@ -254,6 +263,7 @@ clean:
 .PHONY:test
 test: buildbox
    docker run \
+       --platform linux/arm64 \
        --env TELEPORT_ETCD_TEST="yes" \
        --env TELEPORT_XAUTH_TEST="yes" \
        $(DOCKERFLAGS) $(NOROOT) -t $(BUILDBOX) \
@@ -267,6 +277,7 @@ test: buildbox
 .PHONY:test-root
 test-root: buildbox
    docker run \
+       --platform linux/arm64 \
        --env TELEPORT_ETCD_TEST="yes" \
        --env TELEPORT_XAUTH_TEST="yes" \
        $(DOCKERFLAGS) -t $(BUILDBOX) \
@@ -279,7 +290,7 @@ test-root: buildbox

 .PHONY:test-sh
 test-sh: buildbox
-   docker run $(DOCKERFLAGS) $(NOROOT) -t $(BUILDBOX) \
+   docker run --platform linux/arm64 $(DOCKERFLAGS) $(NOROOT) -t $(BUILDBOX) \
        /bin/bash -c "make -C $(SRCDIR) BATSFLAGS=$(BATSFLAGS) test-sh"

 .PHONY:test-helm
@@ -295,6 +306,7 @@ test-helm-update-snapshots:
 .PHONY:integration
 integration: buildbox
    docker run \
+       --platform linux/arm64 \
        --env TELEPORT_ETCD_TEST="yes" \
        $(DOCKERFLAGS) $(NOROOT) -t $(BUILDBOX) \
        /bin/bash -c \
@@ -304,6 +316,7 @@ integration: buildbox
 .PHONY:integration-root
 integration-root: buildbox
    docker run $(DOCKERFLAGS) -t $(BUILDBOX) \
+       --platform linux/arm64 \
        /bin/bash -c "make -C $(SRCDIR) FLAGS='-cover' integration-root"

 #
@@ -312,6 +325,7 @@ integration-root: buildbox
 .PHONY:lint
 lint: buildbox
    docker run $(DOCKERFLAGS) $(NOROOT) -t $(BUILDBOX) \
+       --platform linux/arm64 \
        /bin/bash -c "make -C $(SRCDIR) lint"

 .PHONY:lint-helm
@@ -325,6 +339,7 @@ lint-helm: buildbox
 .PHONY:enter
 enter: buildbox
    docker run $(DOCKERFLAGS) -ti $(NOROOT) \
+       --platform linux/arm64 \
        -e HOME=$(SRCDIR)/build.assets -w $(SRCDIR) $(BUILDBOX) /bin/bash

 #
@@ -349,6 +364,7 @@ enter/centos7: buildbox
 .PHONY:release
 release: buildbox
    docker run $(DOCKERFLAGS) $(NOROOT) $(BUILDBOX_NAME) \
+       --platform linux/arm64 \
        /usr/bin/make release -e ADDFLAGS="$(ADDFLAGS)" OS=$(OS) ARCH=$(ARCH) RUNTIME=$(GOLANG_VERSION) FIDO2=$(FIDO2) REPRODUCIBLE=yes

 # These are aliases used to make build commands uniform.
@@ -388,7 +404,7 @@ release-amd64-centos7-fips: buildbox-centos7-fips
 .PHONY:release-fips
 release-fips: buildbox-fips
    @if [ -z ${VERSION} ]; then echo "VERSION is not set"; exit 1; fi
-   docker run $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX_FIPS_NAME) \
+   docker run --platform linux/arm64 $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX_FIPS_NAME) \
        /usr/bin/make -C e release -e ADDFLAGS="$(ADDFLAGS)" OS=$(OS) ARCH=$(ARCH) RUNTIME=$(GOLANG_VERSION) FIPS=yes VERSION=$(VERSION) GITTAG=v$(VERSION) REPRODUCIBLE=yes

 #
@@ -396,7 +412,7 @@ release-fips: buildbox-fips
 #
 .PHONY:release-centos7
 release-centos7: buildbox-centos7
-   docker run $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX_CENTOS7) \
+   docker run --platform linux/arm64 $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX_CENTOS7) \
        /usr/bin/scl enable devtoolset-11 'make release -e ADDFLAGS="$(ADDFLAGS)" OS=$(OS) ARCH=$(ARCH) RUNTIME=$(GOLANG_VERSION) FIDO2=$(FIDO2) REPRODUCIBLE=no'

 #
@@ -405,7 +421,7 @@ release-centos7: buildbox-centos7
 #
 .PHONY:release-centos7-fips
 release-centos7-fips:
-   docker run $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX_CENTOS7_FIPS) \
+   docker run --platform linux/arm64 $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX_CENTOS7_FIPS) \
        /usr/bin/scl enable devtoolset-11 '/usr/bin/make -C e release -e ADDFLAGS="$(ADDFLAGS)" OS=$(OS) ARCH=$(ARCH) RUNTIME=$(GOLANG_VERSION) FIPS=yes VERSION=$(VERSION) GITTAG=v$(VERSION) REPRODUCIBLE=no'

 #
@@ -413,7 +429,7 @@ release-centos7-fips:
 #
 .PHONY:release-windows
 release-windows: buildbox
-   docker run $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX) \
+   docker run --platform linux/arm64 $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX) \
        /usr/bin/make release -e ADDFLAGS="$(ADDFLAGS)" OS=windows RUNTIME=$(GOLANG_VERSION) REPRODUCIBLE=yes

 #
@@ -421,7 +437,7 @@ release-windows: buildbox
 #
 .PHONY:release-windows-unsigned
 release-windows-unsigned: buildbox
-   docker run $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX) \
+   docker run --platform linux/arm64 $(DOCKERFLAGS) -i $(NOROOT) $(BUILDBOX) \
        /usr/bin/make release-windows-unsigned -e ADDFLAGS="$(ADDFLAGS)" OS=windows RUNTIME=$(GOLANG_VERSION) REPRODUCIBLE=yes

 #

[AlverezYari]

Bump. I'm hitting this trying to deploy Teleport via the official helm charts on AWS with T4G workers. I'm going to have to build these myself which is fine but it would be really nice if these could be provided. Thanks!

[webvictim]

I believe the intention is for us to publish arm/arm64 container images by the end of the year.

[russjones]

https://github.com/gravitational/teleport/pull/16688