Closed SgtCoDFish closed 7 months ago
Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all
Hi! When running make image
, I get an error but I assume that this problem isn't new to the changes you introduced.
$ make image
docker buildx build --builder trust-manager-builder --platform=linux/amd64,linux/arm64,linux/arm/v7,linux/ppc64le -t quay.io/jetstack/trust-manager:v0.7.0 --build-arg GOPROXY=https://proxy.golang.org,direct --output type=local,dest=/Users/mvalais/code/cert-manager/trust-manager/bin/trust-manager -f ./Dockerfile .
ERROR: no builder "trust-manager-builder" found
make: *** [trust-manager-save] Error 1
I realized that the Makefile talks about that issue:
# See wait-for-buildx.sh for an explanation of why it's needed
.PHONY: provision-buildx
provision-buildx: ## set up docker buildx for multiarch building
ifeq ($(OS), linux)
# This step doesn't work on macOS and doesn't seem to be required (at least with docker desktop)
# It did seem to be needed in Linux, at least in certain configurations when running on amd64
# TODO: it might be preferable to move away from docker buildx in the long term to avoid the dependency on Docker
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
endif
docker buildx rm $(BUILDX_BUILDER) &>/dev/null || :
./hack/wait-for-buildx.sh $(BUILDX_BUILDER) gone
docker buildx create --name $(BUILDX_BUILDER) --bootstrap --driver docker-container --platform $(IMAGE_PLATFORMS)
./hack/wait-for-buildx.sh $(BUILDX_BUILDER) exists
docker buildx inspect --bootstrap --builder $(BUILDX_BUILDER)
Do I need to run make provision-buildx
before running make image
? It isn't mentioned anywhere 😅
Would be worth adding a thumbs-up to the following issue, if that's the reason we can't use ko
:
Alternatively we could use ko
and remove the command
field from the Deployment and rely on the default entrypoint of the image:
Do I need to run make provision-buildx before running make image? It isn't mentioned anywhere 😅
Yep you do! It'd be a good PR to raise 😁
Would be worth adding a thumbs-up to the following issue, if that's the reason we can't use ko ...
I think there are a few issues with ko
.
Like, I'm not religiously opposed to using ko
- but it seems like the tool currently basically forces us to make a breaking change for our users, and that makes me worry a bit about adopting it more widely because if ko
forces us to break to adopt it, it seems likely it'll forcibly break us later after we've adopted it.
I think Tim's experience has been that ko
also makes life hard when it comes to other dependencies from the base image. Like ko
works if you've got a greenfield go binary, but once you've got an established project or you've got more complicated things going on, it seems to be really tricky.
I don't love buildx
but at least using dockerfiles gives us the control over what happens, you know? And like I said this morning - with these changes it at least works!
@SgtCoDFish - I tried to run this locally although perhaps I ran the wrong command?
(⎈|kind-argocd-demo:cert-manager)➜ cert-manager-trust-manager git:(more-local-buildx) make image
docker buildx build --builder trust-manager-builder --platform=linux/amd64,linux/arm64,linux/arm/v7,linux/ppc64le -t quay.io/jetstack/trust-manager:v0.7.0 --build-arg GOPROXY=https://proxy.golang.org,direct --output type=local,dest=/home/pfiddes/projects/jetstack-open-source/cert-manager-trust-manager/bin/trust-manager -f ./Dockerfile .
[+] Building 382.9s (27/27) FINISHED docker-container:trust-manager-builder
=> [internal] booting buildkit 36.6s
=> => pulling image moby/buildkit:buildx-stable-1 36.0s
=> => creating container buildx_buildkit_trust-manager-builder0 0.5s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.35kB 0.0s
=> [linux/amd64 internal] load metadata for docker.io/library/golang:1.21 1.7s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 402.54kB 0.0s
=> [linux/amd64 builder 1/11] FROM docker.io/library/golang:1.21@sha256:672a2286da3ee7a854c3e0a56e0838918d0dbb1c18652992 136.8s
=> => resolve docker.io/library/golang:1.21@sha256:672a2286da3ee7a854c3e0a56e0838918d0dbb1c18652992930293312de898a6 0.0s
=> => sha256:49fe8346018f3c4b4a77420527e6ea9e36e332519ade61067af355526d85b4db 156B / 156B 0.1s
=> => sha256:76a498520fee85f05c762d5075ba3814f09ee1c123f14a54dc68c2f14652ec12 66.98MB / 66.98MB 87.0s
=> => sha256:bed956b3d08c0b1903d9af3e6850930eacf865b23f62bd8cf46eb1de1325c39b 92.36MB / 92.36MB 127.8s
=> => sha256:917ee5330e73737d6095a802333d311648959399ff2c067150890162e720f863 64.13MB / 64.13MB 116.3s
=> => sha256:b5de22c0f5cd2ea2bb6c0524478db95bff5a294c99419ccd4a9d3ccc9873bad9 24.05MB / 24.05MB 49.0s
=> => sha256:bc0734b949dcdcabe5bfdf0c8b9f44491e0fce04cb10c9c6e76282b9f6abdf01 49.56MB / 49.56MB 78.0s
=> => extracting sha256:bc0734b949dcdcabe5bfdf0c8b9f44491e0fce04cb10c9c6e76282b9f6abdf01 1.6s
=> => extracting sha256:b5de22c0f5cd2ea2bb6c0524478db95bff5a294c99419ccd4a9d3ccc9873bad9 0.5s
=> => extracting sha256:917ee5330e73737d6095a802333d311648959399ff2c067150890162e720f863 2.1s
=> => extracting sha256:bed956b3d08c0b1903d9af3e6850930eacf865b23f62bd8cf46eb1de1325c39b 2.1s
=> => extracting sha256:76a498520fee85f05c762d5075ba3814f09ee1c123f14a54dc68c2f14652ec12 3.2s
=> => extracting sha256:49fe8346018f3c4b4a77420527e6ea9e36e332519ade61067af355526d85b4db 0.0s
=> [linux/amd64 builder 2/11] WORKDIR /workspace 0.4s
=> [linux/amd64 builder 3/11] COPY go.mod go.mod 0.0s
=> [linux/amd64 builder 4/11] COPY go.sum go.sum 0.0s
=> [linux/amd64 builder 5/11] COPY Makefile Makefile 0.0s
=> [linux/amd64 builder 6/11] COPY make/ make/ 0.0s
=> [linux/amd64 builder 7/11] COPY cmd/ cmd/ 0.0s
=> [linux/amd64 builder 8/11] COPY pkg/ pkg/ 0.0s
=> [linux/amd64 builder 9/11] COPY hack/ hack/ 0.0s
=> [linux/amd64->arm/v7 builder 10/11] RUN GOPROXY=https://proxy.golang.org,direct go mod download 83.4s
=> [linux/amd64->ppc64le builder 10/11] RUN GOPROXY=https://proxy.golang.org,direct go mod download 82.0s
=> [linux/amd64 builder 10/11] RUN GOPROXY=https://proxy.golang.org,direct go mod download 78.9s
=> [linux/amd64->arm64 builder 10/11] RUN GOPROXY=https://proxy.golang.org,direct go mod download 84.4s
=> [linux/amd64 builder 11/11] RUN make build-linux-amd64 116.1s
=> [linux/amd64->ppc64le builder 11/11] RUN make build-linux-ppc64le 124.0s
=> [linux/amd64->arm/v7 builder 11/11] RUN make build-linux-arm 122.4s
=> [linux/amd64->arm64 builder 11/11] RUN make build-linux-arm64 121.2s
=> [linux/amd64 stage-1 1/2] COPY --from=builder /workspace/bin/trust-manager-linux-amd64 /usr/bin/trust-manager 0.1s
=> [linux/arm64 stage-1 1/2] COPY --from=builder /workspace/bin/trust-manager-linux-arm64 /usr/bin/trust-manager 0.1s
=> [linux/arm/v7 stage-1 1/2] COPY --from=builder /workspace/bin/trust-manager-linux-arm /usr/bin/trust-manager 0.1s
=> [linux/ppc64le stage-1 1/2] COPY --from=builder /workspace/bin/trust-manager-linux-ppc64le /usr/bin/trust-manager 0.1s
=> exporting to client directory 0.8s
=> => copying files linux/ppc64le 56.41MB 0.8s
=> => copying files linux/arm64 55.11MB 0.8s
=> => copying files linux/amd64 56.92MB 0.8s
=> => copying files linux/arm/v7 54.07MB 0.8s
make/trust-package-debian.mk:33: *** DEBIAN_TRUST_PACKAGE_VERSION must be set for trust-package-debian-save. Stop.
I couldn't see what to set DEBIAN_TRUST_PACKAGE_VERSION
to, perhaps you can give me a default to test the full make image
.
I ran make local-images
and that ran through.
Working on Ubuntu Linux 22.04.3 LTS.
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: inteon
The full list of commands accepted by this bot can be found here.
The pull request process is described here
@hawksight
I couldn't see what to set DEBIAN_TRUST_PACKAGE_VERSION to, perhaps you can give me a default to test the full make image. I ran make local-images and that ran through.
I totally understand the confusion here.
I think it would make sense to have make image
not build trust package stuff by default to be honest. I think it mostly exists as a helper for developing trust-manager itself. I raised #269 for this!
See https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/
This PR adds that blogpost for this repo. It also allows me to build trust-manager on macOS, when I currently cannot because of bugs in emulation when building for different platforms.
This will also improve performance as well as fixing the build on macOS.
We might want to use a different process down the road, but I'm confident this is an improvement to the status quo.