com-lihaoyi / mill

Your shiny new Java/Scala build tool!
https://mill-build.com/
MIT License
2k stars 308 forks source link

Add multi platform support to contrib.docker through docker buildx #3143

Closed GeorgOfenbeck closed 2 months ago

GeorgOfenbeck commented 2 months ago

A very minor change to the contrib.docker plugin (and its docs).

Allows to build for other architectures - a use case that becomes more and more useful with ARM architectures becoming more relevant.

Relevant docs: docker buildx (Buildkit)

yankay commented 2 months ago

It's fascinating to see the multi-arch images supported by Mill.

There are two comments on the PR:

  1. From the https://docs.docker.com/reference/cli/docker/buildx/build/#platform , we can build multi-arch images with:

    docker buildx build --platform=linux/arm64 .
    docker buildx build --platform=linux/amd64,linux/arm64,linux/arm/v7 .
    docker buildx build --platform=darwin .

    The --platform is a string array, not a string.

  2. The image built by docker buildx should be pushed with docker buildx --push and can not use docker push .

GeorgOfenbeck commented 2 months ago

For 1. I ignored the multi-arch image option a bit as currently that is not enabled by default on docker: see: https://docs.docker.com/build/building/multi-platform/

"By default, Docker can build for only one platform at a time."

One can of course make it work as described in the article. For the option I felt that going for the single target platform is the better choice as by default otherwise one gets the error "ERROR: Multi-platform build is not supported for the docker driver. Switch to a different driver, or turn on the containerd image store, and try again. Learn more at https://docs.docker.com/go/build-multi-platform/" Which might misslead people on first glance that other platforms are not working

For those that know what they are doing, they could always pass it as a single string. Changing to an Array is of course an option, but one should definetly point out that issue then in the docu.

For 2. https://github.com/com-lihaoyi/mill/blob/1792ac4442dfcff4ff726ba54e08b2eb580eb86b/contrib/docker/src/mill/contrib/docker/DockerModule.scala#L190 doesnt use either build nor buildx - therefore i dont think it should be an issue. calling "_.docker.push" works as expected for me. Are you referring to that push?

yankay commented 2 months ago

For 1. I ignored the multi-arch image option a bit as currently that is not enabled by default on docker: see: https://docs.docker.com/build/building/multi-platform/

"By default, Docker can build for only one platform at a time."

One can of course make it work as described in the article. For the option I felt that going for the single target platform is the better choice as by default otherwise one gets the error "ERROR: Multi-platform build is not supported for the docker driver. Switch to a different driver, or turn on the containerd image store, and try again. Learn more at https://docs.docker.com/go/build-multi-platform/" Which might misslead people on first glance that other platforms are not working

For those that know what they are doing, they could always pass it as a single string. Changing to an Array is of course an option, but one should definetly point out that issue then in the docu.

For 2.

https://github.com/com-lihaoyi/mill/blob/1792ac4442dfcff4ff726ba54e08b2eb580eb86b/contrib/docker/src/mill/contrib/docker/DockerModule.scala#L190

doesnt use either build nor buildx - therefore i dont think it should be an issue. calling "_.docker.push" works as expected for me. Are you referring to that push?

HI @GeorgOfenbeck

  1. Follow the Blog: https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/

The buildx is usually used to build Multi-Platform Image.

# building an image for two platforms
docker buildx build --platform=linux/amd64,linux/arm64 .

The command can work correctly in most environments when the docker build is installed. :-) For example Ubuntu x86.

2. The command of build and push cannot split: https://github.com/docker/buildx/issues/1152

For multi-arch image

docker buildx build --platform=linux/amd64,linux/arm64 --push .

can work correctly

GeorgOfenbeck commented 2 months ago
  1. can definitely be done in separate phases - the push code was not introduced in this PR! Also see docu here: https://docs.docker.com/reference/cli/docker/image/push/

It can easily be verified locally by running docker push for any architecture image available locally.

For 1. as written - one can definitely go that route - but as the rest of the plugin also "just" covers the most common cases (e.g. not being able to add any files etc) i opted for the simpler approach that should always work