Closed tkrah closed 1 year ago
buildx should be part of your default install:
docker info:
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.10.2
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.16.0
Path: /usr/libexec/docker/cli-plugins/docker-compose
scan: Docker Scan (Docker Inc.)
Version: v0.23.0
Path: /usr/libexec/docker/cli-plugins/docker-scan
This should run on intel macs, it does run on an amd64 platform. This should work on all arm64 platforms including M1, M2 and M3. We can ask the people in the #1300 to test that PR before merging it if it does work for them. I don't have arm64 myself to test the final arm64 image myself too (although cross-building it for arm64 does succeed) - but I'll try to get someone to test that before merging.
@garris buildx
is now the default build method, when you run docker build
, it's actually running docker buildx build
.
$ docker buildx
Usage: docker buildx [OPTIONS] COMMAND
(...)
Commands:
bake Build from a file
build Start a build
(...)
$ docker build
ERROR: "docker buildx build" requires exactly 1 argument.
See 'docker buildx build --help'.
Usage: docker buildx build [OPTIONS] PATH | URL | -
Start a build
M1 Mac here, the docker image built. Internally we use a custom docker image based off the backstop one to run the tests and it'll take me a while to try these changes in there. What's the best way to confirm the docker works, trying things like smoke-test-docker
install the image from backstopjs/backstopjs
M1 Mac here, the docker image built. Internally we use a custom docker image based off the backstop one to run the tests and it'll take me a while to try these changes in there. What's the best way to confirm the docker works, trying things like
smoke-test-docker
install the image frombackstopjs/backstopjs
Build the image locally and use that one as your new base. You can use a custom tag to be sure you use your locally build one. Besides that, nothing changed imho from the backstopjs perspective. You can't fetch that image yet from the official docker registry because it is not yet merged nor published there.
Sorry I'm really not very proficient with docker, none of that makes any sense to me. Our docker image for our project is based off ruby:3.1
as we need to use the same docker image to run other things for our CI tests, inside there we have pieces from the old backstopjs dockerfile with some fixes for M1.
Besides that, nothing changed imho from the backstopjs perspective.
So if the image built OK then you don't need me to test anything else?
Sorry I'm really not very proficient with docker, none of that makes any sense to me. Our docker image for our project is based off
ruby:3.1
as we need to use the same docker image to run other things for our CI tests, inside there we have pieces from the old backstopjs dockerfile with some fixes for M1.Besides that, nothing changed imho from the backstopjs perspective.
So if the image built OK then you don't need me to test anything else?
Ah sorry - you said you used the image to build your custom one so I thought you used a FROM line with backstopjs as a source, which you did not like your wrote now, you just used parts of the Dockerfile, not the image itself ;). If you have the built image, just try to run backstopjs from that built image and do something with it - if that works, that's all we need to know.
This is looking pretty good to me. If it works locally I'll push it out and we can ask a few people with m1 macs to just run the sanity check.
I am not a docker expert, but I tried to install this today and it failed because
docker buildx
is only the command if you installed Docker Desktop. If you installed docker via the command line (such as using brew on macOS), you need to usedocker-buildx
instead ofdocker buildx
.
Doesn't matter for the issue here - although it looks weird that you don't have docker buildx syntax available. Look here:
https://github.com/docker/buildx#building-with-buildx
That's the official plugin syntax I used, what does a docker info
output for you?
PS: If your docker install is too old that it does not have plugin support you need to call the binary directly, see the docs - so maybe it is just your dated docker installation.
That's the official plugin syntax I used, what does a
docker info
output for you?
docker info
Client:
Context: colima
Debug Mode: false
Server:
Containers: 16
Running: 2
Paused: 0
Stopped: 14
Images: 24
Server Version: 20.10.18
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
runc version: 5fd4c4d144137e991c4acebb2146ab1483a97925
init version:
Security Options:
seccomp
Profile: default
Kernel Version: 5.15.68-0-virt
Operating System: Alpine Linux v3.16
OSType: linux
Architecture: aarch64
CPUs: 5
Total Memory: 7.752GiB
Name: colima
ID: VJ7F:UUVU:4ML7:GUBA:B2AN:NDJH:QQKZ:WG43:5OK4:EOJY:RDVM:Q7AO
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
I'm pretty sure this is the most recent version of docker, just installed via colima instead of Docker Desktop. I'm using colima because it works better with ddev, and there are others who use something like colima to avoid the licensing restrictions on Docker Desktop. So this isn't a huge deal, but I would suggest consider noting that "docker buildx" is specific to Docker Desktop, which not everyone using docker has.
That's the official plugin syntax I used, what does a
docker info
output for you?docker info Client: Context: colima Debug Mode: false Server: Containers: 16 Running: 2 Paused: 0 Stopped: 14 Images: 24 Server Version: 20.10.18 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: cgroupfs Cgroup Version: 1 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc Default Runtime: runc Init Binary: docker-init containerd version: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6 runc version: 5fd4c4d144137e991c4acebb2146ab1483a97925 init version: Security Options: seccomp Profile: default Kernel Version: 5.15.68-0-virt Operating System: Alpine Linux v3.16 OSType: linux Architecture: aarch64 CPUs: 5 Total Memory: 7.752GiB Name: colima ID: VJ7F:UUVU:4ML7:GUBA:B2AN:NDJH:QQKZ:WG43:5OK4:EOJY:RDVM:Q7AO Docker Root Dir: /var/lib/docker Debug Mode: false Registry: https://index.docker.io/v1/ Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false
I'm pretty sure this is the most recent version of docker, just installed via colima instead of Docker Desktop. I'm using colima because it works better with ddev, and there are others who use something like colima to avoid the licensing restrictions on Docker Desktop. So this isn't a huge deal, but I would suggest consider noting that "docker buildx" is specific to Docker Desktop, which not everyone using docker has.
That is wrong. It is not docker desktop specific. Please read:
https://github.com/abiosoft/colima/blob/main/docs/FAQ.md#docker-plugins-are-missing-buildx-scan
You just need to install the plugins for colima like written in the FAQ.
You just need to install the plugins for colima like written in the FAQ.
Thank you, I'm glad that it's way easier than I mistakenly thought.
Thank you @tkrah for this update! Will test a bit and follow up.
@tkrah I have a question -- what do I need to do to resolve this?
6.2.0
[+] Building 0.0s (0/0)
error: multiple platforms feature is currently not supported for docker driver. Please switch to a different driver (eg. "docker buildx create --use")
@tkrah I have a question -- what do I need to do to resolve this?
6.2.0 [+] Building 0.0s (0/0) error: multiple platforms feature is currently not supported for docker driver. Please switch to a different driver (eg. "docker buildx create --use")
@garris Please do the first part of the readme section with the multi arch build. You need to use your own builder.
@tkrah I assume you are talking about this?
I am sorry -- i looked at this doc for details and had some trouble parsing how this tool works. Is there a TLDR version of how this step works? E.g. it possible to use a default context? Or -- is there a node script I can create to initialize the correct context? Looking for ways to encapsulate this part.
@tkrah I assume you are talking about this?
![]()
I am sorry -- i looked at this doc for details and had some trouble parsing how this tool works. Is there a TLDR version of how this step works? E.g. it possible to use a default context? Or -- is there a node script I can create to initialize the correct context? Looking for ways to encapsulate this part.
@garris yeah that's the part. Just run that and it will work if you have no idea how it works ;-). You can't encapsulate the whole thing because it all depends on what the user wants to have for builder, names, defaults etc. in the used build environment. For example, my default builder is already multi platform capable and I don't need the first command, but if you just want a working builder, run that first one and be done with it ;-). It is usually a one time thing when you setup your build chain like installing docker, configuring networks etc.
Thanks @tkrah -- that helps.
I tried to run npm run build-docker
looks like the docker build is choking at...
RUN npm install -g --unsafe-perm=true --allow-root backstopjs@${BACKSTOPJS_VERSION}
Any thoughts?
Here's the output...
gshipon@gshipon-mn1 BackstopJS % npm run build-docker
> backstopjs@6.2.0 build-docker
> PV=$(node -p -e "require('./package.json').version"); echo $PV; docker buildx build --platform linux/amd64,linux/arm64 -t backstopjs/backstopjs:$PV --build-arg BACKSTOPJS_VERSION=$PV docker; docker buildx build --platform linux/amd64,linux/arm64 -t backstopjs/backstopjs:latest --build-arg BACKSTOPJS_VERSION=$PV docker
6.2.0
WARN[0000] No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
[+] Building 2.1s (10/15)
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 747B 0.0s
=> [linux/arm64 internal] load metadata for docker.io/library/node:16-bullseye 1.4s
=> [linux/amd64 internal] load metadata for docker.io/library/node:16-bullseye 1.0s
=> [auth] library/node:pull token for registry-1.docker.io 0.0s
=> [linux/amd64 1/5] FROM docker.io/library/node:16-bullseye@sha256:2944cdeed57569ddc51cef468a4b6f9610293eafe10603d26c752b0007ec00c5 0.0s
=> => resolve docker.io/library/node:16-bullseye@sha256:2944cdeed57569ddc51cef468a4b6f9610293eafe10603d26c752b0007ec00c5 0.0s
=> CACHED [linux/arm64 1/5] FROM docker.io/library/node:16-bullseye@sha256:2944cdeed57569ddc51cef468a4b6f9610293eafe10603d26c752b0007ec00c 0.0s
=> => resolve docker.io/library/node:16-bullseye@sha256:2944cdeed57569ddc51cef468a4b6f9610293eafe10603d26c752b0007ec00c5 0.0s
=> CACHED [linux/amd64 2/5] RUN apt-get -qq update >/dev/null && apt-get install -qq fonts-liberation chromium >/dev/null 0.0s
=> ERROR [linux/amd64 3/5] RUN npm install -g --unsafe-perm=true --allow-root backstopjs@6.2.0 0.7s
=> CANCELED [linux/arm64 2/5] RUN apt-get -qq update >/dev/null && apt-get install -qq fonts-liberation chromium >/dev/null 0.7s
------
> [linux/amd64 3/5] RUN npm install -g --unsafe-perm=true --allow-root backstopjs@6.2.0:
#8 0.634 npm notice
#8 0.634 npm notice New major version of npm available! 8.19.3 -> 9.6.2
#8 0.635 npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.6.2>
#8 0.636 npm notice Run `npm install -g npm@9.6.2` to update!
#8 0.637 npm notice
#8 0.641 npm ERR! code ETARGET
#8 0.643 npm ERR! notarget No matching version found for backstopjs@6.2.0.
#8 0.643 npm ERR! notarget In most cases you or one of your dependencies are requesting
#8 0.643 npm ERR! notarget a package version that doesn't exist.
#8 0.644
#8 0.645 npm ERR! A complete log of this run can be found in:
#8 0.645 npm ERR! /root/.npm/_logs/2023-03-21T04_53_56_146Z-debug-0.log
------
Dockerfile:15
--------------------
13 | # skip download, we already have it installed
14 | ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true
15 | >>> RUN npm install -g --unsafe-perm=true --allow-root backstopjs@${BACKSTOPJS_VERSION}
16 | RUN rm -rf /var/lib/apt/lists/* && apt-get -qq clean >/dev/null
17 |
--------------------
Thanks @tkrah -- that helps.
I tried to run
npm run build-docker
looks like the docker build is choking at...RUN npm install -g --unsafe-perm=true --allow-root backstopjs@${BACKSTOPJS_VERSION}
Any thoughts?
Here's the output...
gshipon@gshipon-mn1 BackstopJS % npm run build-docker > backstopjs@6.2.0 build-docker > PV=$(node -p -e "require('./package.json').version"); echo $PV; docker buildx build --platform linux/amd64,linux/arm64 -t backstopjs/backstopjs:$PV --build-arg BACKSTOPJS_VERSION=$PV docker; docker buildx build --platform linux/amd64,linux/arm64 -t backstopjs/backstopjs:latest --build-arg BACKSTOPJS_VERSION=$PV docker 6.2.0 WARN[0000] No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load [+] Building 2.1s (10/15) => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 747B 0.0s => [linux/arm64 internal] load metadata for docker.io/library/node:16-bullseye 1.4s => [linux/amd64 internal] load metadata for docker.io/library/node:16-bullseye 1.0s => [auth] library/node:pull token for registry-1.docker.io 0.0s => [linux/amd64 1/5] FROM docker.io/library/node:16-bullseye@sha256:2944cdeed57569ddc51cef468a4b6f9610293eafe10603d26c752b0007ec00c5 0.0s => => resolve docker.io/library/node:16-bullseye@sha256:2944cdeed57569ddc51cef468a4b6f9610293eafe10603d26c752b0007ec00c5 0.0s => CACHED [linux/arm64 1/5] FROM docker.io/library/node:16-bullseye@sha256:2944cdeed57569ddc51cef468a4b6f9610293eafe10603d26c752b0007ec00c 0.0s => => resolve docker.io/library/node:16-bullseye@sha256:2944cdeed57569ddc51cef468a4b6f9610293eafe10603d26c752b0007ec00c5 0.0s => CACHED [linux/amd64 2/5] RUN apt-get -qq update >/dev/null && apt-get install -qq fonts-liberation chromium >/dev/null 0.0s => ERROR [linux/amd64 3/5] RUN npm install -g --unsafe-perm=true --allow-root backstopjs@6.2.0 0.7s => CANCELED [linux/arm64 2/5] RUN apt-get -qq update >/dev/null && apt-get install -qq fonts-liberation chromium >/dev/null 0.7s ------ > [linux/amd64 3/5] RUN npm install -g --unsafe-perm=true --allow-root backstopjs@6.2.0: #8 0.634 npm notice #8 0.634 npm notice New major version of npm available! 8.19.3 -> 9.6.2 #8 0.635 npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.6.2> #8 0.636 npm notice Run `npm install -g npm@9.6.2` to update! #8 0.637 npm notice #8 0.641 npm ERR! code ETARGET #8 0.643 npm ERR! notarget No matching version found for backstopjs@6.2.0. #8 0.643 npm ERR! notarget In most cases you or one of your dependencies are requesting #8 0.643 npm ERR! notarget a package version that doesn't exist. #8 0.644 #8 0.645 npm ERR! A complete log of this run can be found in: #8 0.645 npm ERR! /root/.npm/_logs/2023-03-21T04_53_56_146Z-debug-0.log ------ Dockerfile:15 -------------------- 13 | # skip download, we already have it installed 14 | ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true 15 | >>> RUN npm install -g --unsafe-perm=true --allow-root backstopjs@${BACKSTOPJS_VERSION} 16 | RUN rm -rf /var/lib/apt/lists/* && apt-get -qq clean >/dev/null 17 | --------------------
@garris It wants to install backstopjs version 6.2.0 - which can't be found upstream, you need to release that one first:
https://www.npmjs.com/package/backstopjs
6.1.4 still the latest one yet. ;-)
Ok -- everything is done -- 6.2.0 is all published. Thanks again @tkrah !
Ok -- everything is done -- 6.2.0 is all published. Thanks again @tkrah !
Thanks for considering and merging it and I hope it's going to work well for all.
Proposal to support amd64 / arm64 platform for the docker image, just a draft to discuss it.