Add support for `ssh` with remote driver

[trajano]

Currently remote driver only supports TLS encryption. It would make it easier for people who have existing machines to just set up the SSH connection much like the Docker SSH Context

[jedevc]

This is actually currently supported in buildx - however, the docs just need updating to include this as an option.

[trajano]

Hmm ok weird. I couldn't get it to work. It has a connection but it never actually sends anything for building.

[trajano]

I was experimenting with buildkitd.default.toml (there wasn't much in terms of information aside from the example) but because I had that file

WARNING: Using default BuildKit config in C:\Users\trajano.docker\buildx/buildkitd.default.toml ERROR: failed to initialize builder ec2 (ec20): setting config file is not supported for remote driver

Anyway I removed the file for now and did this

docker buildx create \
  --name ec2 \
  --driver remote \
  ssh://ec2

Then ran

> docker build .
[+] Building 3.6s (16/29)                                                          ** docker:default **

It appears to still be using the default docker.

[jedevc]

You need to also run docker buildx use ec2 or add the --use flag to your buildx create command.

[trajano]

Darn I missed that line

I did do that

[trajano]

But it shows this

On remote it shows it's running

[trajano]

I also tried

docker buildx create   --name ec2   --driver remote  --bootstrap --use ssh://ec2

But all it does is show me

[+] Building 0.0s (0/0)

And nothing else

[buldezir]

to bootstrap remote ssh node u need to have rights for buildkit socket. i think good way is to make it same as docker.sock

sudo chown root:docker /var/run/buildkit/buildkitd.sock

[nikelborm]

Can somebody who successfully created ssh builder give me the syntax of the command you used to do it? The command I try

docker buildx create --name wdsr --driver remote --bootstrap --use ssh://ubuntu@remote_ip_adress:32

fails after timeout with error

[+] Building 20.0s (1/1) FINISHED                                                                                                                             
 => ERROR [internal] waiting for connection             20.0s
------
 > [internal] waiting for connection:
------
ERROR: context deadline exceeded

[trajano]

to bootstrap remote ssh node u need to have rights for buildkit socket. i think good way is to make it same as docker.sock

sudo chown root:docker /var/run/buildkit/buildkitd.sock

cannot access '/var/run/buildkit/buildkitd.sock': No such file or directory

[kctang]

Sharing my findings. My use case is to have a remote machine build docker images. I am able to use this:

# create builder
docker buildx create --name amd64-builder --driver docker-container --platform linux/amd64 ssh://user@remote-machine

# use builder
docker buildx build --builder amd64-builder --load .

Key is to use docker-container as the driver instead of remote. I do know if there are downsides to using docker-container driver.

[trajano]

@kctang Wouldn't docker-container just run it locally?

[kctang]

@kctang Wouldn't docker-container just run it locally?

Based on the example I provided, when you build with the builder, docker build will connect via ssh to ssh://user@remote-machine and start a docker container on remote-machine to execute the actual image building. You will need to setup SSH key based auth for this to work.

When the build process is running, you should be able to see the docker process in the remote (builder) machine:

$ docker ps

be83a6a5ed6f   moby/buildkit:buildx-stable-1   "buildkitd"              3 days ago   Up About a minute                                               buildx_buildkit_remote

[trajano]

@kctang I can confirm that I am experiencing the same behaviour as you.

It also works with docker compose when doing docker buildx use mybuilder

[trajano]

Not sure if something has changed, but in Docker Desktop 4.30.0 windows docker compose yields

error during connect: Get "http://docker.example.com/v1.39/containers/buildx_buildkit_dit0/json": command [ssh -o ConnectTimeout=30 -- dit docker system dial-stdio] has exited with exit status 255, please make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=Connection closed by xxxxx port 22

but docker buildx build still works.