`network=host` is not picking up entries in host's `/etc/hosts` in docker 23

tgquan67 commented 1 year ago

Contributing guidelines

[X] I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

[X] ... the documentation does not mention anything about my problem
[x] ... there are no open or closed issues that are related to my problem

Description

Even when I enabled network=host, buildx is not picking up entries in host's /etc/hosts. First I created a new builder with

docker buildx create --bootstrap --use --config ~/.config/buildkit/buildkitd.toml --name custom-builder --driver docker-container --driver-opt network=host

with the content of ~/.config/buildkit/buildkitd.toml as

debug = true
insecure-entitlements = [ "network.host", "security.insecure" ]

I also added 127.0.0.1 random.blah to host's /etc/hosts and tried to build the following dockerfile

FROM ubuntu:bionic

RUN apt update && apt install dnsutils -y
RUN --network=host nslookup random.blah &&  sleep 15

with the following command

docker buildx build --allow network.host --network=host --no-cache .

Expected behaviour

When I have DOCKER_BUILDKIT=0, the result is as follow

Step 3/3 : RUN nslookup random.blah &&  sleep 15
 ---> Running in 2af6847b213e
Server:         127.0.0.53
Address:        127.0.0.53#53

Non-authoritative answer:
Name:   random.blah
Address: 127.0.0.1

Actual behaviour

When buildkit is used, I get this result

[+] Building 18.3s (6/6) FINISHED
 => [internal] load .dockerignore                                                                                                              0.0s
 => => transferring context: 2B                                                                                                                0.0s
 => [internal] load build definition from Dockerfile                                                                                           0.0s
 => => transferring dockerfile: 139B                                                                                                           0.0s
 => [internal] load metadata for docker.io/library/ubuntu:bionic                                                                               1.5s
 => [1/3] FROM docker.io/library/ubuntu:bionic@sha256:8aa9c2798215f99544d1ce7439ea9c3a6dfd82de607da1cec3a8a2fae005931b                         1.5s
 => => resolve docker.io/library/ubuntu:bionic@sha256:8aa9c2798215f99544d1ce7439ea9c3a6dfd82de607da1cec3a8a2fae005931b                         0.0s
 => => sha256:0c5227665c11379f79e9da3d3e4f1724f9316b87d259ac0131628ca1b923a392 25.69MB / 25.69MB                                               0.6s
 => => extracting sha256:0c5227665c11379f79e9da3d3e4f1724f9316b87d259ac0131628ca1b923a392                                                      0.9s
 => [2/3] RUN apt update && apt install dnsutils -y                                                                                           15.0s
 => ERROR [3/3] RUN nslookup random.blah &&  sleep 15                                                                                          0.2s
------
 > [3/3] RUN nslookup random.blah &&  sleep 15:
#0 0.134 Server:                8.8.8.8
#0 0.134 Address:       8.8.8.8#53
#0 0.134
#0 0.134 ** server can't find random.blah: NXDOMAIN
#0 0.134
------
WARNING: buildx: failed to get git commit: fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Dockerfile:4
--------------------
   2 |
   3 |     RUN apt update && apt install dnsutils -y
   4 | >>> RUN nslookup random.blah &&  sleep 15
   5 |
   6 |
--------------------
ERROR: failed to solve: process "/bin/sh -c nslookup random.blah &&  sleep 15" did not complete successfully: exit code: 1

Buildx version

github.com/docker/buildx v0.10.2 00ed17d

Docker info

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.16.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
  scan: Docker Scan (Docker Inc.)
    Version:  v0.23.0
    Path:     /usr/libexec/docker/cli-plugins/docker-scan

Server:
 Containers: 3
  Running: 1
  Paused: 0
  Stopped: 2
 Images: 3
 Server Version: 23.0.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2456e983eb9e37e47538f59ea18f2043c9a73640
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
 Kernel Version: 5.4.0-144-generic
 Operating System: Ubuntu 20.04.6 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 6
 Total Memory: 62.85GiB
 Name: quan-work-ubuntu
 ID: DPTV:QXJW:TN6C:CVLT:2DU7:DGNV:7IRF:UGNG:AOPX:T3VE:JS3X:6KIP
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Experimental: false
 Insecure Registries:
  localhost:32000
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No swap limit support

Builders list

NAME/NODE           DRIVER/ENDPOINT             STATUS  BUILDKIT PLATFORMS
custom-builder *  docker-container
  custom-builder0 unix:///var/run/docker.sock running v0.11.4  linux/amd64, linux/amd64/v2, linux/386
default             docker
  default           default                     running 23.0.1   linux/amd64, linux/amd64/v2, linux/386

Configuration

FROM ubuntu:bionic

RUN apt update && apt install dnsutils -y
RUN --network=host nslookup random.blah &&  sleep 15

Logs

WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
[+] Building 18.6s (6/6) FINISHED
 => [internal] load build definition from Dockerfile                                                                                           0.0s
 => => transferring dockerfile: 154B                                                                                                           0.0s
 => [internal] load .dockerignore                                                                                                              0.0s
 => => transferring context: 2B                                                                                                                0.0s
 => [internal] load metadata for docker.io/library/ubuntu:bionic                                                                               2.0s
 => CACHED [1/3] FROM docker.io/library/ubuntu:bionic@sha256:8aa9c2798215f99544d1ce7439ea9c3a6dfd82de607da1cec3a8a2fae005931b                  0.0s
 => => resolve docker.io/library/ubuntu:bionic@sha256:8aa9c2798215f99544d1ce7439ea9c3a6dfd82de607da1cec3a8a2fae005931b                         0.0s
 => [2/3] RUN apt update && apt install dnsutils -y                                                                                           16.4s
 => ERROR [3/3] RUN --network=host nslookup random.blah &&  sleep 15                                                                           0.1s
------
 > [3/3] RUN --network=host nslookup random.blah &&  sleep 15:
#0 0.111 Server:                8.8.8.8
#0 0.111 Address:       8.8.8.8#53
#0 0.111
#0 0.111 ** server can't find random.blah: NXDOMAIN
#0 0.111
------
WARNING: buildx: failed to get git commit: fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Dockerfile:4
--------------------
   2 |
   3 |     RUN apt update && apt install dnsutils -y
   4 | >>> RUN --network=host nslookup random.blah &&  sleep 15
   5 |
   6 |
--------------------
ERROR: failed to solve: process "/bin/sh -c nslookup random.blah &&  sleep 15" did not complete successfully: exit code: 1

Additional info

No response

mattpopovich commented 1 year ago

I can second this and can recreate it.

docker info:

```console $ docker info Client: Context: default Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.10.4 Path: /usr/libexec/docker/cli-plugins/docker-buildx compose: Docker Compose (Docker Inc.) Version: v2.17.3 Path: /usr/libexec/docker/cli-plugins/docker-compose Server: Containers: 222 Running: 2 Paused: 0 Stopped: 220 Images: 1092 Server Version: 23.0.5 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Using metacopy: false Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: systemd Cgroup Version: 2 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: runc io.containerd.runc.v2 nvidia Default Runtime: runc Init Binary: docker-init containerd version: 2806fc1057397dbaeefbea0e4e17bddfbd388f38 runc version: v1.1.5-0-gf19387a init version: de40ad0 Security Options: apparmor seccomp Profile: builtin cgroupns Kernel Version: 5.15.0-71-generic Operating System: Ubuntu 22.04.2 LTS OSType: linux Architecture: x86_64 CPUs: 64 Total Memory: 251.6GiB Name: ID: 6GXK:4BDF:RJG4:HDPI:U4NP:4R7Q:7C4M:PZZQ:PDJP:VUTN:G6EG:LN2F Docker Root Dir: /fast/docker/data Debug Mode: false Registry: https://index.docker.io/v1/ Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: true ```

Dockerfile:

FROM ubuntu:22.10
RUN cat /etc/hosts

Build with DOCKER_BUILDKIT=0 (/etc/hosts is propagated as expected):

$ DOCKER_BUILDKIT=0 docker build --network=host  . 
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
            BuildKit is currently disabled; enable it by removing the DOCKER_BUILDKIT=0
            environment-variable.

Sending build context to Docker daemon  15.06MB
Step 1/2 : FROM ubuntu:22.10
 ---> 91e34ae906cf
Step 2/2 : RUN cat /etc/hosts
 ---> Running in 706f6313eb3b
127.0.0.1       localhost
123.123.123.123 customhost
[...]
contents from /etc/hosts on local machine
[...]
Removing intermediate container 706f6313eb3b
 ---> 95c682678349
Successfully built 95c682678349

Build with DOCKER_BUILDKIT=1 (/etc/hosts is not propagated):

$ docker build --network=host --progress=plain . 
#1 [internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.0s

#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 95B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/ubuntu:22.10
#3 DONE 0.0s

#4 [1/2] FROM docker.io/library/ubuntu:22.10
#4 CACHED

#5 [2/2] RUN cat /etc/hosts
#0 0.098 127.0.0.1      localhost buildkitsandbox
#0 0.098 ::1    localhost ip6-localhost ip6-loopback
#5 DONE 0.1s

#6 exporting to image
#6 exporting layers 0.0s done
#6 writing image sha256:b4f9a7fd4d366808fb6660a62b63d00290cb21bee684d2d8733c7b98b9a1daee done
#6 DONE 0.0s

Is this an expected result? Am I doing something wrong? Or is this a bug?

omartrigui commented 1 year ago

At this moment, I'm afraid that our sole option is to utilize an external DNS resolver that we manage and point the buildx builders towards it. Following a sample /etc/docker/daemon.json DNS config.

{
    "dns": [
        "172.x.x.x"
    ]
}

Oscaner commented 10 months ago

Hi, guys

Does any workaround?

In my case, --network=host does not affect /etc/hosts, and --add-host host:ip works fine. But I have entries large than 20, I don't want to write 20 --add-host for docker buildx build

docker / buildx