Open rassie opened 4 years ago
What version of docker is this?
I can give you an exact version when I'm at the console again, but it's whatever the current version for Ubuntu is (Docker's repositories, not Ubuntu's). I've also reproduced the problem with a current -dind
Docker container.
@tonistiigi my version is
Client:
Debug Mode: false
Server:
Containers: 7
Running: 0
Paused: 0
Stopped: 7
Images: 106
Server Version: 19.03.5
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
init version: fec3683
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 5.3.0-21-generic
Operating System: Ubuntu 18.04.3 LTS
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 31.22GiB
[snip]
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No swap limit support
Yes, this is somewhat expected by the current implementation. These files are configured by the container runtime. Even with the old implementation, these files do not really persist, you can write into them but the next command will not see the data you wrote, leading to a different error case. BuildKit will not allow writes instead of silently ignoring them. We could try to keep compatibility but it does have a bit of a performance overhead that would apply to all the containers, not just the ones the try to write to these files.
I edited them to fake the docker hostname for applications that rely on these files and it helped and needed just during build stage due the limit to change the host. For running containers we can specify the exactly hostname and edit /etc/hosts without any limits so this feature is missing in buildx
I was having this issue, under the impression that --add-host
was a runtime only flag -- it turns out I was wrong, and the solution for me was to use docker build --add-host=hostname:ip ...
and remove modification of /etc/hosts
from my Dockerfile. Not sure this will work for every use-case, though.
Hi,
If you want to set specific hostname for during build time, now it implemented under this pull request: https://github.com/moby/buildkit/pull/1339 You can vote for it to be merged asap.
Also running into this... Is there any way to allow DOCKER_BUILDKIT=1 docker build .
with below Dockerfile to succeed, so a legacy package installation does not have to be rewritten?
It's okay if the /etc/hosts
changes are thrown away (pre-DOCKER_BUILDKIT behavior), the package installer just can't get the Read-only file system
error.
I tried RUN --mount=type=tmpfs ...
but that needs to replace the entire /etc
directory and I don't want to lose other changes made by the legacy package installation.
FROM centos:6.10 as intermediate
# A legacy package installation tries something like this, which fails with DOCKER_BUILDKIT=1
# /bin/sh: /etc/hosts: Read-only file system
#
RUN echo 127.0.0.1 foo >> /etc/hosts
PS: this older issue is not really related: https://github.com/moby/moby/issues/11950
I am also running into this, just trying to build a simple archlinux container. Turning off buildkit solves this problem. It would appear recently (sometime, in 2020 maybe, between June and December?) Docker for Mac made a signfigant shift to build kit? This same Dockerfile that used to work now does not.
I posted this question before finding this thread, for full details: https://unix.stackexchange.com/questions/631177/arch-linux-in-docker-on-a-free-system-is-running-out-of-space/631178#631178
I don't think there are plans to "fix" this. As I explained before these files are not modifiable in containers and managed by the runtime. Changing them leads to confusing behavior and broken images. --add-host
works as intended also in BuildKit.
I understand. But I think what will happen is that people are going to hit this and it will be "obscure" to them because it's indirect. I'm not directly doing anything with resolve or hosts. I'm merely building the same docker image that I've built for years, in the same way, and now suddenly it doesn't work. The error that gets posted out, as I wrote in that stackexchange post highlights the error here, albeit with a minor red herring about space, but again to the user it's Indirect unless they are doing things (purposefully) with the hosts or resolve file. People running say just an OS like Arch or Ubuntu, this could be a "hidden" landmine in their typical work flow; especially since they may not be aware that common docker tools have recently switched to buildkit "under the covers" (docker for mac for example)
I don't think there are plans to "fix" this. As I explained before these files are not modifiable in containers and managed by the runtime. Changing them leads to confusing behavior and broken images. --add-host works as intended also in BuildKit.
It's a tricky one; I agree that for users that want to customise during build, --add-host
should be the recommendation; I guess (reading the discussion above) the problem, is that some packages (for whatever reason) appear to want to have these files writable. While it's worth looking why they want to do so, generally this would be out of control for the user that performs the build.
I saw https://github.com/moby/moby/issues/11950 was linked above; probably relevant here are https://github.com/moby/moby/issues/2267, and https://github.com/moby/moby/pull/5129, which (although not "persisting"), makes both of these writable by mounting the files.
I do share the concerns about performance though, so not sure what's best.
@thaJeztah sorry to labour the issue, but am i right in saying that there is still no workaround for the original issue, where installing arch linux package 'filesystem' causes a read only error due to bind mounts to the host?, i have gone through the links above but cannot see a way around this issue that relate to installation of a package that needs write access to /etc/hosts and /etc/resolv.conf please correct me if i'm wrong here.
@binhex correct; BuildKit currently mounts these read-only;
With BuildKit:
DOCKER_BUILDKIT=1 docker build --no-cache --progress=plain -<<EOF
FROM busybox
RUN mount | grep /etc
RUN cat /etc/hosts
EOF
#5 [2/2] RUN mount | grep /etc
#5 sha256:30dbb0d55d6b8a70aaf13d4655662c2ee23283d7fcbccceb70ff0a011adef4f1
#5 0.277 /dev/vda1 on /etc/resolv.conf type ext4 (ro,nosuid,nodev,noexec,relatime)
#5 0.277 /dev/vda1 on /etc/hosts type ext4 (ro,nosuid,nodev,noexec,relatime)
#5 DONE 0.4s
Without BuildKit:
DOCKER_BUILDKIT=0 docker build --no-cache --progress=plain -<<EOF
FROM busybox
RUN mount | grep /etc
EOF
Step 2/2 : RUN mount | grep /etc
---> Running in fbe072913354
/dev/vda1 on /etc/resolv.conf type ext4 (rw,relatime)
/dev/vda1 on /etc/hostname type ext4 (rw,relatime)
/dev/vda1 on /etc/hosts type ext4 (rw,relatime)
If your intent is to install the filesystem
package to build a custom/more recent "base image" for arch linux, please refer to https://docs.docker.com/develop/develop-images/baseimages/
Just for anybody who finds this useful, i stumbled across the OP issue when using GitHub Actions and attempting to build a Arch Linux Docker image 'from scratch' using the buildx step:-
uses: docker/setup-buildx-action@v1
if i then used the following 'uses' action then it blew up with the same read only issue for /etc/hosts
and /etc/resolv.conf
as OP.
uses: docker/build-push-action@v2
the solution for me is simply to ignore the filesystem package during upgrade by doing the following:-
sed -i -e 's~#IgnorePkg.*~IgnorePkg = filesystem~g' '/etc/pacman.conf'
pacman then happily ignores the upgrade of filesystem and completes, this is obviously not ideal but it works and im happy enough to share this as a workaround for now until when/if the filesystem
package stops attempting to modify the two files mentioned above.
A lot of posts here mention --add-host
as the proper way to change /etc/hosts
, but what about /etc/resolv.conf
? My organization uses different nameservers on the host than we do during build, so what we'd probably use ideally is the --dns
option that docker run
has. However there appears to be no such option for docker build
, so for now using BuildKit doesn't seem to be an option for us.
Is there any way to explicitly set nameservers for a single docker build
command?
I don't think there are plans to "fix" this. As I explained before these files are not modifiable in containers and managed by the runtime. Changing them leads to confusing behavior and broken images.
--add-host
works as intended also in BuildKit.
I understand the situation to use --add-host for docker build, but how to use it for buildctl? I couldn't find any way to use --add-host in buildctl build
@TingxinLi --opt add-hosts=name=ip,name2=ip2
@TingxinLi
--opt add-hosts=name=ip,name2=ip2
Thanks, I figured it out after reading the source code.
how can i add-host
to a bakefile?
edit: this does the trick for me, i can pass network=host when creating the builder (note: there's security implications for that)
docker buildx create --driver-opt network=host
I have to add to this issue as well... I either need /etc/hosts to be writable or need the hostname to not resolve to a loopback address. Preferably, the hostname in /etc/hosts should resolve to the primary IPv4 address of the container like it does in non-buildkit mode.
Reasoning: I have to prebuild an Oracle database during a container build, sigh... (With their current free database version 23.5, they don't accept loopback addresses as IPs for the given hostname and this hostname is taken from /etc/hostname, which is also read-only and correlates with the 127.0.0.1 entry in /etc/hosts. Adding the hostname with a different IP again using --add-host is ignored because it gets appended to the end of /etc/hosts. For now, I have to stick to the non-buildkit Docker build mode because it seems easier than getting a fix from Oracle...)
My Dockerfile:
The
filesystem
package contains most of the base system files, as can be seen at https://www.archlinux.org/packages/core/x86_64/filesystem/. If it's upgraded, it tries to write/etc/hosts
and/etc/resolv.conf
, which leads to errors with BuildKit ("classical" Docker build is fine):