Closed NavyD closed 11 months ago
The hostname obtained by using 'chezmoi data' in container b is always
debuerreotype
, while the output of the commandhostname
executed in the container isbuildkitsandbox
debuerreotype
seems to be some kind of Debian rootfs builder. This is overriding the hostname, likely to ensure that builds are repeatable.
So, I strongly suspect that this is coming from your Docker setup. Under the hood, chezmoi uses a number of methods to determine the hostname.
Sorry, I'm not very familiar with the relevant content, but I still encountered the same problem when installing the new Debian 12 and Install Docker using the convenience script in the vmware
To clarify: are you experiencing this inside the docker image you are using / building, or from the host you are building the docker image from?
To clarify: are you experiencing this inside the docker image you are using / building, or from the host you are building the docker image from?
I encountered this situation on the host where I am building the Docker image
The physical host I am using is a laptop windows 11 x64 platform. I develop in debian linux installed with the wsl2 feature enabled in win, and install docker-ce in wsl2. I try to use the docker in it to build an image that uses chezmoi to test dotfiles. After that, I discovered and raised this problem.
In order to reduce the possible impact on the environment, I used vmware workstation pro 16 on my laptop win to create a new Linux debian 12 virtual machine, installed docker-ce, and reproduced the problem after running this command using docker in it.
docker build --progress plain -f - . <<'EOF'
FROM debian:12
ARG CZ_VERSION=2.40.4
RUN export DEBIAN_FRONTEND=noninteractive && \
apt-get update && \
apt-get install -yq curl jq && \
sh -c "$(curl -fsLS get.chezmoi.io)" -- -t "v${CZ_VERSION}" && \
chezmoi --version | grep "${CZ_VERSION}"
SHELL ["/bin/sh", "-xc"]
RUN [ "$(chezmoi data | jq -r '.chezmoi.hostname')" = "$(hostname)" ]
EOF
If you need any system information, please let me know. By the way, has anyone else encountered this problem?
In your docker container, what is the output of the following commands?
cat /etc/hosts
cat /etc/hostname
grep -r debuerreotype /etc
grep -r buildkitsandbox /etc
uname -a
In your docker container, what is the output of the following commands?
cat /etc/hosts cat /etc/hostname grep -r debuerreotype /etc grep -r buildkitsandbox /etc uname -a
Thank you very much for your attention. I apologize for the delayed response as I have been quite busy recently.
I have tried running in both the build container and the run container on WSL2 debian 12 and vmware debian 12. Although there is some unnecessary output, I hope it will be helpful in solving this issue.
In wsl2
$ docker build --progress plain -f - . <<'EOF'
FROM debian:12
ARG CZ_VERSION=2.40.4
RUN export DEBIAN_FRONTEND=noninteractive && \
apt-get update && \
apt-get install -yq curl jq && \
sh -c "$(curl -fsLS get.chezmoi.io)" -- -t "v${CZ_VERSION}" && \
chezmoi --version | grep "${CZ_VERSION}"
RUN set -x; cat /etc/hosts ; \
cat /etc/hostname ; \
grep -r debuerreotype /etc ; \
grep -r buildkitsandbox /etc ; \
uname -a
RUN set -x; [ "$(chezmoi data | jq -r '.chezmoi.hostname')" = "$(hostname)" ]
EOF
#0 building with "default" instance using docker driver
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 518B done
#1 DONE 0.0s
#2 [internal] load .dockerignore
#2 transferring context: 2.33kB done
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/debian:12
#3 DONE 0.0s
#4 [1/4] FROM docker.io/library/debian:12
#4 DONE 0.0s
#5 [2/4] RUN export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get install -yq curl jq && sh -c "$(curl -fsLS get.chezmoi.io)" -- -t "v2.40.4" && chezmoi --version | grep "2.40.4"
#5 CACHED
#6 [3/4] RUN set -x; cat /etc/hosts ; cat /etc/hostname ; grep -r debuerreotype /etc ; grep -r buildkitsandbox /etc ; uname -a
#6 0.321 + cat /etc/hosts
#6 0.321 127.0.0.1 localhost buildkitsandbox
#6 0.321 ::1 localhost ip6-localhost ip6-loopback
#6 0.322 + cat /etc/hostname
#6 0.322 debuerreotype
#6 0.322 + grep -r debuerreotype /etc
#6 0.327 /etc/apt/apt.conf.d/docker-clean:# open to better suggestions (https://github.com/debuerreotype/debuerreotype/issues).
#6 0.327 /etc/hostname:debuerreotype
#6 0.327 + grep -r buildkitsandbox /etc
#6 0.331 /etc/hosts:127.0.0.1 localhost buildkitsandbox
#6 0.331 + uname -a
#6 0.332 Linux buildkitsandbox 5.15.133.1-microsoft-standard-WSL2 #1 SMP Thu Oct 5 21:02:42 UTC 2023 x86_64 GNU/Linux
#6 DONE 0.4s
#7 [4/4] RUN set -x; [ "$(chezmoi data | jq -r '.chezmoi.hostname')" = "$(hostname)" ]
#7 0.470 + chezmoi data+
#7 0.470 jq -r .chezmoi.hostname
#7 0.502 + hostname
#7 0.502 + [ debuerreotype = buildkitsandbox ]
#7 ERROR: process "/bin/sh -c set -x; [ \"$(chezmoi data | jq -r '.chezmoi.hostname')\" = \"$(hostname)\" ]" did not complete successfully: exit code: 1
------
> [4/4] RUN set -x; [ "$(chezmoi data | jq -r '.chezmoi.hostname')" = "$(hostname)" ]:
0.470 + chezmoi data+
0.470 jq -r .chezmoi.hostname
0.502 + hostname
0.502 + [ debuerreotype = buildkitsandbox ]
------
Dockerfile:13
--------------------
11 | grep -r buildkitsandbox /etc ; \
12 | uname -a
13 | >>> RUN set -x; [ "$(chezmoi data | jq -r '.chezmoi.hostname')" = "$(hostname)" ]
14 |
--------------------
ERROR: failed to solve: process "/bin/sh -c set -x; [ \"$(chezmoi data | jq -r '.chezmoi.hostname')\" = \"$(hostname)\" ]" did not complete successfully: exit code: 1
$ docker run --rm -it debian:12 sh -xc 'cat /etc/hosts ; \
cat /etc/hostname ; \
grep -r debuerreotype /etc ; \
grep -r buildkitsandbox /etc ; \
uname -a'
+ cat /etc/hosts
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.2 b8b673435508
+ cat /etc/hostname
b8b673435508
+ grep -r debuerreotype /etc
/etc/apt/apt.conf.d/docker-clean:# open to better suggestions (https://github.com/debuerreotype/debuerreotype/issues).
+ grep -r buildkitsandbox /etc
+ uname -a
Linux b8b673435508 5.15.133.1-microsoft-standard-WSL2 #1 SMP Thu Oct 5 21:02:42 UTC 2023 x86_64 GNU/Linux
In vmware
$ docker build --progress plain -f - . <<'EOF'
FROM debian:12
ARG CZ_VERSION=2.40.4
RUN export DEBIAN_FRONTEND=noninteractive && \
apt-get update && \
apt-get install -yq curl jq && \
sh -c "$(curl -fsLS get.chezmoi.io)" -- -t "v${CZ_VERSION}" && \
chezmoi --version | grep "${CZ_VERSION}"
RUN set -x; cat /etc/hosts ; \
cat /etc/hostname ; \
grep -r debuerreotype /etc ; \
grep -r buildkitsandbox /etc ; \
uname -a
RUN set -x; [ "$(chezmoi data | jq -r '.chezmoi.hostname')" = "$(hostname)" ]
EOF
#0 building with "default" instance using docker driver
#1 [internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.0s
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 518B done
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/debian:12
#3 DONE 0.0s
#4 [1/4] FROM docker.io/library/debian:12
#4 DONE 0.0s
#5 [2/4] RUN export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get install -yq curl jq && sh -c "$(curl -fsLS get.chezmoi.io)" -- -t "v2.40.4" && chezmoi --version | grep "2.40.4"
#5 CACHED
#6 [3/4] RUN set -x; cat /etc/hosts ; cat /etc/hostname ; grep -r debuerreotype /etc ; grep -r buildkitsandbox /etc ; uname -a
#6 0.494 + cat /etc/hosts
#6 0.500 127.0.0.1 localhost buildkitsandbox
#6 0.500 ::1 localhost ip6-localhost ip6-loopback
#6 0.502 + cat /etc/hostname
#6 0.505 debuerreotype
#6 0.507 + grep -r debuerreotype /etc
#6 0.578 /etc/apt/apt.conf.d/docker-clean:# open to better suggestions (https://github.com/debuerreotype/debuerreotype/issues).
#6 0.578 /etc/hostname:debuerreotype
#6 0.579 + grep -r buildkitsandbox /etc
#6 0.609 /etc/hosts:127.0.0.1 localhost buildkitsandbox
#6 0.610 + uname -a
#6 0.614 Linux buildkitsandbox 6.1.0-13-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.55-1 (2023-09-29) x86_64 GNU/Linux
#6 DONE 0.7s
#7 [4/4] RUN set -x; [ "$(chezmoi data | jq -r '.chezmoi.hostname')" = "$(hostname)" ]
#7 1.155 + chezmoi data
#7 1.155 + jq -r .chezmoi.hostname
#7 1.356 + hostname
#7 1.363 + [ debuerreotype = buildkitsandbox ]
#7 ERROR: process "/bin/sh -c set -x; [ \"$(chezmoi data | jq -r '.chezmoi.hostname')\" = \"$(hostname)\" ]" did not complete successfully: exit code: 1
------
> [4/4] RUN set -x; [ "$(chezmoi data | jq -r '.chezmoi.hostname')" = "$(hostname)" ]:
1.155 + chezmoi data
1.155 + jq -r .chezmoi.hostname
debuerreotype = buildkitsandbox ]
------
Dockerfile:13
--------------------
11 | grep -r buildkitsandbox /etc ; \
12 | uname -a
13 | >>> RUN set -x; [ "$(chezmoi data | jq -r '.chezmoi.hostname')" = "$(hostname)" ]
14 |
--------------------
ERROR: failed to solve: process "/bin/sh -c set -x; [ \"$(chezmoi data | jq -r '.chezmoi.hostname')\" = \"$(hostname)\" ]" did not complete successfully: exit code: 1
$ docker run --rm -it debian:12 sh -xc 'cat /etc/hosts ; \
cat /etc/hostname ; \
grep -r debuerreotype /etc ; \
grep -r buildkitsandbox /etc ; \
uname -a'
+ cat /etc/hosts
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.2 8f1c3ca97b64
+ cat /etc/hostname
8f1c3ca97b64
+ grep -r debuerreotype /etc
/etc/apt/apt.conf.d/docker-clean:# open to better suggestions (https://github.com/debuerreotype/debuerreotype/issues).
+ grep -r buildkitsandbox /etc
+ uname -a
Linux 8f1c3ca97b64 6.1.0-13-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.55-1 (2023-09-29) x86_64 GNU/Linux
Thanks for the detailed information!
#6 [3/3] RUN [ "$(chezmoi data | jq -r '.chezmoi.hostname')" = "$(hostname)" ] #6 0.802 + chezmoi data #6 0.804 + jq -r .chezmoi.hostname #6 1.635 + hostname #6 1.640 + [ debuerreotype = buildkitsandbox ]
#6 0.322 + cat /etc/hostname #6 0.322 debuerreotype
This demonstrates the root cause. The hostname
command inside the container returns buildkitsandbox
whereas /etc/hostname
contains debuerreotype
.
Basically, this looks like a bug in debuerreotype. It looks like debuerreotype is trying to override the hostname. debuerreotype updates /etc/hostname
in the container, but does not call the sethostname(2)
system call and so the container ends up in an inconsistent state: the hostname will be either debuerreotype
(if the program reads the hostname from the filesystem) or buildkitsandbox
(if the program uses the gethostname(2)
syscall).
More fundamentally, determining the hostname reliably is a hard problem (and determining the FQDN hostname is even harder) and there does not seem to be a solution that works in all cases.
Therefore, I would recommend that you either treat the hostnames buildkitsandbox
and debuerreotype
as equivalent, or use a different variable to .chezmoi.hostname
in your templates.
This strange issue indeed seems to require careful handling in the code. I have made the necessary modifications to pass the tests, and I just hope that no one else encounters such a problem.
Thank you very much for your patient explanation
I wanted to give a bit more detail on what was reported here, so I added some debug logging to FQDNHostname
and ran it in the docker build process. Exact changes are below, but the output is this:
#0 building with "colima" instance using docker driver
#1 [internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.0s
#2 [internal] load build definition from Dockerfile.hostname
#2 transferring dockerfile: 424B done
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/debian:12
#3 DONE 0.2s
#4 [1/4] FROM docker.io/library/debian:12@sha256:133a1f2aa9e55d1c93d0ae1aaa7b94fb141265d0ee3ea677175cdb96f5f990e5
#4 DONE 0.0s
#5 [internal] load build context
#5 transferring context: 33.04MB 0.2s done
#5 DONE 0.2s
#6 [2/4] RUN export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get install -yq curl jq
#6 CACHED
#7 [3/4] COPY ./dist/chezmoi-nocgo_linux_arm64/chezmoi /usr/bin/chezmoi
#7 DONE 0.1s
#8 [4/4] RUN chezmoi --version && chezmoi execute-template 'chezmoi: {{ .chezmoi.hostname }}' && echo 'hostname: '$(hostname)
#8 0.114 + chezmoi --version
#8 0.136 chezmoi version v2.42.1-alpha.0, commit 971e03fda9b631456561eb996616ce0e30a3ab28, built at 2023-11-27T17:51:58Z, built by goreleaser
#8 0.138 + chezmoi execute-template chezmoi: {{ .chezmoi.hostname }}
#8 0.155 os.Hostname() = "buildkitsandbox"
#8 0.155 etcHostsFQDNHostname = ""
#8 0.155 etcHostFQDNHostname = "debuerreotype"
#8 0.156 chezmoi: debuerreotype+ hostname
#8 0.157 + echo hostname: buildkitsandbox
#8 0.157 hostname: buildkitsandbox
#8 DONE 0.2s
#9 exporting to image
#9 exporting layers 0.1s done
#9 writing image sha256:e8c23e2dfb03d419f8aecdcfaa1fcb2accdb20c5262b1d04a102afb15bf24e39 done
#9 DONE 0.1s
The key point is that because neither debuerrotype
nor buildkitsandbox
are FQDNs, chezmoi just returns what is used from os.Hostname()
, and that because sethostname
is not called by the Debian 12 build process, the effect is exactly what @twpayne said previously.
Changes to reproduce this:
FROM debian:12
RUN export DEBIAN_FRONTEND=noninteractive && \
apt-get update && \
apt-get install -yq curl jq
COPY ./dist/chezmoi-nocgo_linux_arm64/chezmoi /usr/bin/chezmoi
SHELL ["/bin/sh", "-xc"]
RUN chezmoi --version && \
chezmoi execute-template 'chezmoi: {{ .chezmoi.hostname }}' && \
echo 'hostname: '$(hostname)
If you're on x86 you'd need to use chezmoi-nocgo_linux_amd64/chezmoi
after having built it with goreleaser
. This requires both a commit and a tag (I used v2.4.1-alpha.0
) because goreleaser
will not run with a dirty index.
You can apply this patch to the chezmoi code:
971e03fda9b6 2023-11-27 | investigate-hostname (HEAD -> investigate, tag: v2.42.1-alpha.0) [Austin Ziegler]
diff --git a/internal/chezmoi/chezmoi.go b/internal/chezmoi/chezmoi.go
index 21422a38fe33..97d1e2cca38b 100644
--- a/internal/chezmoi/chezmoi.go
+++ b/internal/chezmoi/chezmoi.go
@@ -136,6 +136,8 @@ func FQDNHostname(fileSystem vfs.FS) (string, error) {
// First, try os.Hostname. If it returns something that looks like a FQDN
// hostname, or we're on Windows, return it.
osHostname, err := os.Hostname()
+ fmt.Printf("os.Hostname() = %#v\n", osHostname)
+
if runtime.GOOS == "windows" || (err == nil && strings.Contains(osHostname, ".")) {
return osHostname, err
}
@@ -148,11 +150,17 @@ func FQDNHostname(fileSystem vfs.FS) (string, error) {
}
}
+ v_etcHostsFQDNHostname, err := etcHostsFQDNHostname(fileSystem)
+ fmt.Printf("etcHostsFQDNHostname = %#v\n", v_etcHostsFQDNHostname)
+
// Otherwise, try /etc/hosts.
if fqdnHostname, err := etcHostsFQDNHostname(fileSystem); err == nil && fqdnHostname != "" {
return fqdnHostname, nil
}
+ v_etcHostnameFQDNHostname, err := etcHostnameFQDNHostname(fileSystem)
+ fmt.Printf("etcHostFQDNHostname = %#v\n", v_etcHostnameFQDNHostname)
+
// Otherwise, try /etc/hostname.
if fqdnHostname, err := etcHostnameFQDNHostname(fileSystem); err == nil && fqdnHostname != "" {
return fqdnHostname, nil
If you're on x86 you'd need to use
chezmoi-nocgo_linux_amd64/chezmoi
after having built it withgoreleaser
. This requires both a commit and a tag (I usedv2.4.1-alpha.0
) becausegoreleaser
will not run with a dirty index.
Note that you don't have to use goreleaser. You can also cross-compile chezmoi for linux/amd64 using:
GOOS=linux GOARCH=amd64 go build .
Hopefully this is now resolved. Please re-open if needed.
Describe the bug
The hostname obtained by using 'chezmoi data' in container b is always
debuerreotype
, while the output of the commandhostname
executed in the container isbuildkitsandbox
To reproduce
Execute in bash using the following command
Expected behavior
In the following command, build arg BUILDKIT_SANDBOX_HOSTNAME was used. After reexecuting, the output of command
hostname
was the same asBUILDKIT_SANDBOX_HOSTNAME=chezmoi-test
, but the output of commandchezmoi data
was stilldebuerreotype
, which was not as expectedOutput of command with the
--verbose
flagOutput of
chezmoi doctor
Additional context
docker version info: