doh_server fails to build

[willis936]

• DoTH-DNS version: 7.0.0
• Bash version: 5.0.3(1)-release (arm-unknown-linux-gnueabihf)
• Operating System: DietPi 7.0.2, Raspbian GNU/Linux 10 (buster), kernel 5.10.17-v7l+, bootloader: 1599135103, VL805: 00138a1

Description

Running dothdns fails to build doh_server.

I have also tried installing from source (version 7.0.2.dev1+ga5ce6eb) and observed the same behavior.

# dothdns run
SUCCESS:   Set environment variables in `.env` file.
WARNING:   No ['cert.crt', 'key.key'] files found. Dashboards, DoH and DoT need both a 'certificate' and a corresponding 'key'. If you have not set those files up on another way encryption will not work properly.
SUCCESS:   Set environment variables in `.env` file.
INFO:      Image for 'doh_server' not found.
INFO:      Compiling image for 'doh_server' for version 2.2.4. This may last a bit.
ERROR:     The build of 'doh_server' image raised an error: The command '/bin/sh -c set -x     && delgroup ping     && addgroup -g 8053 doh     && adduser -D -G doh -u 8053 doh     && apk add -q --no-cache --virtual .build-deps gcc git go musl-dev     && apk add -q --no-cache bash ca-certificates shadow su-exec tzdata     && go get github.com/m13253/dns-over-https/doh-server     && cp -r /go/bin/* /usr/local/bin     && apk del -q --purge .build-deps     && rm -rf /go /root/.cache/* /tmp/* /var/cache/apk/*' returned a non-zero code: 5
Aborted!

What I Did

Install DoTH-DNS via pip then run dothdns run.

[Cielquan]

Unfortunately the error msg does not give much info about the root cause, except that is has something to do with the RUN part in the Dockerfile for the DoH server.

Could you locate your DoTH-DNS installation and manipulate the Dockerfile, which should reside in <venv>/lib/pyhtonX.Y/site-packages/dothdns/container_configs/doh-docker/Dockerfile.

Please change the RUN entry

RUN \
    set -x \
    && delgroup ping \
    && addgroup -g 8053 doh \
    && adduser -D -G doh -u 8053 doh \
    && apk add -q --no-cache --virtual .build-deps gcc git go musl-dev \
    && apk add -q --no-cache bash ca-certificates shadow su-exec tzdata \
    && go get github.com/m13253/dns-over-https/doh-server \
    && cp -r /go/bin/* /usr/local/bin \
    && apk del -q --purge .build-deps \
    && rm -rf /go /root/.cache/* /tmp/* /var/cache/apk/*

with this:

RUN \
    set -x \
    && (delgroup ping || exit exit 11) \
    && (addgroup -g 8053 doh || exit exit 22) \
    && (adduser -D -G doh -u 8053 doh || exit exit 33) \
    && (apk add -q --no-cache --virtual .build-deps gcc git go musl-dev || exit exit 44) \
    && (apk add -q --no-cache bash ca-certificates shadow su-exec tzdata || exit exit 55) \
    && (go get github.com/m13253/dns-over-https/doh-server || exit exit 66) \
    && (cp -r /go/bin/* /usr/local/bin || exit 77) \
    && (apk del -q --purge .build-deps || exit 88) \
    && (rm -rf /go /root/.cache/* /tmp/* /var/cache/apk/* || exit 99)

Afterwards try running dothdns again. I added exit codes to each subcommand .. hopefully with the exit code you get then we can localize the erroring subcommand and see further to find a fix.

[willis936]

I’m new to containers, so pardon mistakes.

I couldn’t find a directory that had a site-packages directory in lib/python3.7, but I did find a Dockerfile in DoTH-DNS/doh-docker after running dothdns config.

I made the changes to the run command in that Dockerfile and ran dothdns run again.

The error code is now 2 instead of 5, but this doesn’t appear to be at any of your new exit codes.

# dothdns run
SUCCESS:   Set environment variables in `.env` file.
SUCCESS:   Set environment variables in `.env` file.
INFO:      Image for 'doh_server' not found.
INFO:      Compiling image for 'doh_server' for version 2.2.4. This may last a bit.
ERROR:     The build of 'doh_server' image raised an error: The command '/bin/sh -c set -x     && (delgroup ping || exit exit 11)     && (addgroup -g 8053 doh || exit exit 22)     && (adduser -D -G doh -u 8053 doh || exit exit 33)     && (apk add -q --no-cache --virtual .build-deps gcc git go musl-dev || exit exit 44)     && (apk add -q --no-cache bash ca-certificates shadow su-exec tzdata || exit exit 55)     && (go get github.com/m13253/dns-over-https/doh-server || exit exit 66)     && (cp -r /go/bin/* /usr/local/bin || exit 77)     && (apk del -q --purge .build-deps || exit 88)     && (rm -rf /go /root/.cache/* /tmp/* /var/cache/apk/* || exit 99)' returned a non-zero code: 2
Aborted!

Update: I also tried changing set -x to (set -x || exit exit 12) but still got exit code 2.

EDIT: If anyone reads this in the future and doesn't know how to find their site-packages directory: run this command pip list -v

Mine ended up being /usr/local/lib/python3.7/dist-packages.

[Cielquan]

I also only worked a bit with docker, so I only know some basics. But for this container I never had any issues regardless of the system I used.

Could you again change the RUN part in the Dockerfile to this:

RUN set -x
RUN delgroup ping
RUN addgroup -g 8053 doh
RUN adduser -D -G doh -u 8053 doh
RUN apk add -q --no-cache --virtual .build-deps gcc git go musl-dev
RUN apk add -q --no-cache bash ca-certificates shadow su-exec tzdata
RUN go get github.com/m13253/dns-over-https/doh-server
RUN cp -r /go/bin/* /usr/local/bin
RUN apk del -q --purge .build-deps
RUN rm -rf /go /root/.cache/* /tmp/* /var/cache/apk/*

This split should hopefully let us identify the culprit command.

set -x should not fail. It is just a setting for bash to print the commands used afterwards. You can use this e.g. for debugging in bash scripts.

[willis936]

Progress!

It appears to fail on the apk add line. I'm not sure if apk needs to be installed in the host system, but running apk in the root shell returns -bash: apk: command not found. I'm on a debian-based distro that uses apt as the package manager.

# dothdns run
SUCCESS:   Set environment variables in `.env` file.
SUCCESS:   Set environment variables in `.env` file.
INFO:      Image for 'doh_server' not found.
INFO:      Compiling image for 'doh_server' for version 2.2.4. This may last a bit.
ERROR:     The build of 'doh_server' image raised an error: The command '/bin/sh -c apk add -q --no-cache --virtual .build-deps gcc git go musl-dev' returned a non-zero code: 5
Aborted!

[Cielquan]

Nice!

Containers only need the host's kernel (to my knowledge). So software is not transferred from the host into the container. Therefore the apk command is there to install the dependencies into the container image.

By convention exit code 127 is "command not found" so apk should be available. But cannot find something on exit code 5 for apk.

I'll try some things when I'm back home.

[Cielquan]

So .. I tried dothdns run on my linux mint 19.3 VM and it build the doh_server image without problem.

As it seems to be a problem with docker and not doth-dns lets skip doth-dns for now and work directly with docker. Could you cd into the directory where the Dockerfile is and run: docker build . Without doth-dns and the underlying docker lib for python the output will be much more verbose and hopefully give as more clues.

[willis936]

I think I read somewhere that someone who had a similar issue fixed it by changing their DNS resolver. My pi's upstream DNS server right now is another pi-hole. I haven't run into issues with filter lists filtering repos before, but perhaps that is happening here.

# docker --version
Docker version 20.10.5, build 55c4c88

# docker build .
Sending build context to Docker daemon  6.144kB
Step 1/18 : FROM alpine
 ---> 9c74a18b2325
Step 2/18 : ARG BUILD_DATE
 ---> Using cache
 ---> eead8807458d
Step 3/18 : ARG VERSION
 ---> Using cache
 ---> 2822bb03d93f
Step 4/18 : LABEL     org.label-schema.vendor="Cielquan - cielquan@protonmail.com"     org.label-schema.url="https://github.com/Cielquan/DoTH-DNS/"     org.label-schema.name="DoH Server"     org.label-schema.version=$VERSION     org.label-schema.build-date=$BUILD_DATE     org.label-schema.license="GPL-3.0"     org.label-schema.schema-version="1.0"
 ---> Running in 0c8cea81b888
Removing intermediate container 0c8cea81b888
 ---> 0e3dca947028
Step 5/18 : ENV     GOPATH="/go"     GOCACHE="/tmp/gocache"
 ---> Running in 61d9200d2c24
Removing intermediate container 61d9200d2c24
 ---> a8c0d5f926a3
Step 6/18 : WORKDIR /opt/dns-over-https
 ---> Running in ed655c8a475f
Removing intermediate container ed655c8a475f
 ---> 2df7a501e79b
Step 7/18 : COPY configs/ conf/
 ---> 4219715e0758
Step 8/18 : RUN set -x
 ---> Running in 25487a8381a0
Removing intermediate container 25487a8381a0
 ---> d190ce0042a4
Step 9/18 : RUN delgroup ping
 ---> Running in b856e5307aba
Removing intermediate container b856e5307aba
 ---> ba099c1ae032
Step 10/18 : RUN addgroup -g 8053 doh
 ---> Running in ede7b3d27915
Removing intermediate container ede7b3d27915
 ---> d3a436b7e5ad
Step 11/18 : RUN adduser -D -G doh -u 8053 doh
 ---> Running in 4a60898246a9
Removing intermediate container 4a60898246a9
 ---> 33278237b9dc
Step 12/18 : RUN apk add -q --no-cache --virtual .build-deps gcc git go musl-dev
 ---> Running in ca92a92f4b37
ERROR: unable to select packages:
  gcc (no such package):
    required by: .build-deps-19700101.000000[gcc]
  git (no such package):
    required by: .build-deps-19700101.000000[git]
  go (no such package):
    required by: .build-deps-19700101.000000[go]
  musl-dev (no such package):
    required by: .build-deps-19700101.000000[musl-dev]
The command '/bin/sh -c apk add -q --no-cache --virtual .build-deps gcc git go musl-dev' returned a non-zero code: 5

[Cielquan]

Yeah something like connectivity or filtering would have been my next guess according to the log above.

I'm curios if the pihole is the issue here.

[willis936]

I disabled DNS filtering the upstream pi-hole and observed the same results. I added RUN apk update just above the first apk add in case the repo list needed to be initialized and got this error:

Step 12/19 : RUN apk update
 ---> Running in 0a815477021a
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/main/armv7/APKINDEX.tar.gz
ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.13/main: temporary error (try again later)
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.13/main: No such file or directory
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/community/armv7/APKINDEX.tar.gz
ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.13/community: temporary error (try again later)
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.13/community: No such file or directory
2 errors; 14 distinct packages available
The command '/bin/sh -c apk update' returned a non-zero code: 2

I might try a fresh distro install tonight.

Update: I've also tried uncommenting DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4" in \etc\default\docker, rebooting, and trying to build again, with the same results (both with and without the RUN apk update line).

[Cielquan]

I tried the URLs mentioned in the output and all work when I copy them into my browser. So for me the No such file or directory error` hints towards some connectivity problem of some sorts.

Could you confirm that the URLs work for you too in a browser or with curl?

[willis936]

I am able to curl from the machine that is failing to builder doh server, even with DNS filtering enabled on the upstream pi-hole. I will try a fresh docker install and if that fails then I will try a fresh distro.

# curl https://dl-cdn.alpinelinux.org/alpine/v3.13/main/armv7/APKINDEX.tar.gz --output APKINDEX.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  609k  100  609k    0     0  1507k      0 --:--:-- --:--:-- --:--:-- 1507k

Update: same issue with a full removal of docker and docker-compose (via dietpi-software), then reinstall of docker.

[Cielquan]

I found this issue: https://github.com/gliderlabs/docker-alpine/issues/334

Maybe docker build . --network=host works. If it does doth-dns should find the pre-built image and proceed with it.

[willis936]

I've been playing around with build options (including setting the network to host during disabled dns filtering) but haven't observed any change in behavior. I'm not sure how to get docker network debug output yet.

Update: I did a fresh install of dietpi and am still observing the same behavior. I feel like this might be an issue for an upstream repo but I'm not sure which.

I'm going to try a good old-fashioned kernel compile then call it a day. I'm not sure what to try next.

Update: the kernel compile (commit: https://github.com/raspberrypi/linux/commit/13c43880f2ee6db7d26949cb6a3e8db1a4b76736) didn't change the behavior. I have noticed that some of my scripts do not work the way I expect them to in a debian OS. I'm wondering if there is something up with dietpi's root shell. I will try a raspbian lite install tomorrow.

Update: I have tried removing docker via the dietpi-software interface and instead installing it via get-docker.sh. I also tried running as a non-root user that was added to the docker user group. Same behavior in both cases.

[willis936]

I ran dothdns run on another raspberry pi 4 I have with an existing full raspberry pi os image. I installed docker via the curl link and dothdns via the pip compile command. I did notice that I had to change the command from pip to pip3 to get the script to use python 3.7.3 instead of the also installed python 2.7.16.

When I ran dothdns run (or just docker build . in the doh server directory) I get the same error. I don’t think this is a dietpi specific issue.

[Cielquan]

I tested it on my RasPi4 also:

I used the official Raspberry Pi OS Lite (Release date: January 11th 2021 Kernel version: 5.4) on my RasPi 4B 4GB. After boot I did via ssh:

# Upgrade system
$ sudo apt update && sudo apt upgrade -y
# Install docker and check version
$ sudo apt install docker.io -y
$ docker -v
Docker version 18.09.1, build 4c52b90
# Add user to docker group to run docker command without sudo
$ sudo usermod -aG docker pi
# Reboot to apply group change
$ sudo reboot
# Check if docker daemon runs
$ docker ps
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
# I needed to start docker manually as the daemon was not running after the reboot
$ sudo systemctl start docker
# Check python3 version
$ python3 --version
Python 3.7.3
# Install python3-venv (which not always comes with python by default)
$ sudo apt install python3-venv -y
# Create python virtual environment
$ python3 -m venv .venv
# Activate the venv
$ source .venv/bin/activate
# Install DoTH-DNS (I don't need pip3 here explicitly as I use a venv and the pip version
# there will be used which will be pip3 because I  used python3 for the venv creation above)
$ pip install doth-dns
# Run DoTH-DNS and get the same error
$ dothdns run
SUCCESS:   Set environment variables in `.env` file.
WARNING:   No ['cert.crt', 'key.key'] files found. Dashboards, DoH and DoT need both a 'certificate' and a corresponding 'key'. If you have not set those files up on another way encryption will not work properly.
SUCCESS:   Set environment variables in `.env` file.
INFO:      Image for 'doh_server' not found.
INFO:      Compiling image for 'doh_server' for version 2.2.4. This may last a bit.
ERROR:     The build of 'doh_server' image raised an error: The command '/bin/sh -c set -x     && delgroup ping     && addgroup -g 8053 doh     && adduser -D -G doh -u 8053 doh     && apk add -q --no-cache --virtual .build-deps gcc git go musl-dev     && apk add -q --no-cache bash ca-certificates shadow su-exec tzdata     && go get github.com/m13253/dns-over-https/doh-server     && cp -r /go/bin/* /usr/local/bin     && apk del -q --purge .build-deps     && rm -rf /go /root/.cache/* /tmp/* /var/cache/apk/*' returned a non-zero code: 5
Aborted!
# Change the Dockerfile
$ nano DoTH-DNS/doh-docker/Dockerfile
# Run again and see apk failing
$ dothdns run
SUCCESS:   Set environment variables in `.env` file.
WARNING:   No ['cert.crt', 'key.key'] files found. Dashboards, DoH and DoT need both a 'certificate' and a corresponding 'key'. If you have not set those files up on another way encryption will not work properly.
SUCCESS:   Set environment variables in `.env` file.
INFO:      Image for 'doh_server' not found.
INFO:      Compiling image for 'doh_server' for version 2.2.4. This may last a bit.
ERROR:     The build of 'doh_server' image raised an error: The command '/bin/sh -c apk add -q --no-cache --virtual .build-deps gcc git go musl-dev' returned a non-zero code: 5
Aborted!
# Run docker build for more output
$ docker build DoTH-DNS/doh-docker/
Sending build context to Docker daemon  6.656kB
Step 1/18 : FROM alpine
 ---> 9c74a18b2325
Step 2/18 : ARG BUILD_DATE
 ---> Using cache
 ---> f6771d7e8956
Step 3/18 : ARG VERSION
 ---> Using cache
 ---> 5b76f8dc59e1
Step 4/18 : LABEL     org.label-schema.vendor="Cielquan - cielquan@protonmail.com"     org.label-schema.url="https://github.com/Cielquan/DoTH-DNS/"     org.label-schema.name="DoH Server"     org.label-schema.version=$VERSION     org.label-schema.build-date=$BUILD_DATE     org.label-schema.license="GPL-3.0"     org.label-schema.schema-version="1.0"
 ---> Running in 77389d46ede4
Removing intermediate container 77389d46ede4
 ---> 83eafa454e96
Step 5/18 : ENV     GOPATH="/go"     GOCACHE="/tmp/gocache"
 ---> Running in 678b0ee25adf
Removing intermediate container 678b0ee25adf
 ---> 5eda845c0e92
Step 6/18 : WORKDIR /opt/dns-over-https
 ---> Running in 97917072f224
Removing intermediate container 97917072f224
 ---> aca188e759e2
Step 7/18 : COPY configs/ conf/
 ---> c522e52da0b3
Step 8/18 : RUN set -x
 ---> Running in 180ab5f6666a
Removing intermediate container 180ab5f6666a
 ---> 43b889674008
Step 9/18 : RUN delgroup ping
 ---> Running in ab4f471e2603
Removing intermediate container ab4f471e2603
 ---> 26bdfc26137a
Step 10/18 : RUN addgroup -g 8053 doh
 ---> Running in d555ba3a7bc3
Removing intermediate container d555ba3a7bc3
 ---> 47a0e16b5e70
Step 11/18 : RUN adduser -D -G doh -u 8053 doh
 ---> Running in db722f50ab09
Removing intermediate container db722f50ab09
 ---> a727712f4f75
Step 12/18 : RUN apk add -q --no-cache --virtual .build-deps gcc git go musl-dev
 ---> Running in a5a64e9c23cf
ERROR: unable to select packages:  gcc (no such package):

    required by: .build-deps-19700101.000000[gcc]
  git (no such package):
    required by: .build-deps-19700101.000000[git]
  go (no such package):
    required by: .build-deps-19700101.000000[go]
  musl-dev (no such package):
    required by: .build-deps-19700101.000000[musl-dev]
The command '/bin/sh -c apk add -q --no-cache --virtual .build-deps gcc git go musl-dev' returned a non-zero code: 5

I then tested on my Linux Mint 20.1 VM (last time was 19.3) and the Image built just fine.

The difference are the hardware and OS.

I thought about moving the Dockerfile to ubuntu, but the transition is not that trivial as it seems because of the dependencies. I originally copied the Dockfile from https://github.com/goofball222/dns-over-https/blob/master/stable/Dockerfile and modified it to DoTH-DNS needs. If I remember correctly it was mostly simplification by removing unnecessary parts and changing the entrypoint.

In a last attempt I ran the same procedure above again on the again fresh RasPi 4 but changed the base image from FROM alpine to FROM alpine:3.12 in the Dockerfile prior running dothdns run the first time and it worked. Without a tag it defaults to FROM alpine:latest which is currently FROM alpine:3.13. (https://hub.docker.com/_/alpine/)

So there must be a problem with alpine 3.13 + arm hardware (?) + apk I guess?!

I guess changing the alpine version a feature release back could be a temporary solution.

What you could also test to see if maybe a patch version and not the 3.13 feature version is the culprit:

• FROM alpine:3.13.0
• FROM alpine:3.13.1
• FROM alpine:3.13.2 (which is latest)

[willis936]

I ran docker build . in the doh server directory on my raspberry pi 4 dietpi.

FROM alpine version: 3.12: succeeded (!!) 3.13.0: failed 3.13.1: failed 3.13.2: failed

It looks like this issue: https://github.com/dotnet/runtime/issues/47423#issuecomment-772573602

Thanks for the investigation.

[willis936]

When I run dothdns run the first time this is the output. The pihole web interface is not accessible via browser on a LAN client. Note that the shell waits 10 minutes before timing out on pihole setup.

SUCCESS:   Set environment variables in `.env` file.
SUCCESS:   Set environment variables in `.env` file.
INFO:      Image for 'doh_server' not found.
INFO:      Compiling image for 'doh_server' for version 2.2.4. This may last a bit.
SUCCESS:   New 'doh_server' image was build: <Image: 'cielquan/doh_server:latest'>
INFO:      Pulling image for 'mvance/unbound-rpi'. This may last a bit.
SUCCESS:   Pulled image for 'mvance/unbound-rpi'.
INFO:      Pulling image for 'pihole/pihole'. This may last a bit.
SUCCESS:   Pulled image for 'pihole/pihole'.
INFO:      Pulling image for 'traefik'. This may last a bit.
SUCCESS:   Pulled image for 'traefik'.
SUCCESS:   Created 'doth_dns_network' network.
INFO:      Using 'doth_dns_network' network.
INFO:      Starting 'traefik'.
INFO:      Starting 'doh_server'.
INFO:      Starting 'pihole'.
INFO:      Starting 'unbound'.
SUCCESS:   Boot of 'traefik' finished.
SUCCESS:   Boot of 'doh_server' finished.
WARNING:   Pihole blocklist setup exceeded 600 seconds. Please check 'pihole' logs for more information.
Aborted!

docker stats shows all 4 containers are running.

CONTAINER ID   NAME         CPU %     MEM USAGE / LIMIT   MEM %     NET I/O           BLOCK I/O     PIDS
f99b127478c5   unbound      4.55%     0B / 0B             0.00%     17.2MB / 95.8kB   0B / 19.6MB   1
c3ed34e44f3f   pihole       0.18%     0B / 0B             0.00%     1.23MB / 679kB    0B / 23.7MB   21
07dff7de9f72   doh_server   0.00%     0B / 0B             0.00%     1.27kB / 0B       0B / 0B       6
3633f8de6ea9   traefik      0.07%     0B / 0B             0.00%     75.6kB / 8.14kB   0B / 0B       11

Here are details of the images.

# docker images
REPOSITORY            TAG       IMAGE ID       CREATED          SIZE
cielquan/doh_server   latest    df10a860db19   39 minutes ago   17.3MB
alpine                3.12      f2988fc9fa05   3 weeks ago      3.77MB
pihole/pihole         latest    1b529812cd4b   4 weeks ago      329MB
mvance/unbound-rpi    latest    3a208ca6e27d   4 weeks ago      147MB
traefik               v2.1      fe7ab131d970   12 months ago    58.4MB

And docker info.

# docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)

Server:
 Containers: 4
  Running: 4
  Paused: 0
  Stopped: 0
 Images: 12
 Server Version: 20.10.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: journald
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc version: 12644e614e25b05da6fd08a38ffa0cfe1903fdec init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.23-v7l+
 Operating System: Raspbian GNU/Linux 10 (buster)
 OSType: linux
 Architecture: armv7l
 CPUs: 4
 Total Memory: 1.871GiB
 Name: DietPi
 ID: NQX2:IH5W:PXLE:4JQR:7DED:6JUT:5YKE:S7X3:HTUC:DF7U:IPGM:G75G
 Docker Root Dir: /mnt/dietpi_userdata/docker-data
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory TCP limit support
WARNING: No oom kill disable support

[Cielquan]

Could show me the result of docker log <container>. And replace <container> with the 4 container names.

You could also log into the container with docker exec -it <container> bash if you want to investigate inside the container. But I think not all 4 container are accessable via this method if I remember correctly

[willis936]

Thanks for the tips! Everything appears to be working correctly.

When I run a dig, the upstream pihole is still in use, so I modified /etc/resolv.conf to use the loopback IP address, rebooted, and ran another dig. It appears to be working correctly. I’m not sure how to access the pihole web interface though. I’ll look through documentation later to see what I’m missing.

The webpage error I get is “ERR_SSL_PROTOCOL_ERROR”. Chrome says the site “sent an invalid response”. I manually generate SSL with a script I wrote. My issue may be in improperly generating SSL files. I will try fresh without any DoTH-DNS config or SSL files later today.

This is the script I use to initialize DoTH-DNS and generate the SSL files.

# https://doth-dns.readthedocs.io/en/latest/

# directories / files
dTop=/root
dDoth=$dTop/DoTH-DNS
dCert=$dDoth/certificates

# install doth-dns
cd $dTop
#pip install git+https://github.com/Cielquan/DoTH-DNS.git
pip install doth-dns

# make base config
dothdns config

# patch alpine 3.13 not working on raspberry pi
sed -i 's/FROM alpine/FROM alpine:3.12/' $dDoth/doh-docker/Dockerfile

# generate certificate
apt install openssl
openssl genrsa -passout pass:x -out key.key 2048
openssl rsa -passin pass:x -in key.key -out $dCert/key.key
rm key.key
openssl req -new -key $dDoth/certificates/key.key -out $dCert/csr.csr
openssl x509 -req -days 365 -in $dCert/csr.csr -signkey $dCert/key.key -out $dCert/cert.crt

# traefik password
install apache2-utils
echo Set traefik password.
htpasswd -c $dDoth/traefik-docker/shared/.htpasswd root

cd $dTop
dothdns run

# dig github.com

; <<>> DiG 9.11.5-P4-5.1+deb10u3-Raspbian <<>> github.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 9167
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;github.com.                    IN      A

;; ANSWER SECTION:
github.com.             60      IN      A       140.82.113.3

;; Query time: 47 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Fri Mar 19 10:22:31 CDT 2021
;; MSG SIZE  rcvd: 55

# docker logs pihole
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...[fix-attrs.d] 01-resolver-resolv: applying... 
s6-chown: fatal: unable to chown /etc/resolv.conf: Read-only file system
s6-chmod: fatal: unable to change mode of /etc/resolv.conf: Read-only file system
[fix-attrs.d] 01-resolver-resolv: exited 0.
[fix-attrs.d] 02-chown-pihole-configs: applying... 
[fix-attrs.d] 02-chown-pihole-configs: exited 0.
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-conf-dnsmasq.sh: executing... 
[cont-init.d] 01-conf-dnsmasq.sh: exited 0.
[cont-init.d] 20-start.sh: executing... 
 ::: Starting docker specific checks & setup for docker pihole/pihole
Assigning random password: bg5IOq_o
  [i] Installing configs from /etc/.pihole...
  [i] Existing dnsmasq.conf found... it is not a Pi-hole file, leaving alone!
  [i] Copying 01-pihole.conf to /etc/dnsmasq.d/01-pihol  [✓] Copying 01-pihole.conf to /etc/dnsmasq.d/01-pihole.conf
Converting DNS1 to PIHOLE_DNS_
Setting DNS servers based on PIHOLE_DNS_ variable
Setting password: bg5IOq_o
+ pihole -a -p bg5IOq_o bg5IOq_o
  [✓] New password set
DNSMasq binding to default interface: eth0
Added ENV to php:
                        "PHP_ERROR_LOG" => "/var/log/lighttpd/error.log",
                        "ServerIP" => "10.111.222.3",
                        "VIRTUAL_HOST" => "10.111.222.3",
Using IPv4 and IPv6
::: setup_blocklists now setting default blocklists up: 
::: TIP: Use a docker volume for /etc/pihole/adlists.list if you want to customize for first boot
::: Blocklists (/etc/pihole/adlists.list) now set to:
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
::: Testing pihole-FTL DNS: FTL started!
::: Testing lighttpd config: Syntax OK
::: All config checks passed, cleared for startup ...
::: Enabling Query Logging
  [i] Enabling logging...
  [✓] Logging has been enabled!
 ::: Docker start setup complete
  [i] Creating new gravity database
  [i] Migrating content of /etc/pihole/adlists.list into new database
  [i] Neutrino emissions detected...
  [✓] Pulling blocklist source list into range
  [✓] Preparing new gravity database
  [i] Using libz compression
  [i] Target: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
  [✓] Status: Retrieval successful
  [i] Received 67402 domains
  [i] Storing downloaded domains in new gravity databas  [✓] Storing downloaded domains in new gravity database
  [✓] Building tree
  [✓] Swapping databases
  [i] Number of gravity domains: 67402 (67402 unique domains)
  [i] Number of exact blacklisted domains: 0
  [i] Number of regex blacklist filters: 0
  [i] Number of exact whitelisted domains: 0
  [i] Number of regex whitelist filters: 0
  [✓] Cleaning up stray matter
  [✓] DNS service is listening
     [✓] UDP (IPv4)
     [✓] TCP (IPv4)
     [✓] UDP (IPv6)
     [✓] TCP (IPv6)
  [i] Pi-hole blocking will be enabled
  [i] Enabling blocking
  [✓] Pi-hole Enabled
  Pi-hole version is v5.2.4 (Latest: ERROR)
  AdminLTE version is v5.4 (Latest: ERROR)
  FTL version is v5.7 (Latest: ERROR)
[cont-init.d] 20-start.sh: exited 0.
[cont-init.d] 21-conf-dns.sh: executing... 
[cont-init.d] 21-conf-dns.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
Starting pihole-FTL (no-daemon) as root
Starting lighttpd
Starting crond
[services.d] done.

# docker exec -it pihole bash
/# pihole status
  [✓] DNS service is listening
     [✓] UDP (IPv4)
     [✓] TCP (IPv4)
     [✓] UDP (IPv6)
     [✓] TCP (IPv6)

  [✓] Pi-hole blocking is enabled

[Cielquan]

from what I see in the script I think the certificate is missing a target/hostname .. don't have the correct word/phrase at hand .. for which it is valid

[willis936]

I fixed the ssl issue by changing the lines used to generate files.

openssl req -new -newkey RSA:2048 -nodes -keyout $dCert/key.key -out $dCert/csr.csr
openssl x509 -req -days 365 -in $dCert/csr.csr -signkey $dCert/key.key -out $dCert/cert.crt

These lines work.

I do have another issue: when going to the DoTH-DNS in a browser the response is a text file containing only 404 page not found. I have tried using both the DNS entry name and direct IP address. When ssl is used only the DNS entry returns a response (as expected). When ssl is not used, then the 404 page is returned in both cases (as expected).

I have tried going to both the root page and /admin (ie hostname.tld and hostname.tld/admin), both return the 404 page.

docker stats shows all 4 containers running. Opening a bash to pihole container and checking pihole status shows that everything is working normally. The DoTH-DNS machine resolves DNS entries via unbound and does serve DNS requests, but I do not know how to access the pihole interface.

[Cielquan]

This sounds more link an issue with the traefik proxy container, which takes all the traffic incoming on ports 80, 443 and 853. traefik then forwards the traffic according to the labels on the other containers. Therefore http requests are send to traefik first which will result in a forward to the same resource but with https. https requests are decrypted by traefik and forwarded as http request to pihole in the internal network. The same principle applies to DoT (853) and DoH (443) traffic.

I have traefik rules in place which ensure that /admin is present. Therefore it should be irrelevant if you type it yourself or not.

Could you share the docker logs traefik output?

[Cielquan]

I would like to move these additional issue to new ones, as the OP issue is solved.

EDIT: I marked the comments off-topic. @willis936 please see the last hidden comment for my answer.