Closed eku closed 3 years ago
The other ticket on lighttpd issues for Fedora/Centos were because they added an extra mount and it messed up the /var/run folder. This looks like /usr is having problems instead of /var but I'm still curious about your volume mounts. What does this command look like:
docker inspect -f '{{json .Mounts}}' pihole | jq
You don't have to pipe it to JQ but it makes it look much prettier if you do.
For reference here is what mine looks like and what docker-compose.yaml volume mounts are:
volumes:
- ./pihole.log:/var/log/pihole.log
- ./etc-pihole/:/etc/pihole/
- ./etc-dnsmasqd/:/etc/dnsmasq.d/
...
$ docker inspect -f '{{json .Mounts}}' pihole | jq
[
{
"Type": "bind",
"Source": "/home/a/projects/LAN_helpers/pihole.log",
"Destination": "/var/log/pihole.log",
"Mode": "rw",
"RW": true,
"Propagation": "rprivate"
},
{
"Type": "bind",
"Source": "/home/a/projects/LAN_helpers/etc-dnsmasqd",
"Destination": "/etc/dnsmasq.d",
"Mode": "rw",
"RW": true,
"Propagation": "rprivate"
},
{
"Type": "bind",
"Source": "/home/a/projects/LAN_helpers/etc-pihole",
"Destination": "/etc/pihole",
"Mode": "rw",
"RW": true,
"Propagation": "rprivate"
}
]
@diginc the mapping works
{
"Type": "bind",
"Source": "/var/lib/pihole/pihole",
"Destination": "/etc/pihole",
"Mode": "Z",
"RW": true,
"Propagation": "rprivate"
},
{
"Type": "bind",
"Source": "/var/lib/pihole/dnsmasq.d",
"Destination": "/etc/dnsmasq.d",
"Mode": "Z",
"RW": true,
"Propagation": "rprivate"
}
]
The question is, why a self-built image from Dockerfile_debian_amd64
does not start lighttpd
.
Please take a closer look to the logging supplied in my first post. How did you built the official image docker.io/diginc/pi-hole:latest
? Could it be that the base debian image changed in a way, that mod_fastcgi
fails?
The build is automatic by docker cloud service and there are tests to ensure fastcgi is working which run separately on travis-ci. Take a look at the checks on the most recent pull requests, they're continuously building the latest Dockerfile_debian_amd64 by running tox
, which coordinates a bunch of python to build the docker file and image. https://github.com/diginc/docker-pi-hole/blob/master/TESTING.md
Try running the tests locally to try and see if they are failing. If they don't there's some condition you're running into when starting the image that the tests may not cover. If they do error we've got some weird difference in Fedora again potentially, similar to the other fedora issue I was referring to, which is here https://github.com/diginc/docker-pi-hole/issues/217
Are you customizing the dockerfile or just reproducing a stock build?
If you're trying to modify the docker image may I suggest instead of rebuilding the entire thing you inherit form the known good image built by docker cloud? Make a new dockerfile project with FROM diginc/pi-hole:debian
and add any customizations you'd like on top of that.
@diginc yes, I wanted to optimize your Dockerfile
, because it wastes space by using to many layers (https://docs.docker.com/develop/develop-images/dockerfile_best-practices/). The result did not start lighttp. So I just tried to rebuild the original image by using an unmodified Dockerfile from this project. But again, this fails with the same problem.
I'm familiar with best practices and optimizing Dockerfiles. How were you thinking changing it to improve size?
Keep in mind some things were intentionally put at the end of the Dockerfile in order to make rebuilding faster and moving them higher or merging the commands won't necessarily make the build smaller, but it will make the RE-build (development cycles) take significantly longer. Large intensive steps which do no change often are as high as possible in the dockerfile so they get cached, small often changing steps are low in the file.
Taking a look at docker history --no-trunc diginc/pi-hole:debian
I don't see any greater than 1MB layers outside the FROM debian and main install:
/bin/sh -c apt-get update && apt-get install -y wget curl net-tools cron procps && curl -L -s $S6OVERLAY_RELEASE | tar xvzf - -C / && docker-install.sh && rm -rf /var/cache/apt/archives /var/lib/apt/lists/* && mv /init /s6-init 263MB
@diginc I did as you suggested
# python Dockerfile.py -v --arch=amd64
and the same problem. lighttpd
fails to start.
Directory /var/run/lighttpd
exists.
018-03-13 08:14:08: (mod_fastcgi.c.1478) invalid "bin-path" => "/usr/bin/php-cgi" (check that file exists, is regular file, and is executable by lighttpd)
2018-03-13 08:14:08: (mod_fastcgi.c.1159) the fastcgi-backend /usr/bin/php-cgi failed to start:
2018-03-13 08:14:08: (mod_fastcgi.c.1163) child exited with status 13 /usr/bin/php-cgi
2018-03-13 08:14:08: (mod_fastcgi.c.1166) If you're trying to run your app as a FastCGI backend, make sure you're using the FastCGI-enabled version.\nIf this is PHP on Gentoo, add 'fastcgi' to the USE flags.
2018-03-13 08:14:08: (mod_fastcgi.c.1518) [ERROR]: spawning fcgi failed.
2018-03-13 08:14:08: (server.c.1269) Configuration of plugins failed. Going down.
root@d048bba107ef:/var/log/lighttpd# ls -l /usr/bin/php-cgi
lrwxrwxrwx. 1 root root 25 Mar 13 07:57 /usr/bin/php-cgi -> /etc/alternatives/php-cgi
root@d048bba107ef:/var/log/lighttpd# ls -l /etc/alternatives/php-cgi
lrwxrwxrwx. 1 root root 19 Mar 13 07:57 /etc/alternatives/php-cgi -> /usr/bin/php-cgi7.0
root@d048bba107ef:/var/log/lighttpd# ls -l /usr/bin/php-cgi7.0
-rwxr-xr-x. 1 root root 4283768 Jan 5 13:51 /usr/bin/php-cgi7.0
root@d048bba107ef:/var/log/lighttpd# ldd /usr/bin/php-cgi7.0
linux-vdso.so.1 (0x00007ffc6c1ad000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f95a15c8000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f95a13ae000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f95a113b000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f95a0e37000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f95a0c33000)
libxml2.so.2 => /usr/lib/x86_64-linux-gnu/libxml2.so.2 (0x00007f95a0878000)
libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f95a060c000)
libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f95a0179000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f959fdda000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f959fbbd000)
/lib64/ld-linux-x86-64.so.2 (0x00007f95a1e10000)
libicui18n.so.57 => /usr/lib/x86_64-linux-gnu/libicui18n.so.57 (0x00007f959f743000)
libicuuc.so.57 => /usr/lib/x86_64-linux-gnu/libicuuc.so.57 (0x00007f959f39b000)
libicudata.so.57 => /usr/lib/x86_64-linux-gnu/libicudata.so.57 (0x00007f959d91e000)
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f959d6f8000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f959d376000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f959d15f000)
Looks fine to me.
Could you push any changes you've made to your fork (https://github.com/eku/docker-pi-hole) so I can try to reproduce your failure?
@diginc well, as I wrote it even fails without any changes. I'm wondering what could influence the image creation. Host system? Docker version?
I started to analyze the startup of lighttpd
with ltrace
and strace
, but failed to localize the root cause for the abort.
I don't know if it related with this error, but since the last updated I made (docker pull), the lighttpd server can't start, due to this error:
2018-03-20 17:30:15: (configfile.c.1154) source: /etc/lighttpd/lighttpd.conf line: 39 pos: 1 parser failed somehow near here: 53
I'm using the stock image without modifying anything
@AdriVillaB sounds like a configuration issue in lighttpd.conf, check line number 39
@eku I'm guessing Fedora is the issue here, especially if docker is not coming straight from an official docker inc source. Could you try a newer version than 1.13.1 like docker-ci 2017-2018? https://docs.docker.com/install/linux/docker-ce/fedora/#install-using-the-repository
Here's my LDD output and the versions from the latest version of php packages:
root@6ccf4c1c4917:/# ldd /usr/bin/php-cgi7.0
linux-vdso.so.1 (0x00007fff4285f000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f86f79b3000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f86f7799000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f86f7526000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f86f7222000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f86f701e000)
libxml2.so.2 => /usr/lib/x86_64-linux-gnu/libxml2.so.2 (0x00007f86f6c61000)
libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f86f69f5000)
libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f86f6562000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f86f61c3000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f86f5fa6000)
/lib64/ld-linux-x86-64.so.2 (0x0000560d427a6000)
libicui18n.so.57 => /usr/lib/x86_64-linux-gnu/libicui18n.so.57 (0x00007f86f5b2c000)
libicuuc.so.57 => /usr/lib/x86_64-linux-gnu/libicuuc.so.57 (0x00007f86f5782000)
libicudata.so.57 => /usr/lib/x86_64-linux-gnu/libicudata.so.57 (0x00007f86f3d05000)
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f86f3adf000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f86f375d000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f86f3546000)
root@6ccf4c1c4917:/# dpkg -l | grep php
ii php-cgi 1:7.0+49 all server-side, HTML-embedded scripting language (CGI binary) (default)
ii php-common 1:49 all Common files for PHP packages
ii php-sqlite3 1:7.0+49 all SQLite3 module for PHP [default]
ii php7.0-cgi 7.0.27-0+deb9u1 amd64 server-side, HTML-embedded scripting language (CGI binary)
ii php7.0-cli 7.0.27-0+deb9u1 amd64 command-line interpreter for the PHP scripting language
ii php7.0-common 7.0.27-0+deb9u1 amd64 documentation, examples and common module for PHP
ii php7.0-json 7.0.27-0+deb9u1 amd64 JSON module for PHP
ii php7.0-opcache 7.0.27-0+deb9u1 amd64 Zend OpCache module for PHP
ii php7.0-readline 7.0.27-0+deb9u1 amd64 readline module for PHP
ii php7.0-sqlite3 7.0.27-0+deb9u1 amd64 SQLite3 module for PHP
@diginc i didn't change the lighttp.conf file before or after the update. anyway, this is the part of the configuration file that fails:
29 server.document-root = "/var/www/html"
30 server.error-handler-404 = "pihole/index.php"
31 server.upload-dirs = ( "/var/cache/lighttpd/uploads" )
32 server.errorlog = "/var/log/lighttpd/error.log"
33 server.pid-file = "/var/run/lighttpd.pid"
34 server.username = "www-data"
35 server.groupname = "www-data"
36 server.bind = "10.66.0.60"
37 server.port = 80
38 $SERVER["socket"] == "127.0.0.1:80" {}53
39 accesslog.filename = "/var/log/lighttpd/access.log"
40 accesslog.format = "%{%s}t|%V|%r|%s|%b"
41
42 index-file.names = ( "index.php", "index.html", "index.lighttpd.html" )
43 url.access-deny = ( "~", ".inc", ".md", ".yml", ".ini" )
44 static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )
The line 39 is the configuration of the logfile, and i think that the error is on the line 38. Could be an error on the bash_functions file?
@AdriVillaB sounds like a different issue, could you fill out a new issue with the information the new template requests?
oki doki @diginc :)
@diginc unfortunately same problem with Docker version 17.12.1-ce, build 7390fc6
2018-03-21 15:31:00: (log.c.217) server started
2018-03-21 15:31:00: (mod_fastcgi.c.1478) invalid "bin-path" => "/usr/bin/php-cgi" (check that file exists, is regular file, and is executable by lighttpd)
2018-03-21 15:31:00: (mod_fastcgi.c.1159) the fastcgi-backend /usr/bin/php-cgi failed to start:
2018-03-21 15:31:00: (mod_fastcgi.c.1163) child exited with status 13 /usr/bin/php-cgi
2018-03-21 15:31:00: (mod_fastcgi.c.1166) If you're trying to run your app as a FastCGI backend, make sure you're using the FastCGI-enabled version.\nIf this is PHP on Gentoo, add 'fastcgi' to the USE flags.
2018-03-21 15:31:00: (mod_fastcgi.c.1518) [ERROR]: spawning fcgi failed.
2018-03-21 15:31:00: (server.c.1269) Configuration of plugins failed. Going down.
Any ideas what I could try out?
Did you compare the dpkg -l | grep php
output to mine to see if there are any differences?
I may need to try this out on a Fedora VM to see if I have the same problem.
Can you share your docker run command, even if it's a default one, just so I can have all the details for reproducing on fedora.
What i'll probably do when I get fedora setup is run the container with out the init system and manually run the Lighttpd and or php-cgi in the foreground with as much debug+verbose options as possible and like you mentioned, strace it and such...
This stops the s6 init system from spamming restarts of the failing lighttpd service (it acts like a supervisord) which may or may not be hurting progress to debugging.
example: docker run -it --name piholetest -e ServerIP=0.0.0.0 -e PH_VERBOSE=1 --entrypoint=bash diginc/pi-hole:latest
Also check these:
root@af30473f1cd5:/# ls -lat /usr/lib/lighttpd/ | grep cgi
-rw-r--r-- 1 root root 26464 Jan 14 2017 mod_cgi.so
-rw-r--r-- 1 root root 51544 Jan 14 2017 mod_fastcgi.so
-rw-r--r-- 1 root root 43304 Jan 14 2017 mod_scgi.so
root@af30473f1cd5:/# lighttpd -tt -f /etc/lighttpd/lighttpd.conf
root@af30473f1cd5:/# echo $?
0
Edit another idea:
php-cgi /var/www/html/admin/settings.php
should output an HTML page and Direct access to this script is forbidden!
at the end if PHP CLI is working.@diginc here we go
# dpkg -l | grep php
ii php-cgi 1:7.0+49 all server-side, HTML-embedded scripting language (CGI binary) (default)
ii php-common 1:49 all Common files for PHP packages
ii php-sqlite3 1:7.0+49 all SQLite3 module for PHP [default]
ii php7.0-cgi 7.0.27-0+deb9u1 amd64 server-side, HTML-embedded scripting language (CGI binary)
ii php7.0-cli 7.0.27-0+deb9u1 amd64 command-line interpreter for the PHP scripting language
ii php7.0-common 7.0.27-0+deb9u1 amd64 documentation, examples and common module for PHP
ii php7.0-json 7.0.27-0+deb9u1 amd64 JSON module for PHP
ii php7.0-opcache 7.0.27-0+deb9u1 amd64 Zend OpCache module for PHP
ii php7.0-readline 7.0.27-0+deb9u1 amd64 readline module for PHP
ii php7.0-sqlite3 7.0.27-0+deb9u1 amd64 SQLite3 module for PHP
# ls -lat /usr/lib/lighttpd/ | grep cgi
-rw-r--r--. 1 root root 26464 Jan 14 2017 mod_cgi.so
-rw-r--r--. 1 root root 51544 Jan 14 2017 mod_fastcgi.so
-rw-r--r--. 1 root root 43304 Jan 14 2017 mod_scgi.so
# lighttpd -tt -f /etc/lighttpd/lighttpd.conf
2018-03-22 16:52:06: (mod_fastcgi.c.1478) invalid "bin-path" => "/usr/bin/php-cgi" (check that file exists, is regular file, and is executable by lighttpd)
# php-cgi /var/www/html/admin/settings.php
outputs a HTML page and Direct access to this script is forbidden!
at the end .
My docker run command is
DOCKER_TAG=eku/pi-hole:latest
DOCKER_CONFIGS=/var/lib/pihole
DOCKER_SERVERIP=192.168.88.2
DOCKER_DNS1=85.214.20.141
DOCKER_DNS2=213.73.91.35
DOCKER_VHOST=pihole.docker.zx-spectrum
docker run -d \
--name pihole \
-v "${DOCKER_CONFIGS}/pihole/:/etc/pihole/:Z" \
-v "${DOCKER_CONFIGS}/dnsmasq.d/:/etc/dnsmasq.d/:Z" \
-e IPv6="false" -e ServerIP=$(DOCKER_SERVERIP) \
-e DNS1=$(DOCKER_DNS1) -e DNS2=$(DOCKER_DNS2) \
-e VIRTUAL_HOST=$(DOCKER_VHOST) \
$(DOCKER_TAG)
Needless to say, that this command works fine with docker.io/diginc/pi-hole:latest
.
@diginc I recently upgraded from FC27 to FC28, but problem still persists.
@diginc I hope it's okay to add on to this thread. I have a similar issue with lighttpd not starting after a fresh install. I have a relatively new raspian install. This is repeatable with each image pull.
pi@raspberrypi:~ $ ./docker_run.sh
RTNETLINK answers: Network is unreachable
### Make sure your IPs are correct, hard code ServerIP ENV VARs if necessary\nIP: 192.168.40.188\nIPv6:
Unable to find image 'pihole/pihole:v4.0_armhf' locally
v4.0_armhf: Pulling from pihole/pihole
a892dc37f295: Pull complete
57fb22af2576: Pull complete
421170bac0f0: Pull complete
9fc3b46571d4: Pull complete
e1d361690d98: Pull complete
38aa6a550dc6: Pull complete
8a5e6608572b: Pull complete
732c5f0c08bb: Pull complete
Digest: sha256:dba4c40348735bf876410b9a24b037d52d4d35766ae1d8c0f464f2272f5c81ca
Status: Downloaded newer image for pihole/pihole:v4.0_armhf
100c8289680eaa8c8e21069d48a616834592b9dfb10bb71b1bd2e72408098bf2
Your password for https://192.168.40.188/admin/ is pi@raspberrypi:~ $ docker exec -it 100c bash
root@100c8289680e:/# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.2 0.0 192 4 ? Ss 01:36 0:00 s6-svscan -t0 /var/run/s6/services
root 29 0.0 0.0 180 8 ? S 01:36 0:00 foreground if /etc/s6/init/init-stage2-redirfd foreground i
root 30 0.0 0.0 192 4 ? S 01:36 0:00 s6-supervise s6-fdholderd
root 37 0.0 0.0 180 8 ? S 01:36 0:00 if /etc/s6/init/init-stage2-redirfd foreground if if s6-
root 38 0.0 0.0 180 12 ? S 01:36 0:00 foreground if if s6-echo -n -- [s6-init] making user
root 43 0.0 0.0 176 4 ? S 01:36 0:00 if if -t s6-test -d /var/run/s6/etc/cont-init.d if s6-
root 212 0.0 0.0 176 4 ? S 01:36 0:00 if pipeline s6-ls -0 -- /var/run/s6/etc/cont-init.d pip
root 215 0.0 0.0 192 4 ? S 01:36 0:00 forstdin -o 0 -0 -- i importas -u i i if s6-echo -- [cont-init.d
root 216 0.0 0.0 0 0 ? Z 01:36 0:00 [s6-ls] <defunct>
root 217 0.0 0.0 0 0 ? Z 01:36 0:00 [s6-sort] <defunct>
root 218 0.0 0.0 176 4 ? S 01:36 0:00 foreground /var/run/s6/etc/cont-init.d/20-start.sh importas -u ?
root 220 0.0 0.1 2732 1868 ? S 01:36 0:00 bash /var/run/s6/etc/cont-init.d/20-start.sh
root 334 16.3 2.8 54672 28800 ? S 01:36 0:00 pihole-FTL test
root 349 0.5 0.2 2968 2264 ? S 01:36 0:00 bash /opt/pihole/gravity.sh
root 363 2.0 0.2 2864 2268 pts/0 Ss 01:37 0:00 bash
root 370 0.0 0.0 1372 324 ? S 01:37 0:00 sleep 1
root 371 0.0 0.1 5052 2000 pts/0 R+ 01:37 0:00 ps aux
root@100c8289680e:/# ls -lat /usr/lib/lighttpd/ | grep cgi
-rw-r--r-- 1 root root 17792 Jan 14 2017 mod_cgi.so
-rw-r--r-- 1 root root 38576 Jan 14 2017 mod_fastcgi.so
-rw-r--r-- 1 root root 30352 Jan 14 2017 mod_scgi.so
root@100c8289680e:/# php-cgi /var/www/html/admin/settings.php
...
<div class="content-wrapper">
<!-- Main content -->
<section class="content">
Direct access to this script is forbidden!
pi@raspberrypi:~ $ cat docker_run.sh
#!/bin/bash
# Lookups may not work for VPN / tun0
IP_LOOKUP="$(ip route get 8.8.8.8 | awk '{ print $NF; exit }')"
IPv6_LOOKUP="$(ip -6 route get 2001:4860:4860::8888 | awk '{for(i=1;i<=NF;i++) if ($i=="src") print $(i+1)}')"
# Just hard code these to your docker server's LAN IP if lookups aren't working
IP="${IP:-$IP_LOOKUP}" # use $IP, if set, otherwise IP_LOOKUP
IPv6="${IPv6:-$IPv6_LOOKUP}" # use $IPv6, if set, otherwise IP_LOOKUP
# Default of directory you run this from, update to where ever.
DOCKER_CONFIGS="$(pwd)"
echo "### Make sure your IPs are correct, hard code ServerIP ENV VARs if necessary\nIP: ${IP}\nIPv6: ${IPv6}"
# Default ports + daemonized docker container
docker run -d \
--name pihole \
-p 53:53/tcp -p 53:53/udp \
-p 67:67/udp \
-p 80:80 \
-p 443:443 \
-v "${DOCKER_CONFIGS}/pihole/:/etc/pihole/" \
-v "${DOCKER_CONFIGS}/dnsmasq.d/:/etc/dnsmasq.d/" \
-e ServerIP="${IP}" \
-e ServerIPv6="${IPv6}" \
--restart=unless-stopped \
pihole/pihole:v4.0_armhf
echo -n "Your password for https://${IP}/admin/ is "
docker logs pihole 2> /dev/null | grep 'password:'
@overkill32 no, your problem is different. You use the original docker image. This issue is about buildung own docker image from the supplied Dockerfile
.
I've created a docker image based on
Dockerfile_debian_amd64
frommaster
d83a950a23338982c2c038214316cc5c5664984a . Thelighttpd
fails to start with errordocker.io/diginc/pi-hole:latest
works fine when started with the very same environment variables.System: Fedora 27 with Docker 1.13.1