NPM fails to start using Pihole dns in docker

[pittbull]

Describe the bug When starting docker app (2.7.2) using PiHole dns running in a separate container on the same host, the following is logged during startup:

app_1  | [2/18/2021] [2:03:37 PM] [Migrate  ] › ℹ  info      Current database version: 20201014143841
app_1  | [2/18/2021] [2:04:20 PM] [Global   ] › ✖  error     Command failed: pip3 install certbot-dns-cloudflare==1.8.0 cloudflare
app_1  | WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f9c8898ae20>: Failed to establish a new connection: [Errno -3] Try again')': /simple/certbot-dns-cloudflare/
app_1  | WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f9c8898ad60>: Failed to establish a new connection: [Errno -3] Try again')': /simple/certbot-dns-cloudflare/
app_1  | WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f9c8896a490>: Failed to establish a new connection: [Errno -3] Try again')': /simple/certbot-dns-cloudflare/
app_1  | WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f9c8896a340>: Failed to establish a new connection: [Errno -3] Try again')': /simple/certbot-dns-cloudflare/
app_1  | WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f9c8896a4c0>: Failed to establish a new connection: [Errno -3] Try again')': /simple/certbot-dns-cloudflare/
app_1  | ERROR: Could not find a version that satisfies the requirement certbot-dns-cloudflare==1.8.0 (from versions: none)
app_1  | ERROR: No matching distribution found for certbot-dns-cloudflare==1.8.0

If I add the 'dns' option to my docker-compose file and point it to 8.8.8.8 NPM loads without issues.

I have tried setting the same option to the various docker and physical ips, but they yield the same error.

I have several other containers, servers and clients that successfully use the Pihole container as their dns.

Any advise on this strange issue?

[Zerwin]

Both PiHole and Nginx use the port 80 and 443, how did you work around that ? I don't think you can have the same ports in use for 2 containers on 1 host.

[SteveGBuck]

@Zerwin, I use "expose" rather than "ports" in my pihole container config. Then let NPM proxy to pihole using the docker network. The only thing Im unsure of is that this is not blocking SSL based adverts which I'm looking at right now.

Here's my docker-compose.yml for pihole (but NPM uses the same external "proxy" network).

version: "3.2"

# More info at https://github.com/pi-hole/docker-pi-hole/ and https://docs.pi-hole.net/
services:
  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    ports:
      - 53:53/tcp
      - 53:53/udp
    expose:
      - 67/udp
      - 80/tcp
      - 443/tcp
    environment:
      TZ: 'Europe/London'
      WEBPASSWORD: '***********'
    # Volumes store your data between container upgrades
    volumes:
      - etc-pihole:/etc/pihole/
      - etc-dnsmasq.d:/etc/dnsmasq.d/
    # Recommended but not required (DHCP needs NET_ADMIN)
    #   https://github.com/pi-hole/docker-pi-hole#note-on-capabilities
    #cap_add:
    #  - NET_ADMIN
    restart: unless-stopped

volumes:
  etc-pihole:
  etc-dnsmasq.d:

networks:
  default:
    external:
      name: proxy

[Zerwin]

From the PiHole docker hub site:

Port 443 is to provide a sinkhole for ads that use SSL. If only port 80 is used, then blocked HTTPS queries will fail to connect to port 443 and may cause long loading times. Rejecting 443 on your firewall can also serve this same purpose. Ubuntu firewall example: sudo ufw reject https

---

As you are only exposing port 443 it's never available to anything outside NPM and PiHole. You need to properly publish port 443 for this to work.

Overall I would suggest moving away from exposing ports and putting PiHole on a different VM/Server. You can still use docker there if you like. Exposing ports like you are now is probably also what is causing the original problem.

[SteveGBuck]

@Zerwin I'm not the OP, but I have no issue starting up NPM and Pihole this way (i.e no port conflict) and at the moment Pihole looks like its working. But as you rightly point out Im not sure I've resolved the whole SSL sinkhole problem (I need to find a site that uses SSL adverts to test). I did also follow the instruction in the guides to set the default site in NPM to redirect to my pihole address - maybe this is addressing the SSL sinkhole problem?

[github-actions[bot]]

Issue is now considered stale. If you want to keep it open, please comment :+1: