search

Cielquan / DoTH-DNS

Your server doth DNS the safe way if you use DoTH-DNS.
GNU General Public License v3.0
29 stars 1 forks source link

HTTP site inaccessible #10

Open willis936 opened 3 years ago

willis936 commented 3 years ago

Description

HTTP and HTTPS (when configured) sites return a simple 404 page not found.

Continuing discussion from here:

https://github.com/Cielquan/DoTH-DNS/issues/8#issuecomment-802899638

What I Did

Check that pihole and unbound working correctly:

~# docker exec -it pihole bash
/# pihole status
/# dig github.com

pihole status shows pihole is up and dig shows that the localhost is the referenced DNS server.

Continuing from this comment:

https://github.com/Cielquan/DoTH-DNS/issues/8#issuecomment-809208969

~# docker logs traefik
time="2021-03-27T17:29:59Z" level=info msg="Configuration loaded from file: /etc/traefik/traefik.toml"
time="2021-03-27T17:29:59Z" level=info msg="Traefik version 2.1.9 built on 2020-03-23T16:55:31Z"
time="2021-03-27T17:29:59Z" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://docs.traefik.io/contributing/data-collection/\n"
time="2021-03-27T17:29:59Z" level=info msg="Starting provider aggregator.ProviderAggregator {}"
time="2021-03-27T17:29:59Z" level=info msg="Starting provider *file.Provider {\"directory\":\"/etc/traefik/traefik.conf.d/\",\"watch\":true}"
time="2021-03-27T17:29:59Z" level=info msg="Starting provider *traefik.Provider {}"
time="2021-03-27T17:29:59Z" level=info msg="Starting provider *docker.Provider {\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"swarmModeRefreshSeconds\":15000000000}"
time="2021-03-27T17:30:03Z" level=info msg="Skipping same configuration" providerName=docker
time="2021-03-27T17:30:06Z" level=info msg="Skipping same configuration" providerName=docker
10.111.222.128 - - [27/Mar/2021:17:30:26 +0000] "GET / HTTP/2.0" 404 19 "-" "-" 1 "-" "-" 0ms
10.111.222.128 - - [27/Mar/2021:17:30:26 +0000] "GET /favicon.ico HTTP/2.0" 404 19 "-" "-" 2 "-" "-" 0ms
time="2021-03-27T17:40:01Z" level=warning msg="A new release has been found: 2.4.8. Please consider updating."10.111.222.128 - - [27/Mar/2021:17:48:10 +0000] "GET / HTTP/2.0" 404 19 "-" "-" 3 "-" "-" 0ms
time="2021-03-28T18:30:01+01:00" level=warning msg="A new release has been found: 2.4.8. Please consider updating."
10.111.222.130 - - [29/Mar/2021:11:56:29 +0000] "GET / HTTP/2.0" 404 19 "-" "-" 4 "-" "-" 0ms
10.111.222.130 - - [29/Mar/2021:11:56:29 +0000] "GET /favicon.ico HTTP/2.0" 404 19 "-" "-" 5 "-" "-" 0ms

I just noticed that the readme update says that this project was made for pihole 4. It’s possible that this issue is related to pihole 5.

Cielquan commented 3 years ago

The setup log part looks fine to me.

But you can see the 404s in the lower half of the log. I'll have to check with my stack how it looks when I have a "real" 404 later.

Pihole 4/5 should have nothing to do with this issue, because the path (/admin) in the URL did not change. I still run this stack (manually set up the old way with docker-compose) on my aws VM but with pihole 5 and I cannot remember changing anything URL-path related.

EDIT: @willis936 But one thing you can to in the meantime: Can you access the traefik dashboard? (https://traefik.<DOMAIN> should redirect to the appropriate path)

willis936 commented 3 years ago

When I try to open the traefik dashboard by going to https://traefik.<FQDN> and https://traefik.<IP> on a LAN machine’s web browser, the address does not resolve. Originally the DNS server was set to an upstram pihole but I updated the DNS server reference on this machine to only be the DoTH-DNS machine. The same results were observed

The DoTH-DNS machine is headless, so I can’t check with a browser. From the host machne’s shell I rua ping, dig, and wget on traefik.<FQDN> and traefik.<IP>. The IP does not resolve. The DNS server is set to 127.0.0.1 in etc/resolv.conf and is referenced in dig.

Before I set the upstream DNS to the DoTH-DNS machine, the docker log showed some new entries. Nothing else I’ve tried in this comment resulted in traefik log entries.

time="2021-03-28T18:30:01+01:00" level=warning msg="A new release has been found: 2.4.8. Please consider updating."
10.111.222.130 - - [29/Mar/2021:11:56:29 +0000] "GET / HTTP/2.0" 404 19 "-" "-" 4 "-" "-" 0ms
10.111.222.130 - - [29/Mar/2021:11:56:29 +0000] "GET /favicon.ico HTTP/2.0" 404 19 "-" "-" 5 "-" "-" 0ms
10.111.222.2 - - [29/Mar/2021:16:37:50 +0000] "GET / HTTP/2.0" 404 19 "-" "-" 6 "-" "-" 0ms
time="2021-03-29T17:38:15+01:00" level=error msg="Error while Peeking first byte: read tcp 172.19.0.2:80->10.111.222.2:53753: read: connection reset by peer"
time="2021-03-29T17:38:15+01:00" level=error msg="Error while Peeking first byte: read tcp 172.19.0.2:80->10.111.222.2:53751: read: connection reset by peer"
10.111.222.2 - - [29/Mar/2021:16:40:01 +0000] "GET / HTTP/1.1" 301 17 "-" "-" 7 "rou_GlobalHttps@file" "-" 1ms
10.111.222.2 - - [29/Mar/2021:16:40:13 +0000] "GET / HTTP/1.1" 301 17 "-" "-" 8 "rou_GlobalHttps@file" "-" 1ms
Cielquan commented 3 years ago

What traefik can route

An info beforehand: Via IP neither the traefik nor the pihole dashboards will be available because traefik does get all requests on ports 80, 443 and 853 but does not have any information how to route via the direct IP, because it can only route:

HTTP traffic:

HTTPS traffic:

TCP traffic (with TLS encryption):

Automatic "forwarding"

traefik automatically forwards traffic to traefik.<FQDN> to traefik.<FQDN>/dashboard/ with HTTP code 302:

172.31.42.2 - admin [29/Mar/2021:19:28:16 +0000] "GET / HTTP/2.0" 302 34 "-" "-" 10939 "rou_Traefik@docker" "-" 26ms
172.31.42.2 - admin [29/Mar/2021:19:28:16 +0000] "GET /dashboard/ HTTP/2.0" 200 3026 "-" "-" 10940 "rou_Traefik@docker" "-" 38ms

traefik manipulates every request to pihole.<FQDN> with a regex substitution which in the end removes /admin if it exists in the path, takes everything after /admin and overwrites the old path with /admin/<EXTRACTED REST>. The thing here is that this manipulated path is only send to pihole's webserver and not send back to the user with a 30x http response. Therefore the user can access the pihole dashboard with and without typing /admin.

Both result in the same resource. https://pihole.<FQDN>:

172.31.42.2 - - [29/Mar/2021:19:30:51 +0000] "GET / HTTP/2.0" 200 16900 "-" "-" 11009 "rou_PiholeGui@docker" "http://172.16.1.5:80" 732ms

https://pihole.<FQDN>/admin:

172.31.42.2 - - [29/Mar/2021:20:03:55 +0000] "GET /admin HTTP/2.0" 200 16897 "-" "-" 11183 "rou_PiholeGui@docker" "http://172.16.1.5:80" 610ms

Different responses to unavailable URLs

Here are traefik logs on a working system for the path /test for different domains:

https://pihole.<FQDN>/test

You can see the rou_PiholeGui@docker in the log lines which says that traefik's rou_PiholeGui router is doing its work and sending the data to pihole. pihole forwards to its blocking page, even though I have non set up, but this does not count for the dashboard paths I think.

https://pihole.<FQDN>/admin/test would result in the same outcome (as mentioned above) except that the paths in the log would have a preceeding /admin.

172.31.42.2 - - [29/Mar/2021:20:17:32 +0000] "GET /test HTTP/2.0" 200 3430 "-" "-" 11339 "rou_PiholeGui@docker" "http://172.16.1.5:80" 45ms
172.31.42.2 - - [29/Mar/2021:20:17:32 +0000] "GET /pihole/blockingpage.css HTTP/2.0" 200 608 "-" "-" 11340 "rou_PiholeGui@docker" "http://172.16.1.5:80" 1ms
172.31.42.2 - - [29/Mar/2021:20:17:32 +0000] "GET /admin/scripts/vendor/jquery.min.js HTTP/2.0" 304 0 "-" "-" 11341 "rou_PiholeGui@docker" "http://172.16.1.5:80" 1ms
172.31.42.2 - - [29/Mar/2021:20:17:32 +0000] "GET /admin/img/favicons/favicon.ico HTTP/2.0" 304 0 "-" "-" 11342 "rou_PiholeGui@docker" "http://172.16.1.5:80" 0ms
https://traefik.<FQDN>/test

Here you can see traefik's rou_Traefik@docker router in the log, which is sending the request to traefik's webserver which sends a 404 back. The important part here is that not traefik's proxy sends the 404 but it's webserver.

172.31.42.2 - admin [29/Mar/2021:20:19:24 +0000] "GET /test HTTP/2.0" 404 19 "-" "-" 11343 "rou_Traefik@docker" "-" 23ms
https://doh.dns.krys.codes/test

Here you cannot see a router in the log, but you can see the 404 response, which in this case comes directly from traefik's proxy.

172.31.42.2 - - [29/Mar/2021:20:22:04 +0000] "GET /test HTTP/2.0" 404 19 "-" "-" 11352 "-" "-" 0ms

If you compare these three results to the section "What traefik can route" above it does make sense, because any path on traefik.<FQDN> and pihole.<FQDN> gets forwarded but for doh.<FQDN> only the path dns-query is know by traefik's proxy. Therefore the last result is a 404 send by the proxy.

The issue

With the knowledge above we can finally look at you logs and see that the 404s three comments above do not have a router named in the log lines, which means that traefik does not know how to route the request. In the comment above there is the same case for the upper log part, but for the 2 last entries there is a HTTP -> HTTPS 301 response by the rou_GlobalHttps@file router.

From your comments above I assume that the HTTP/S requests do not reach the stack at all. But if the DNS request is not solved that's nothing strange. But it is strange that even the local dig is not resolved.

My DoTH-DNS stack uses my domain krys.codes which results in a private IP-address for the relevant subdomains. I do not have my machine (AWS VM) set to localhost because of that and for emergency if the stack goes down. But if I remember correctly I had my old machine (local RasPi) set to localhost for DNS and it worked.

Do you have a firewall active that could block something, even though it should not block something local?

Do you get the same result for:

$ docker inspect traefik -f "{{.NetworkSettings.Ports}}"
map[443/tcp:[{0.0.0.0 443}] 80/tcp:[{0.0.0.0 80}] 8080/tcp:[{0.0.0.0 8080}] 853/tcp:[{0.0.0.0 853}]]

Can you monitor the logs with docker logs -f traefik, log into the pihole docker (docker exec -it pihole bash) and run some curls:

The logs should document all those requests. If they do than the stack itself should work and there is "only" an issue for the DNS resolution via the stack.