IPv6-only not working with Photoprism

PennRobotics commented 5 months ago

TL;DR

When trying to run the Photoprism app on a shared server with only ipv6, errors appear in docker compose logs stating "network is unreachable"

Expected behavior

Running the Photoprism image would generate a certificate for a domain that already has its AAAA record defined on a server with an ipv6 address.

Observed behavior

A certificate is not generated, but the server can be accessed by its IP address as well as its AAAA-linked URL (therefore the photoprism part of the image works and DNS records are probably set up correctly).

Minimal working example

No response

Log output

traefik     | time="2024-03-06T15:54:13Z" level=error msg="Unable to obtain ACME certificate for domains \" {URL} \": cannot get ACME client get directory at 'https://acme-v02.api.letsencrypt.org/directory': Get \"https://acme-v02.api.letsencrypt.org/directory\": dial tcp 172.65.<>.<>:443: connect: network is unreachable" providerName=mytlschallenge.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory" routerName=photoprism@docker rule="Host(` {URL} `)"

^{URL redacted, server IP addr partially redacted}

Additional information

This was with reverse DNS enabled and AAAA records pointing to the server's IPv6 address. At no point did I add A or CNAME or ALIAS records for the ipv4 address, so I don't believe ipv4 is being used or is necessary between the visitor and server.

After adding an ipv4 address in the cloud console and some /etc/docker/daemon.conf workarounds on the server, such as adding a pair for registry-mirrors and ipv6 and ip6tables, the problem seems to be resolved (or the certificate generated by manually running certbot via pip was slow to get approved and then was imported by the docker image. I also tried about a dozen different things over a few hours, so it's possible some other change was the eventual turning of the "https" tide.

However, I also recreated the server image to start anew, and the moment that I enabled an ipv4 address in console, a certificate was generated and I could then access the site via https.

I don't know much about Docker but can try to run each Dockerfile manually, independently, and verbosely to get more info.

ilka-schulz commented 3 months ago

I just had the same issue, although no outgoing connections to hosts with both IPv4 and IPv6 were possible (blackbox exporter defaults to IPv6 and does not retry on IPv4 if IPv6 fails). Docker has disabled IPv6 by default and it needs to be enabled.

Working solution

Following Docker's official guide "Enable IPv6 support", I got it working:

create a docker config file under /etc/docker/daemon.json:
```
+{
+  "experimental": true,
+  "ip6tables": true
+}
```

change /opt/containers/prometheus-grafana/docker-compose.yml:

---
version: "3.8"

volumes:
prometheus_data: {}
grafana_data: {}
caddy_data: {}

networks:
proxy:
monitoring:
+    enable_ipv6: true
+    ipam:
+     config:
+       - subnet: 2001:0DB8::/112
services:

restart docker: systemctl restart docker.service
start containers again: cd /opt/containers/prometheus-grafana && docker compose up -d

PennRobotics commented 3 months ago

Thanks for the suggestion.

Life blockaded my progress on this issue, so I never got around to stepping through each command as I'd planned. (Also, the Storage Share has a "good enough" photo album for nearly the same cost—lacking only ffmpeg for video thumbnails—so I switched to that in the meantime.)

I can recall running the IPv6 enablement step-by-step early on and it still not working for some reason, as that was one of the top search results for "network is unreachable docker ipv6". Maybe I'm misremembering or missed a step or changed too much other stuff. I'll try again on a clean install with just the diff you've provided.

hetznercloud / apps