search

robbertkl / docker-ipv6nat

Extend Docker with IPv6 NAT, similar to IPv4
MIT License
662 stars 48 forks source link

App not available when activating ipv6nat #56

Closed doc75 closed 4 years ago

doc75 commented 4 years ago

I am trying to use ipv6nat, as it seems that it would allow to have ebtter IPv6 support with docker. Indeed currently I got a generic IPv4 address in X-FORWADED-FR instead of IPv6.

I have an issue when I launch several services and it happens only for php services behind NGINX (but not for all services).

My configuration is the following:

Traefik <= "web" docker network => NGINX reverse proxy <= "default" docker network=> PHP-FPM app

Traefik is the only front end connected to the WWW on port 80 and 443.

ipv6nat is defined as below:

services:
  ipv6nat:
    image: robbertkl/ipv6nat
    privileged: true
    network_mode: "host"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /lib/modules:/lib/modules:ro

I can see the following in my logs:

web_1    | 2020/04/13 22:18:16 [error] 7#7: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.5, server: , request: "GET /apps/files/?dir=/&fileid=2 HTTP/1.1", upstream: "fastcgi://[fd00:dead:beef::9]:9000", host: "cloud.localhost"
web_1    | 2020/04/13 22:18:16 [warn] 7#7: *1 upstream server temporarily disabled while connecting to upstream, client: 172.18.0.5, server: , request: "GET /apps/files/?dir=/&fileid=2 HTTP/1.1", upstream: "fastcgi://[fd00:dead:beef::9]:9000", host: "cloud.localhost"
web_1    | 2020/04/13 22:18:16 [error] 7#7: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.5, server: , request: "GET /apps/files/?dir=/&fileid=2 HTTP/1.1", upstream: "fastcgi://[fd00:dead:beef::8]:9000", host: "cloud.localhost"
web_1    | 2020/04/13 22:18:16 [warn] 7#7: *1 upstream server temporarily disabled while connecting to upstream, client: 172.18.0.5, server: , request: "GET /apps/files/?dir=/&fileid=2 HTTP/1.1", upstream: "fastcgi://[fd00:dead:beef::8]:9000", host: "cloud.localhost"
web_1    | 2020/04/13 22:18:16 [error] 7#7: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.5, server: , request: "GET /apps/files/?dir=/&fileid=2 HTTP/1.1", upstream: "fastcgi://[fd00:dead:beef::6]:9000", host: "cloud.localhost"
web_1    | 2020/04/13 22:18:16 [warn] 7#7: *1 upstream server temporarily disabled while connecting to upstream, client: 172.18.0.5, server: , request: "GET /apps/files/?dir=/&fileid=2 HTTP/1.1", upstream: "fastcgi://[fd00:dead:beef::6]:9000", host: "cloud.localhost"
web_1    | 2020/04/13 22:18:16 [error] 7#7: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.5, server: , request: "GET /apps/files/?dir=/&fileid=2 HTTP/1.1", upstream: "fastcgi://172.18.0.9:9000", host: "cloud.localhost"
web_1    | 2020/04/13 22:18:16 [warn] 7#7: *1 upstream server temporarily disabled while connecting to upstream, client: 172.18.0.5, server: , request: "GET /apps/files/?dir=/&fileid=2 HTTP/1.1", upstream: "fastcgi://172.18.0.9:9000", host: "cloud.localhost"
web_1    | 2020/04/13 22:18:16 [error] 7#7: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.5, server: , request: "GET /apps/files/?dir=/&fileid=2 HTTP/1.1", upstream: "fastcgi://172.18.0.8:9000", host: "cloud.localhost"
web_1    | 2020/04/13 22:18:16 [warn] 7#7: *1 upstream server temporarily disabled while connecting to upstream, client: 172.18.0.5, server: , request: "GET /apps/files/?dir=/&fileid=2 HTTP/1.1", upstream: "fastcgi://172.18.0.8:9000", host: "cloud.localhost"
web_1    | 2020/04/13 22:18:16 [error] 7#7: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.5, server: , request: "GET /apps/files/?dir=/&fileid=2 HTTP/1.1", upstream: "fastcgi://172.18.0.6:9000", host: "cloud.localhost"
web_1    | 2020/04/13 22:18:16 [warn] 7#7: *1 upstream server temporarily disabled while connecting to upstream, client: 172.18.0.5, server: , request: "GET /apps/files/?dir=/&fileid=2 HTTP/1.1", upstream: "fastcgi://172.18.0.6:9000", host: "cloud.localhost"
web_1    | 172.18.0.5 - - [13/Apr/2020:22:18:16 +0000] "GET /apps/files/?dir=/&fileid=2 HTTP/1.1" 502 157 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "172.18.0.1"

It does not happen on all the containers working on the same schema mentioned above. I have never seen this issue when NGINX and PHP-FPM are in the same container (and therefore Traefik is directly connected to the NGINX/PHP-FPM container).

I can see that the web docker network is mentioned in the ip6tables in the FORWARD chain

How can I debug this ?

doc75 commented 4 years ago

Silly question, is it required to activate ipv6 mode in docker to make ipv6nat to work ?

robbertkl commented 4 years ago

Please see the README: https://github.com/robbertkl/docker-ipv6nat/blob/master/README.md#docker-ipv6-configuration

doc75 commented 4 years ago

@robbertkl sorry, but I misunderstood what was in the README. My fault.

We eventually understood what was going on. The issue is not related to ipv6nat, but ipv6nat revealed an issue. I am putting the explanation here in case it can help someone in the future.

Detailed explanation can be found here

Summary: We had two services (using docker-compose) with the same name. One connected to a docker network called web and one connected to a docker network called default. Another container was connected to both docker networks and therefore sometime it tried to access the wrong service.

The issue never occurs before using ipv6nat, reason for opening this ticket on my side.

robbertkl commented 4 years ago

Thanks for the details! No worries.