search

caprover / caprover

Scalable PaaS (automated Docker+nginx) - aka Heroku on Steroids
https://CapRover.com
Other
12.96k stars 826 forks source link

Port 3000 not acessible! #893

Closed CookieCr2nk closed 3 years ago

CookieCr2nk commented 3 years ago

What is the problem?

My Server is behind a NAT in a DMZ zone and have private IPv4 Address assigned. First i open all ports in ufw and install Docker with the command "curl -sSL https://get.docker.com/ | CHANNEL=stable sh". After deploying CapRover i cannot reach port 3000 in my Internal Network via 10.1.0.10:3000. Port 80 and 443 are working. But " curl localhost:3000 -v" is working and i receive some output. Is this a problem with CapRover? Because on a digitalocean droplet it working smoothly a few days ago.

UFW Rules:
To                         Action      From
--                         ------      ----
80,443,996,2377,3000,4789,7946/tcp ALLOW       Anywhere
2377,4789,7946/udp         ALLOW       Anywhere
22/tcp                     ALLOW       Anywhere
80,443,996,2377,3000,4789,7946/tcp (v6) ALLOW       Anywhere (v6)
2377,4789,7946/udp (v6)    ALLOW       Anywhere (v6)
22/tcp (v6)                ALLOW       Anywhere (v6)

Answers to the following questions where applicable:

githubsaturn commented 3 years ago

Docker overlay network directly changes the IP table and overrides the UFW rules. CapRover uses your public IP address by default to bind to the network adapter. You can override this by passing -e MAIN_NODE_IP_ADDRESS=10.1.0.10 to the install command, like this:

docker run -e MAIN_NODE_IP_ADDRESS=10.1.0.10 -p 80:80 -p 443:443 -p 3000:3000 -v /var/run/docker.sock:/var/run/docker.sock -v /captain:/captain caprover/caprover

PS: Port 80/443 are host-mode mapped ports that they are different from port 3000.

CookieCr2nk commented 3 years ago

Thanks, It's working now from outside the network from a VPN or a Mobile Network.

But when i try to connect to port 3000 from the same Subnet the site won't loading. When i run "docker inspect " i don't see the environment variables MAIN_NODE_IP_ADDRESS.

            "Env": [
                "IS_CAPTAIN_INSTANCE=1",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "NODE_VERSION=14.15.0",
                "YARN_VERSION=1.22.5",
                "FRONTEND_COMMIT_HASH=3d5623f1d6b15a1426a3c995605b5e78e852e79b",
                "NODE_ENV=production",
                "PORT=3000"
CookieCr2nk commented 3 years ago

Here is my Captain Overlay Network Config:

root@vhost02:~# docker network ls
NETWORK ID          NAME                      DRIVER              SCOPE
266181315c54        bridge                    bridge              local
42m8fe5faib3        captain-overlay-network   overlay             swarm
52ff01a7228b        docker_gwbridge           bridge              local
3801e5c7dfdd        host                      host                local
metwwb4i34p0        ingress                   overlay             swarm
568d3766d3dc        none                      null                local
root@vhost02:~# docker inspect 42m8fe5faib3
[
    {
        "Name": "captain-overlay-network",
        "Id": "42m8fe5faib3vhceu73dlmrle",
        "Created": "2020-11-04T15:22:54.74156721Z",
        "Scope": "swarm",
        "Driver": "overlay",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "10.0.1.0/24",
                    "Gateway": "10.0.1.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": true,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "73c5a641b36297c9efed2e989b139800330e303961dbb5413916797c5e6010b9": {
                "Name": "captain-nginx.1.zqjm0bm68rgmhyto33yzzdp5z",
                "EndpointID": "9044c2031d67de860b18b6e92d5004362651c622b1de34731ed4878b7d42b9e0",
                "MacAddress": "02:42:0a:00:01:07",
                "IPv4Address": "10.0.1.7/24",
                "IPv6Address": ""
            },
            "ea3a6ed1098a52b0f989f8ee517c849060f06b47b613630943642ab815e23429": {
                "Name": "captain-captain.1.cwysf29pxhp69ywssakff2q24",
                "EndpointID": "dfc18cf10d1ea3df0a32360ad46ab8f643a47373c1d92a862aa01fe22e1d5404",
                "MacAddress": "02:42:0a:00:01:03",
                "IPv4Address": "10.0.1.3/24",
                "IPv6Address": ""
            },
            "lb-captain-overlay-network": {
                "Name": "captain-overlay-network-endpoint",
                "EndpointID": "491c4a187b1104a69a63835a58a03136e4c81f5d56cf9cf9245274e4621f23df",
                "MacAddress": "02:42:0a:00:01:05",
                "IPv4Address": "10.0.1.5/24",
                "IPv6Address": ""
            }
        },
        "Options": {
            "com.docker.network.driver.overlay.vxlanid_list": "4097"
        },
        "Labels": {},
        "Peers": [
            {
                "Name": "d44d18adb2c4",
                "IP": "10.0.0.10"
            }
        ]
    }
]
root@vhost02:~#
githubsaturn commented 3 years ago

Trying to understand why you want to connect to 3000 from the same subnet? port 3000 is only for the initial setup and you shouldn't be needing it once you set up your domain.

CookieCr2nk commented 3 years ago

I have setup it now. It works now with Port 80 and 443. But i think that CapRover is not working well behind a NAT. On Servers with a Public IPv4 it's working smoothly.

The Containers can't resolve "api.v2.caprover.com" and "oneclickapps.caprover.com" but on the Host with dig it's working. I don't have any Firewall that blocks Port 53/udp.

captain-captain.1.q175c16gl80j@vhost02    | Error: getaddrinfo EAI_AGAIN api.v2.caprover.com
captain-captain.1.q175c16gl80j@vhost02    |     at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:67:26)
captain-captain.1.q175c16gl80j@vhost02    | GET /api/v2/user/system/versioninfo 200 20046.653 ms - 151
captain-captain.1.q175c16gl80j@vhost02    | November 5th 2020, 10:51:06.522 am    Error: getaddrinfo EAI_AGAIN oneclickapps.caprover.com
captain-captain.1.q175c16gl80j@vhost02    | Error: getaddrinfo EAI_AGAIN oneclickapps.caprover.com
captain-captain.1.q175c16gl80j@vhost02    |     at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:67:26)
captain-captain.1.q175c16gl80j@vhost02    | GET /api/v2/user/oneclick/template/list 200 20049.568 ms - 90
captain-captain.1.q175c16gl80j@vhost02    | GET /api/v2/user/apps/appDefinitions 200 1.362 ms - 2749
captain-captain.1.q175c16gl80j@vhost02    | GET /api/v2/user/oneclick/repositories 200 1.212 ms - 87
captain-captain.1.q175c16gl80j@vhost02    | November 5th 2020, 10:53:49.312 am    Error: getaddrinfo EAI_AGAIN oneclickapps.caprover.com
githubsaturn commented 3 years ago

CapRover works behind a NAT. Outgoing connections do not have anything to do NAT. What makes you think it's related to NAT?

CookieCr2nk commented 3 years ago

I want to use my Main Node on my Home Server behind a NAT and for the Workers Node i am planning to use the Hetzner Cloud Servers. I assume that i have the same issue (https://github.com/caprover/caprover/issues/812).

VM IPv4 Address: 10.0.0.10 Router/Gateway: 10.0.0.1

root@vhost02:~# docker exec -it $(docker ps --filter name=captain-captain -q) /bin/sh
# ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=4.85 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=57 time=5.18 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=57 time=4.74 ms
^C
--- 1.1.1.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2062ms
rtt min/avg/max/mdev = 4.749/4.931/5.188/0.203 ms
# ping github.com
^C
#
root@4bcd87c73b7d:/usr/src/app# cat /etc/resolv.conf
search lan
nameserver 127.0.0.11
options ndots:0
root@4bcd87c73b7d:/usr/src/app#
CookieCr2nk commented 3 years ago

I had solved the Issue with the command: "root@vhost02:~# docker service update captain-captain --dns-add 1.1.1.1". Is this still productive? do you have a clue why this happens?

githubsaturn commented 3 years ago

I want to use my Main Node on my Home Server behind a NAT and for the Workers Node i am planning to use the Hetzner Cloud Servers.

I see. Yea you can't do that with CapRover. Worker nodes and main server need to both be public IP servers, or at least on the same subnet. I am not sure even without CapRover, with raw docker, you can do it.

I had solved the Issue with the command: "root@vhost02:~# docker service update captain-captain --dns-add 1.1.1.1". Is this still productive? do you have a clue why this happens?

You shouldn't need this. My best "guess" is that due to specifics of your home network, the DNS in Docker network didn't pick up the parent DNS.

Also, keep in mind that CapRover is really designed to be your easy one stop solution for common use cases. Although you might be able to hack your way out and make it work, you won't get the best out of CapRover. You'd be better off with a manually set up build system and manually set up Docker cluster.