gravitl / netmaker

Netmaker makes networks with WireGuard. Netmaker automates fast, secure, and distributed virtual networks.
https://netmaker.io
Other
9.55k stars 553 forks source link

Not Compatible with Nginx Proxy Manager? #2986

Open kylebial0 opened 5 months ago

kylebial0 commented 5 months ago

Environment I am looking to utilize NetMaker on my homelab that is running Proxmox on a LXC container. I have a Nginx Proxy Manager LXC setup that is forwarding the required sub-domains to the LXC hosting NetMaker, including API.{NM_DOMAIN}, Dashboard.{NM_DOMAIN}, grpc.{NM_DOMAIN}, and broker.{NM_DOMAIN}.

Issue Description When I install NetMaker utilizing the quick installer provided, I receive the following

`-----------------------------------------------------------------
Beginning installation...
-----------------------------------------------------------------
. . .
. . .
. . .
Pulling config files...
Saving the config to /root/netmaker.env
Starting containers...
WARN[0000] /root/docker-compose.yml: `version` is obsolete 
[+] Running 5/5
 ✔ Container caddy        Started                                                                                                                                                                                      11.8s 
 ✔ Container netmaker     Started                                                                                                                                                                                      11.8s 
 ✔ Container coredns      Started                                                                                                                                                                                      12.3s 
 ✔ Container mq           Started                                                                                                                                                                                      12.4s 
 ✔ Container netmaker-ui  Started                                                                                                                                                                                      12.4s 
/root
. . .
. . .
Testing Caddy setup (please be patient, this may take 1-2 minutes)
    Certificates ok
Downloading nmctl...
using server api.netmaker.DOMAIN.com
using master key MASTER KEY VALUE
2024/06/18 23:41:25 Error Status: 502 Response: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>openresty</center>
</body>
</html>
. . .
. . .
. . .
. . .
. . .
2024/06/18 23:41:30 Error Status: 502 Response: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>openresty</center>
</body>
</html>
/root/nm-quick.sh: line 726: [: : integer expression expected
Obtaining a netmaker enrollment key...
2024/06/18 23:41:30 Error Status: 502 Response: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>openresty</center>
</body>
</html>
Error creating an enrollment key`

Expected Outcome

I would expect for NetMaker to install successfully.

Does anyone have any recommendations of what I can try? I have spent hours diagnosing this issue and have not had any luck.

I have tried using the community docker-compose script to run without Caddy, and that also did not work for me. Whenever I try accessing dashboard, I reach the 502 openresty page.

I know my proxy manager itself is working correctly, because traffic will forward to other LXCs I am running. I'm assuming this may have something to do with headers, but I am not nearly experienced enough with Nginx or Caddy to track down the issue.

Any recommendations would be greatly appreciated!

gigadjo commented 1 month ago

Hello, i tried and found a working solution :-)

Basically, you'll need to setup Netmaker "the hard way" :

version: "3.4"

services:

  netmaker:
    container_name: netmaker
    image: gravitl/netmaker:$SERVER_IMAGE_TAG
    env_file: ./netmaker.env
    restart: always
    volumes:
      - dnsconfig:/root/config/dnsconfig
      - sqldata:/root/data
    environment:
      # config-dependant vars
      - STUN_LIST=stun1.netmaker.io:3478,stun2.netmaker.io:3478,stun1.l.google.com:19302,stun2.l.google.com:19302
      # The domain/host IP indicating the mq broker address
      - BROKER_ENDPOINT=wss://broker.${NM_DOMAIN} # For EMQX broker use `BROKER_ENDPOINT=wss://broker.${NM_DOMAIN}/mqtt`
      # For EMQX broker (uncomment the two lines below)
      #- BROKER_TYPE=emqx
      #- EMQX_REST_ENDPOINT=http://mq:18083
      # The base domain of netmaker
      - SERVER_NAME=${NM_DOMAIN}
      - SERVER_API_CONN_STRING=api.${NM_DOMAIN}:443
      # Address of the CoreDNS server. Defaults to SERVER_HOST
      - COREDNS_ADDR=${SERVER_HOST}
      # Overrides SERVER_HOST if set. Useful for making HTTP available via different interfaces/networks.
      - SERVER_HTTP_HOST=api.${NM_DOMAIN}
    ports:
      - "8081:8081"
      - "50051:50051"

  netmaker-ui:
    container_name: netmaker-ui
    image: gravitl/netmaker-ui:$UI_IMAGE_TAG
    env_file: ./netmaker.env
    environment:
      # config-dependant vars
      # URL where UI will send API requests. Change based on SERVER_HOST, SERVER_HTTP_HOST, and API_PORT
      BACKEND_URL: "https://api.${NM_DOMAIN}"
    depends_on:
      - netmaker
    links:
      - "netmaker:api"
    restart: always
    ports:
      - "80:80"

  #caddy:
  #  image: caddy:2.6.2
  #  container_name: caddy
  #  env_file: ./netmaker.env
  #  restart: unless-stopped
  #  extra_hosts:
  #    - "host.docker.internal:host-gateway"
  #  volumes:
  #    - ./Caddyfile:/etc/caddy/Caddyfile
  #    - caddy_data:/data
  #    - caddy_conf:/config
  #  ports:
  #    - "80:80"
  #    - "443:443"

  coredns:
    #network_mode: host
    container_name: coredns
    image: coredns/coredns:1.10.1
    command: -conf /root/dnsconfig/Corefile
    env_file: ./netmaker.env
    restart: always
    depends_on:
      - netmaker
    volumes:
      - dnsconfig:/root/dnsconfig
  mq:
    container_name: mq
    image: eclipse-mosquitto:2.0.15-openssl
    env_file: ./netmaker.env
    depends_on:
      - netmaker
    restart: unless-stopped
    command: [ "/mosquitto/config/wait.sh" ]
    volumes:
      - ./mosquitto.conf:/mosquitto/config/mosquitto.conf
      - ./wait.sh:/mosquitto/config/wait.sh
      - mosquitto_logs:/mosquitto/log
      - mosquitto_data:/mosquitto/data
    ports:
      - "8883:8883"
      - "1883:1883"

volumes:
  caddy_data: { } # runtime data for caddy
  caddy_conf: { } # configuration file for Caddy
  sqldata: { }
  dnsconfig: { } # storage for coredns
  mosquitto_logs: { } # storage for mqtt logs
  mosquitto_data: { } # storage for mqtt data

Create a netmaker.env file, looking like this :

NM_EMAIL=<YourEmail>
NM_DOMAIN=<SubDomainUsedForNetmaker.domain.tld> #you will need to have it pointed to your public facing NPM ip
FRONTEND_URL=
UI_IMAGE_TAG=v0.25.0
NETMAKER_TENANT_ID=
LICENSE_KEY=
SERVER_IMAGE_TAG=v0.25.0
SERVER_HOST=<HostPubIp>
MASTER_KEY=<YourMasterKey> # eg : zTI627VjuwpBZ9gLrfCy3tL9OwDljV 
MQ_USERNAME=netmaker
MQ_PASSWORD=<your-secure-password>
INSTALL_TYPE=ce
NODE_ID=<NameYourNode>
DNS_MODE=on
NETCLIENT_AUTO_UPDATE=enabled
API_PORT=8081
CORS_ALLOWED_ORIGIN=*
DISPLAY_KEYS=on
DATABASE=sqlite
SERVER_BROKER_ENDPOINT=ws://mq:1883
VERBOSITY=1
DEBUG_MODE=off
REST_BACKEND=on
DISABLE_REMOTE_IP_CHECK=off
TELEMETRY=on
ALLOWED_EMAIL_DOMAINS=*
AUTH_PROVIDER=
CLIENT_ID=
CLIENT_SECRET=
AZURE_TENANT=
OIDC_ISSUER=
EXPORTER_API_PORT=8085
JWT_VALIDITY_DURATION=43200
RAC_AUTO_DISABLE=false
CACHING_ENABLED=true
ENDPOINT_DETECTION=true
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
EMAIL_SENDER_ADDR=
EMAIL_SENDER_USER=
EMAIL_SENDER_PASSWORD=
METRICS_EXPORTER=off
PROMETHEUS=off

Then, set your hosts in NGinx Proxy Manager, like explained in this repo : https://github.com/upgrade-computer/netmaker-nginx-proxy-manager-v2

Worked pretty well :-)

/joen