New Install - Can't add Linux Nodes

[toragnetworks]

I have a new NetMaker install using the script found on the Github page. Everything works except adding a node. I can add an external devices, networks, users, ACLs, etc. no problem.

My Linux machines are running Ubuntu 20.XX and I am using the premade script that is displayed when you add a key. Both Wireguard and systemd are installed on my VMs. These VMs are on the same LAN as the NetMaker VM but are using different WAN IPs via outbound NAT rules.

When running the script, it hangs on: [netclient] 2022-05-05 01:58:03 joining MYNETNAME at api.netmaker.mydomain.com:443 followed by: [netclient] 2022-05-05 01:59:04 removed systemd remnants if any existed Then the script terminates.

My firewall rules are all correct on the VM and my upstream firewall. When I visit https://api.netmaker.mydomain.com, the page is accessible and it's blank white with 404 page not found

My Caddyfile is as follows: `# Dashboard https://dashboard.netmaker.mydomain.com {

Apply basic security headers

    header {
            # Enable cross origin access to *.netmaker.mydomain.com
            Access-Control-Allow-Origin *.netmaker.mydomain.com
            # Enable HTTP Strict Transport Security (HSTS)
            Strict-Transport-Security "max-age=31536000;"
            # Enable cross-site filter (XSS) and tell browser to block detected attacks
            X-XSS-Protection "1; mode=block"
            # Disallow the site to be rendered within a frame on a foreign domain (clickjacking protection)
            X-Frame-Options "SAMEORIGIN"
            # Prevent search engines from indexing
            X-Robots-Tag "none"
            # Remove the server name
            -Server
    }

    reverse_proxy http://127.0.0.1:8082

}

API

https://api.netmaker.mydomain.com { reverse_proxy http://127.0.0.1:8081 } ` I've tried on multiple VMs and it always hangs on the first two errors I posted. Any ideas on where I should be looking to troubleshoot this issue?

Any help appreciated!

[afeiszli]

Please run “netclient join -t (token) -vvv” from one of the machine manually and post output here.

[toragnetworks]

Please run “netclient join -t (token) -vvv” from one of the machine manually and post output here.

[netclient] 2022-05-05 13:10:19 joining MYNETWORK at api.netmaker.mydomain.com:443 [netclient] 2022-05-05 13:10:49 error installing: error creating node Post "https://api.netmaker.mydomain.com:443/api/nodes/MYNETWORK": dial tcp 99.99.999.99:443: i/o timeout [netclient] 2022-05-05 13:11:20 removed systemd remnants if any existed URLs all look correct, and the IP is the NetMaker VM's public IP address. I've redacted that for security. I also went ahead and completely disabled the upstream pfsense firewall and the NetMaker VM's firewall. This would allow any and all traffic through to the VM - ran the command again and received the same error.

I also used an online port checker and it confirms port 443 of my NetMaker VM is open.

[toragnetworks]

Quick update, spun up a Linode just to test from outside my server...this is what I received. Clean install of Ubuntu, using the install command from NetMaker's GUI.

[netclient] 2022-05-05 19:22:32 joining MYNETWORK at api.netmaker.mydomain.com:443 [netclient] 2022-05-05 19:22:33 starting wireguard [netclient] 2022-05-05 19:22:35 certificates/key saved [netclient] 2022-05-05 19:23:05 unable to connect to broker, retrying ... [netclient] 2022-05-05 19:23:05 could not connect to broker broker.netmaker.mydomain.com connect timeout [netclient] 2022-05-05 19:23:05 connection issue detected.. attempt connection with new certs [netclient] 2022-05-05 19:23:05 certificates/key saved [netclient] 2022-05-05 19:23:07 error running command: systemctl restart netclient.service [netclient] 2022-05-05 19:23:07 Failed to restart netclient.service: Unit netclient.service not found. [netclient] 2022-05-05 19:23:37 could not connect to broker at broker.netmaker.mydomain.com:8883 [netclient] 2022-05-05 19:23:37 sent a node update to server for node localhost , ff326b2a-rew2-478a-bd21-69c0fd23a9c3 broker.netmaker does not exist in my Caddy file, and the install guide doesn't mention it: ` Caddy will create 3 subdomains with this wildcard, EX:

dashboard.netmaker.example.com

api.netmaker.example.com

grpc.netmaker.example.com

`

[toragnetworks]

Just for fun, I deleted my VM and started from scratch...reinstalled Ubuntu 20.04 clean, updated it, and ran the install script on github with my domain and email.

I am still receiving the same error from VMs on the same LAN: [netclient] 2022-05-05 13:10:19 joining MYNETWORK at api.netmaker.mydomain.com:443 [netclient] 2022-05-05 13:10:49 error installing: error creating node Post "https://api.netmaker.mydomain.com:443/api/nodes/MYNETWORK": dial tcp 99.99.999.99:443: i/o timeout [netclient] 2022-05-05 13:11:20 removed systemd remnants if any existed

And made another fresh install Ubuntu 20.04 Linode, and receiving the same error: [netclient] 2022-05-05 19:22:32 joining MYNETWORK at api.netmaker.mydomain.com:443 [netclient] 2022-05-05 19:22:33 starting wireguard [netclient] 2022-05-05 19:22:35 certificates/key saved [netclient] 2022-05-05 19:23:05 unable to connect to broker, retrying ... [netclient] 2022-05-05 19:23:05 could not connect to broker broker.netmaker.mydomain.com connect timeout [netclient] 2022-05-05 19:23:05 connection issue detected.. attempt connection with new certs [netclient] 2022-05-05 19:23:05 certificates/key saved [netclient] 2022-05-05 19:23:07 error running command: systemctl restart netclient.service [netclient] 2022-05-05 19:23:07 Failed to restart netclient.service: Unit netclient.service not found. [netclient] 2022-05-05 19:23:37 could not connect to broker at broker.netmaker.mydomain.com:8883 [netclient] 2022-05-05 19:23:37 sent a node update to server for node localhost , ff326b2a-rew2-478a-bd21-69c0fd23a9c3

Interestingly on the Linode, even though I get errors, it will show up in my NetMaker GUI with a "WARNING" status

[afeiszli]

Have you followed the new instructions for the broker for 0.13? You must ensure 8883 is open and reachable and that broker.domain points to your machine.

https://gist.github.com/mattkasun/face2a7c1f32031a2126ff7243caad12

[toragnetworks]

Have you followed the new instructions for the broker for 0.13? You must ensure 8883 is open and reachable and that broker.domain points to your machine.

https://gist.github.com/mattkasun/face2a7c1f32031a2126ff7243caad12

No, I followed the documentation here: https://docs.netmaker.org/quick-start.html

Based on what you linked in your comment, it seems the official documentation is out of date because it has no mentioned of port 8883.

[toragnetworks]

Closing issue, moving to other solution