search

gravitl / netmaker-helm

34 stars 36 forks source link

netclient cannot connect to broker #33

Open jakoberpf opened 1 year ago

jakoberpf commented 1 year ago

Hey netmaker people,

I am trying to deploy netmaker (0.17.1), but want to use istio as ingress, because in my case everything uses it. UI and API work fine, but I have issue with the MQTT connections.

When I create a new access key and let a client connect via the netclient join ... command it goes into a loop of 

Feb 16 20:23:46 plex networkd-dispatcher[108]: WARNING:Unknown index 2565 seen, reloading interface list
Feb 16 20:23:46 plex systemd-networkd[96]: nm-test: Link UP
Feb 16 20:23:46 plex systemd-networkd[96]: nm-test: Gained carrier
Feb 16 20:23:46 plex netclient[125]: [netclient] 2023-02-16 20:23:46 [common.go-162] InitWireguard(): waiting for interface...
Feb 16 20:23:46 plex netclient[125]: [netclient] 2023-02-16 20:23:46 [common.go-190] InitWireguard(): interface ready - netclient.. ENGAGE
Feb 16 20:23:46 plex netclient[125]: [netclient] 2023-02-16 20:23:46 [mqpublish.go-52] checkin(): checkin with server(s) for all networks
Feb 16 20:23:47 plex netclient[125]: [netclient] 2023-02-16 20:23:47 [localport.go-40] UpdateLocalListenPort(): network: test local port has changed from  0  to  48022
Feb 16 20:24:17 plex netclient[125]: [netclient] 2023-02-16 20:24:17 [mqpublish.go-252] publish(): could not connect to broker at broker.netmaker.erpf.de:8883
Feb 16 20:24:17 plex netclient[125]: [netclient] 2023-02-16 20:24:17 [localport.go-47] UpdateLocalListenPort(): could not publish local port change connection timeout
Feb 16 20:24:47 plex netclient[125]: [netclient] 2023-02-16 20:24:47 [mqpublish.go-252] publish(): could not connect to broker at broker.netmaker.erpf.de:8883
Feb 16 20:24:47 plex netclient[125]: [netclient] 2023-02-16 20:24:47 [mqpublish.go-149] Hello(): Network: test error publishing ping, connection timeout
Feb 16 20:24:47 plex netclient[125]: [netclient] 2023-02-16 20:24:47 [mqpublish.go-150] Hello(): running pull on test to reconnect
Feb 16 20:24:47 plex systemd-networkd[96]: nm-test: Link DOWN
Feb 16 20:24:47 plex systemd-networkd[96]: nm-test: Lost carrier
Feb 16 20:24:47 plex netclient[125]: [netclient] 2023-02-16 20:24:47 [common.go-389] informPortChange(): network: test UDP hole punching enabled for node plex
Feb 16 20:24:49 plex networkd-dispatcher[108]: WARNING:Unknown index 2566 seen, reloading interface list

In this case I had istio listening on port 8883 without tls, but I also tried it on port 443 with tls. After thinking that istio in itself could be the issue I decided to test using a NodePort for the 8883 port instead with 

---
apiVersion: v1
kind: Service
metadata:
  name: {{ include "netmaker.fullname" . }}-mqtt-external
spec: 
  type: NodePort
  ports:
  - name: mqtt2
    port: 8883
    nodePort: 8883
    protocol: TCP
    targetPort: mqtt2    
  selector:
    app: {{ include "netmaker.fullname" . }}-mqtt
  sessionAffinity: None

And this work only sort of. When I set allow_anonymous true I can connect with a mqtt client from my machine via the dns record broker.netmaker.<masked>.de. Also when I allow_anonymous false I can attempt a connection with a mqtt client from my machine and get an Unauthorized (since I don't have an login credentials). So far so good, I would believe connectivity is possible.

But in all scenarios the actual netclient is not able to connect to the broker, neither with allow_anonymous true or false.

One question would be, as this is NodePort from the service and therefore is no TLS on the connection, is this even supposed to work or do I need TLS anyway?

Any hint or tips would be greatly appreciated... :-)

ashokrajar commented 1 year ago

I am also facing the exact same issue but the difference is I am running in a binary install mode(non-docker).

I have checked that the required ports are open. Due to this the connection is unstable and get disconnected too frequently.

netclient[3749339]: [netclient] 2023-04-09 16:49:32 [mqpublish.go-252] publish(): could not connect to broker at <public-ip-here>:8883
netclient[3749339]: [netclient] 2023-04-09 16:49:32 [localport.go-47] UpdateLocalListenPort(): could not publish local port change connection timeout
netclient[3749339]: [netclient] 2023-04-09 16:50:02 [mqpublish.go-252] publish(): could not connect to broker at <public-ip-here>:8883
netclient[3749339]: [netclient] 2023-04-09 16:50:02 [mqpublish.go-149] Hello(): Network: network1 error publishing ping, connection timeout