netbirdio / netbird

Connect your devices into a secure WireGuard®-based overlay network with SSO, MFA and granular access controls.
https://netbird.io
BSD 3-Clause "New" or "Revised" License
10.04k stars 441 forks source link

Cant connect from mobile-network (iOS & windows) #2016

Closed Berndinox closed 2 months ago

Berndinox commented 2 months ago

Describe the problem I tried connecting my mobile (iOS) and found that i cant connect any peer. I double checked an connected my windows laptop via mobile-phone hotspot and faced the same issue.

A clear and concise description of what the problem is.

To Reproduce

Steps to reproduce the behavior: Connect via Mobile Network Provider.

Expected behavior

Connect the peers.

Are you using NetBird Cloud?

Self-Hosted

NetBird version

netbird version

NetBird status -d output:

latest, installed on 19.05.25 (Quickstart)

Additional context

iOS Logs:

2024-05-20T07:17:32Z DEBG client/internal/login.go:93: connecting to the Management service https://my.domain.com:443
2024-05-20T07:17:32Z DEBG client/internal/login.go:34: connected to the Management service https://my.domain.com:443
2024-05-20T07:17:32Z INFO client/ios/NetBirdSDK/client.go:89: Starting NetBird client
2024-05-20T07:17:32Z DEBG client/ios/NetBirdSDK/client.go:90: Tunnel uses interface: utun7
2024-05-20T07:17:32Z DEBG client/internal/login.go:93: connecting to the Management service https://my.domain.com:443
2024-05-20T07:17:32Z DEBG client/internal/login.go:34: connected to the Management service https://my.domain.com:443
2024-05-20T07:17:32Z DEBG client/internal/login.go:93: connecting to the Management service https://my.domain.com:443
2024-05-20T07:17:32Z DEBG client/internal/login.go:63: connected to the Management service https://my.domain.com:443
2024-05-20T07:17:32Z INFO client/ios/NetBirdSDK/client.go:118: Auth successful
2024-05-20T07:17:32Z INFO client/internal/connect.go:115: starting NetBird client version v0.27.5 on ios/arm64
2024-05-20T07:17:32Z DEBG client/internal/connect.go:176: connecting to the Management service my.domain.com:443
2024-05-20T07:17:33Z DEBG client/internal/connect.go:184: connected to the Management service my.domain.com:443
2024-05-20T07:17:33Z DEBG signal/client/grpc.go:92: connected to Signal Service: my.domain.com:443
2024-05-20T07:17:33Z INFO client/internal/routemanager/manager.go:93: Routing setup complete
2024-05-20T07:17:33Z INFO iface/tun_ios.go:44: create tun interface
2024-05-20T07:17:33Z DEBG iface/tun_ios.go:66: Attaching to interface
2024-05-20T07:17:33Z ERRO iface/uapi.go:15: failed to open uapi socket: mkdir /var/run/wireguard: operation not permitted
2024-05-20T07:17:33Z ERRO iface/wg_configurer_usp.go:189: failed to open uapi listener: mkdir /var/run/wireguard: operation not permitted
2024-05-20T07:17:33Z DEBG iface/wg_configurer_usp.go:37: adding Wireguard private key
2024-05-20T07:17:33Z DEBG iface/tun_ios.go:93: device is ready to use: utun7
2024-05-20T07:17:33Z DEBG signal/client/grpc.go:137: signal connection state READY
2024-05-20T07:17:33Z INFO signal/client/grpc.go:158: connected to the Signal Service stream
2024-05-20T07:17:33Z DEBG client/internal/engine.go:640: connecting to Management Service updates stream
2024-05-20T07:17:33Z INFO client/internal/engine.go:359: Network monitor is disabled, not starting
2024-05-20T07:17:33Z INFO client/internal/connect.go:261: Netbird engine started, the IP is: 100.119.12.202/16
2024-05-20T07:17:33Z DEBG management/client/grpc.go:120: management connection state READY
2024-05-20T07:17:33Z INFO management/client/grpc.go:147: connected to the Management Service stream
2024-05-20T07:17:33Z DEBG management/client/grpc.go:253: got an update message from Management Service
2024-05-20T07:17:33Z DEBG client/internal/engine.go:666: got TURNs update from Management Service, updating
2024-05-20T07:17:33Z DEBG client/internal/engine.go:648: got STUNs update from Management Service, updating
2024-05-20T07:17:33Z WARN client/internal/engine.go:551: running SSH server is not permitted
2024-05-20T07:17:33Z DEBG client/internal/engine.go:697: got peers update from Management Service, total peers to connect to = 1
2024-05-20T07:17:33Z DEBG client/internal/engine.go:933: creating peer connection e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:17:33Z DEBG client/internal/routemanager/client.go:338: Received a new client network route update for 0.0.0.0/0
2024-05-20T07:17:33Z WARN client/internal/routemanager/client.go:154: the network 0.0.0.0/0 has not been assigned a routing peer as no peers from the list [e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=] are currently connected
2024-05-20T07:17:33Z DEBG client/internal/dns/host_ios.go:28: Applying DNS settings: {"domains":null,"routeAll":false,"serverIP":"100.119.255.254","serverPort":53}
2024-05-20T07:17:33Z INFO client/internal/acl/manager.go:52: ACL rules processed in: 149.958µs, total rules count: 2
2024-05-20T07:17:35Z DEBG client/internal/peer/conn.go:280: trying to connect to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:17:35Z DEBG client/internal/peer/conn.go:312: connection offer sent to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=, waiting for the confirmation
2024-05-20T07:17:35Z TRAC signal/client/grpc.go:400: received a new message from Peer [fingerprint: e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=]
2024-05-20T07:17:35Z DEBG client/internal/peer/conn.go:765: OnRemoteAnswer from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= on status Disconnected
2024-05-20T07:17:35Z DEBG client/internal/peer/conn.go:333: received connection confirmation from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= running version 0.27.7 and with remote WireGuard listen port 51820
2024-05-20T07:17:35Z DEBG client/internal/peer/conn.go:663: peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= ICE ConnectionState has changed to Checking
2024-05-20T07:17:35Z DEBG client/internal/peer/conn.go:622: discovered local candidate udp4 host 10.110.127.61:51820
2024-05-20T07:17:35Z DEBG client/internal/peer/conn.go:622: discovered local candidate udp4 host 192.168.4.22:51820
2024-05-20T07:17:35Z TRAC signal/client/grpc.go:400: received a new message from Peer [fingerprint: e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=]
2024-05-20T07:17:35Z DEBG client/internal/peer/conn.go:779: OnRemoteCandidate from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= -> udp4 host 172.17.0.2:51820
2024-05-20T07:17:35Z DEBG iface/bind/udp_mux.go:346: ICE: registered 172.17.0.2:51820 for FdYfEKxGPcGxNewA
2024-05-20T07:17:35Z DEBG client/internal/peer/conn.go:622: discovered local candidate udp4 srflx 176.66.87.43:43632 related 0.0.0.0:51820
2024-05-20T07:17:35Z DEBG iface/bind/udp_mux.go:346: ICE: registered 172.17.0.2:51820 for FdYfEKxGPcGxNewAstun:my.domain.com:3478
2024-05-20T07:17:47Z DEBG client/internal/peer/conn.go:663: peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= ICE ConnectionState has changed to Failed
2024-05-20T07:17:47Z DEBG client/internal/peer/conn.go:519: trying to cleanup e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:17:47Z DEBG iface/iface.go:88: Removing peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= from interface utun7 
2024-05-20T07:17:47Z DEBG client/internal/peer/conn.go:663: peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= ICE ConnectionState has changed to Closed
2024-05-20T07:17:47Z DEBG client/internal/peer/conn.go:577: cleaned up connection to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:17:47Z DEBG client/internal/engine.go:913: connection to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= failed: connecting canceled by caller
2024-05-20T07:17:49Z TRAC signal/client/grpc.go:400: received a new message from Peer [fingerprint: e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=]
2024-05-20T07:17:49Z DEBG client/internal/peer/conn.go:750: OnRemoteOffer from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= on status Disconnected
2024-05-20T07:17:49Z DEBG client/internal/peer/conn.go:756: OnRemoteOffer skipping message from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= on status Disconnected because is not ready
2024-05-20T07:17:49Z DEBG client/internal/peer/conn.go:280: trying to connect to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:17:49Z DEBG client/internal/peer/conn.go:312: connection offer sent to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=, waiting for the confirmation
2024-05-20T07:17:49Z TRAC signal/client/grpc.go:400: received a new message from Peer [fingerprint: e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=]
2024-05-20T07:17:49Z DEBG client/internal/peer/conn.go:765: OnRemoteAnswer from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= on status Disconnected
2024-05-20T07:17:49Z DEBG client/internal/peer/conn.go:333: received connection confirmation from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= running version 0.27.7 and with remote WireGuard listen port 51820
2024-05-20T07:17:49Z DEBG client/internal/peer/conn.go:663: peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= ICE ConnectionState has changed to Checking
2024-05-20T07:17:49Z DEBG client/internal/peer/conn.go:622: discovered local candidate udp4 host 10.110.127.61:51820
2024-05-20T07:17:49Z DEBG client/internal/peer/conn.go:622: discovered local candidate udp4 host 192.168.4.22:51820
2024-05-20T07:17:49Z DEBG client/internal/peer/conn.go:622: discovered local candidate udp4 srflx 176.66.87.43:43632 related 0.0.0.0:51820
2024-05-20T07:17:49Z TRAC signal/client/grpc.go:400: received a new message from Peer [fingerprint: e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=]
2024-05-20T07:17:49Z DEBG client/internal/peer/conn.go:779: OnRemoteCandidate from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= -> udp4 host 172.17.0.2:51820
2024-05-20T07:17:49Z DEBG iface/bind/udp_mux.go:346: ICE: registered 172.17.0.2:51820 for qsiaqUByKTiKZpLW
2024-05-20T07:17:49Z DEBG iface/bind/udp_mux.go:346: ICE: registered 172.17.0.2:51820 for qsiaqUByKTiKZpLWstun:my.domain.com:3478
2024-05-20T07:18:01Z DEBG client/internal/peer/conn.go:663: peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= ICE ConnectionState has changed to Failed
2024-05-20T07:18:01Z DEBG client/internal/peer/conn.go:519: trying to cleanup e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:18:01Z DEBG iface/iface.go:88: Removing peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= from interface utun7 
2024-05-20T07:18:01Z DEBG client/internal/peer/conn.go:663: peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= ICE ConnectionState has changed to Closed
2024-05-20T07:18:01Z DEBG client/internal/peer/conn.go:577: cleaned up connection to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:18:01Z DEBG client/internal/engine.go:913: connection to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= failed: connecting canceled by caller
2024-05-20T07:18:03Z TRAC signal/client/grpc.go:400: received a new message from Peer [fingerprint: e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=]
2024-05-20T07:18:03Z DEBG client/internal/peer/conn.go:750: OnRemoteOffer from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= on status Disconnected
2024-05-20T07:18:03Z DEBG client/internal/peer/conn.go:756: OnRemoteOffer skipping message from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= on status Disconnected because is not ready
2024-05-20T07:18:03Z DEBG client/internal/peer/conn.go:280: trying to connect to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:18:03Z DEBG client/internal/peer/conn.go:312: connection offer sent to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=, waiting for the confirmation
2024-05-20T07:18:03Z TRAC signal/client/grpc.go:400: received a new message from Peer [fingerprint: e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=]
2024-05-20T07:18:03Z DEBG client/internal/peer/conn.go:765: OnRemoteAnswer from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= on status Disconnected
2024-05-20T07:18:03Z DEBG client/internal/peer/conn.go:333: received connection confirmation from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= running version 0.27.7 and with remote WireGuard listen port 51820
2024-05-20T07:18:03Z DEBG client/internal/peer/conn.go:663: peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= ICE ConnectionState has changed to Checking
2024-05-20T07:18:03Z DEBG client/internal/peer/conn.go:622: discovered local candidate udp4 host 10.110.127.61:51820
2024-05-20T07:18:03Z DEBG client/internal/peer/conn.go:622: discovered local candidate udp4 host 192.168.4.22:51820
2024-05-20T07:18:03Z DEBG client/internal/peer/conn.go:622: discovered local candidate udp4 srflx 176.66.87.43:43632 related 0.0.0.0:51820
2024-05-20T07:18:03Z TRAC signal/client/grpc.go:400: received a new message from Peer [fingerprint: e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=]
2024-05-20T07:18:03Z DEBG client/internal/peer/conn.go:779: OnRemoteCandidate from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= -> udp4 host 172.17.0.2:51820
2024-05-20T07:18:03Z DEBG iface/bind/udp_mux.go:346: ICE: registered 172.17.0.2:51820 for FsnhFfNmMIHDJewp
2024-05-20T07:18:03Z DEBG iface/bind/udp_mux.go:346: ICE: registered 172.17.0.2:51820 for FsnhFfNmMIHDJewpstun:my.domain.com:3478
2024-05-20T07:18:15Z DEBG client/internal/peer/conn.go:663: peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= ICE ConnectionState has changed to Failed
2024-05-20T07:18:15Z DEBG client/internal/peer/conn.go:519: trying to cleanup e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:18:15Z DEBG iface/iface.go:88: Removing peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= from interface utun7 
2024-05-20T07:18:15Z DEBG client/internal/peer/conn.go:663: peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= ICE ConnectionState has changed to Closed
2024-05-20T07:18:15Z DEBG client/internal/peer/conn.go:577: cleaned up connection to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:18:15Z DEBG client/internal/engine.go:913: connection to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= failed: connecting canceled by caller
2024-05-20T07:18:16Z TRAC signal/client/grpc.go:400: received a new message from Peer [fingerprint: e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=]
2024-05-20T07:18:16Z DEBG client/internal/peer/conn.go:750: OnRemoteOffer from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= on status Disconnected
2024-05-20T07:18:16Z DEBG client/internal/peer/conn.go:756: OnRemoteOffer skipping message from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= on status Disconnected because is not ready
2024-05-20T07:18:17Z DEBG client/internal/peer/conn.go:280: trying to connect to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:18:17Z DEBG client/internal/peer/conn.go:312: connection offer sent to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=, waiting for the confirmation
2024-05-20T07:18:17Z TRAC signal/client/grpc.go:400: received a new message from Peer [fingerprint: e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=]
2024-05-20T07:18:17Z DEBG client/internal/peer/conn.go:765: OnRemoteAnswer from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= on status Disconnected
2024-05-20T07:18:17Z DEBG client/internal/peer/conn.go:333: received connection confirmation from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= running version 0.27.7 and with remote WireGuard listen port 51820
2024-05-20T07:18:17Z DEBG client/internal/peer/conn.go:663: peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= ICE ConnectionState has changed to Checking
2024-05-20T07:18:17Z DEBG client/internal/peer/conn.go:622: discovered local candidate udp4 host 10.110.127.61:51820
2024-05-20T07:18:17Z DEBG client/internal/peer/conn.go:622: discovered local candidate udp4 host 192.168.4.22:51820
2024-05-20T07:18:17Z DEBG client/internal/peer/conn.go:622: discovered local candidate udp4 srflx 176.66.87.43:43632 related 0.0.0.0:51820
2024-05-20T07:18:17Z TRAC signal/client/grpc.go:400: received a new message from Peer [fingerprint: e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=]
2024-05-20T07:18:17Z DEBG client/internal/peer/conn.go:779: OnRemoteCandidate from peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= -> udp4 host 172.17.0.2:51820
2024-05-20T07:18:17Z DEBG iface/bind/udp_mux.go:346: ICE: registered 172.17.0.2:51820 for wcfOsAXiJBXKWKQx
2024-05-20T07:18:17Z DEBG iface/bind/udp_mux.go:346: ICE: registered 172.17.0.2:51820 for wcfOsAXiJBXKWKQxturn:my.domain.com:3478?transport=udp
2024-05-20T07:18:30Z DEBG client/internal/peer/conn.go:663: peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= ICE ConnectionState has changed to Failed
2024-05-20T07:18:30Z DEBG client/internal/peer/conn.go:519: trying to cleanup e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:18:30Z DEBG iface/iface.go:88: Removing peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= from interface utun7 
2024-05-20T07:18:30Z DEBG client/internal/peer/conn.go:663: peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= ICE ConnectionState has changed to Closed
2024-05-20T07:18:30Z DEBG client/internal/peer/conn.go:577: cleaned up connection to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:18:30Z DEBG client/internal/engine.go:913: connection to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= failed: connecting canceled by caller
2024-05-20T07:18:30Z DEBG client/internal/peer/conn.go:280: trying to connect to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:18:30Z DEBG client/internal/peer/conn.go:312: connection offer sent to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=, waiting for the confirmation
2024-05-20T07:18:31Z DEBG client/internal/routemanager/client.go:321: stopping watcher for network 0.0.0.0/0
2024-05-20T07:18:31Z DEBG client/internal/engine.go:428: removing all peer connections
2024-05-20T07:18:31Z DEBG client/internal/engine.go:440: removing peer from engine e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:18:31Z DEBG signal/client/grpc.go:389: stream canceled (usually indicates shutdown)
2024-05-20T07:18:31Z DEBG signal/client/grpc.go:164: signal connection context has been canceled, this usually indicates shutdown
2024-05-20T07:18:31Z DEBG management/client/grpc.go:249: disconnected from Management Service sync stream: rpc error: code = Canceled desc = context canceled
2024-05-20T07:18:31Z DEBG management/client/grpc.go:157: management connection context has been canceled, this usually indicates shutdown
2024-05-20T07:18:31Z DEBG client/internal/engine.go:638: stopped receiving updates from Management Service
2024-05-20T07:18:31Z DEBG client/internal/peer/conn.go:519: trying to cleanup e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:18:31Z DEBG iface/iface.go:88: Removing peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= from interface utun7 
2024-05-20T07:18:31Z DEBG client/internal/peer/conn.go:663: peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= ICE ConnectionState has changed to Closed
2024-05-20T07:18:31Z DEBG client/internal/peer/conn.go:571: error while updating peer's e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= state, err: peer doesn't exist
2024-05-20T07:18:31Z DEBG client/internal/peer/conn.go:574: failed to reset wireguard stats for peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=: peer doesn't exist
2024-05-20T07:18:31Z DEBG client/internal/peer/conn.go:577: cleaned up connection to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk=
2024-05-20T07:18:31Z DEBG client/internal/engine.go:913: connection to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= failed: connection to peer e6Rarf9LFHH0FhvQH+PlGDYZZCITcdU31Sk5S4itslk= has been closed
2024-05-20T07:18:31Z INFO client/internal/routemanager/manager.go:117: Routing cleanup complete
2024-05-20T07:18:31Z DEBG client/internal/engine.go:1178: removing Netbird interface utun7
2024-05-20T07:18:31Z INFO client/internal/engine.go:246: stopped Netbird Engine
2024-05-20T07:18:31Z INFO client/internal/connect.go:275: stopped NetBird client

I also read about TURN may be an issue. My "external-ip" is set! I tried: https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/ - with success:

Time | Type | Foundation | Protocol | Address | Port | Priority | URL (if present) | relayProtocol (if present)
-- | -- | -- | -- | -- | -- | -- | -- | --
0.007 | host | 0 | udp | 0e87545e-1fb7-497b-93fe-185c151f9948.local | 49951 | 126 \| 32512 \| 255 |   |  
0.012 | host | 3 | tcp | 0e87545e-1fb7-497b-93fe-185c151f9948.local | 9 | 125 \| 32704 \| 255 |   |  
0.013 | host | 0 | udp | 0e87545e-1fb7-497b-93fe-185c151f9948.local | 49952 | 126 \| 32512 \| 254 |   |  
0.014 | host | 3 | tcp | 0e87545e-1fb7-497b-93fe-185c151f9948.local | 9 | 125 \| 32704 \| 254 |   |  
0.141 | srflx | 1 | udp | PUB-IP(Peer) | 21504 | 100 \| 32543 \| 255 |   |  
0.141 | relay | 2 | udp | PUB-IP | 57606 | 5 \| 32543 \| 255 |   |  
0.157 | Done

From same network i can connect the peer successfull.

When coming from external there is a pfsense in between, may thats the issue? From my understanding TURN shold go "through" without any changes.

pascal-fischer commented 2 months ago

HI @Berndinox, do you have a chance to test this from any other external network that is not mobile? We had a case where the iOS app did not work with a certain mobile carrier. This way we know if the mobile network is the issue or something in your setup.

Berndinox commented 2 months ago

UPDATE: The Raspi is in my DMZ where i just allow 80/443 outbound via Forward-Proxy. What outgoing Ports do Clients need? I just find a list for the MGMT Parts: https://docs.netbird.io/about-netbird/faq

@pascal-fischer - thanks for your replay. I was able to test with different devices and was able to isolate the issue.

Devices: iPhone1, iPhone2, Windows Client, Raspberry Pi

I cann connect every devices with each other via Mobile or Wifi connection but never with the Raspberry Pi. So the issue seems to be on this specific device.

The stats on the afected raspi:

OS: linux/arm64
Daemon version: 0.27.7
CLI version: 0.27.7
Management: Connected
Signal: Connected
Relays: 0/2 Available
Nameservers: 0/0 Available
FQDN: home-raspbi.netbird.selfhosted
NetBird IP: 100.119.191.160/16
Interface type: Kernel
Quantum resistance: false
Routes: -
Peers count: 0/3 Connected

The Netbird shows the devices as connected.

Details on Raspi Agent:

Peers detail:
 iphone-bernd-comp.netbird.selfhosted:
  NetBird IP: 100.119.12.202
  Public key: xxxx
  Status: Disconnected
  -- detail --
  Connection type:
  Direct: false
  ICE candidate (Local/Remote): -/-
  ICE candidate endpoints (Local/Remote): -/-
  Last connection update: 9 seconds ago
  Last WireGuard handshake: -
  Transfer status (received/sent) 0 B/0 B
  Quantum resistance: false
  Routes: -
  Latency: 0s

 iphone-bernd-priv.netbird.selfhosted:
  NetBird IP: 100.119.87.208
  Public key: xxxxxx
  Status: Disconnected
  -- detail --
  Connection type:
  Direct: false
  ICE candidate (Local/Remote): -/-
  ICE candidate endpoints (Local/Remote): -/-
  Last connection update: 24 seconds ago
  Last WireGuard handshake: -
  Transfer status (received/sent) 0 B/0 B
  Quantum resistance: false
  Routes: -
  Latency: 0s

 thinkpad.netbird.selfhosted:
  NetBird IP: 100.119.142.139
  Public key: xxxxx
  Status: Connecting
  -- detail --
  Connection type:
  Direct: false
  ICE candidate (Local/Remote): -/-
  ICE candidate endpoints (Local/Remote): -/-
  Last connection update: 4 seconds ago
  Last WireGuard handshake: -
  Transfer status (received/sent) 0 B/0 B
  Quantum resistance: false
  Routes: -
  Latency: 0s

OS: linux/arm64
Daemon version: 0.27.7
CLI version: 0.27.7
Management: Connected to https://connect.xxx.onl:443
Signal: Connected to https://connect.xxx.onl:443
Relays:
  [stun:connect.xxx.onl:3478] is Unavailable, reason: stun request: context deadline exceeded
  [turn:connect.xxx.onl:3478?transport=udp] is Unavailable, reason: allocate: all retransmissions failed for ID-replaced
Nameservers:
FQDN: home-raspbi.netbird.selfhosted
NetBird IP: 100.119.191.160/16
Interface type: Kernel
Quantum resistance: false
Routes: -
Peers count: 0/3 Connected

However, TURN config seems to be fine (see above msg)

Logs look normal (DNS not configured yet, but should not be required for TURN)

2024-05-21T19:49:20+02:00 INFO management/client/grpc.go:147: connected to the Management Service stream
2024-05-21T19:49:20+02:00 WARN client/internal/engine.go:551: running SSH server is not permitted
2024-05-21T19:49:20+02:00 ERRO client/internal/dns/server.go:322: unable to configure DNS for this peer using resolvconf manager without a nameserver group with all domains configured
2024-05-21T19:49:20+02:00 INFO client/internal/acl/manager.go:52: ACL rules processed in: 2.420723ms, total rules count: 0
2024-05-21T19:50:33+02:00 WARN client/internal/engine.go:551: running SSH server is not permitted
2024-05-21T19:50:33+02:00 INFO client/internal/acl/manager.go:52: ACL rules processed in: 1.976112ms, total rules count: 0
2024-05-21T19:51:06+02:00 WARN client/internal/engine.go:551: running SSH server is not permitted
2024-05-21T19:51:06+02:00 ERRO client/internal/dns/server.go:322: unable to configure DNS for this peer using resolvconf ma
Berndinox commented 2 months ago

@pascal-fischer - Solved, sorry for the inconvenience... For reference, if someone comes arround. Outbound: UDP and TCP ports 3478, and UDP/TCP ports 49152-65535.