search

netbirdio / netbird

Connect your devices into a secure WireGuard®-based overlay network with SSO, MFA and granular access controls.
https://netbird.io
BSD 3-Clause "New" or "Revised" License
9.93k stars 434 forks source link

Cannot reinstall client #1511

Open PavelNiedoba opened 5 months ago

PavelNiedoba commented 5 months ago

I'm learning selfhosted netbird. I have 4 peers, which were working fine, then I tried to add route to lan. My client stopped working so I tried to reinstall it. Nothing worked so I tried to apt prune netbird netbird-io I removed all content from /etc/netbird. I rebooted notebook. I removed all routes from web-gui.

image

I deleted corresponding peer

Then I did like always curl -fsSL https://pkgs.netbird.io/install.sh | sh and then netbird up --management-url https://vpn.xx.eu

I put the link from command line into browser and logged in with success. I saw connected on command line.

I see green light: image

When i try to ping other peers:

ping 100.76.64.117
PING 100.76.64.117 (100.76.64.117) 56(84) bytes of data.
From 100.76.141.7 icmp_seq=1 Destination Host Unreachable
ping: sendmsg: Required key not available
From 100.76.141.7 icmp_seq=2 Destination Host Unreachable
ping: sendmsg: Required key not available
From 100.76.141.7 icmp_seq=3 Destination Host Unreachable
ping: sendmsg: Required key not available

in /var/log/netbird/client.log

2024-01-30T21:18:59+01:00 INFO client/internal/config.go:115: generating new config /etc/netbird/config.json
2024-01-30T21:18:59+01:00 WARN client/server/server.go:134: failed login: rpc error: code = InvalidArgument desc = invalid setup-key or no sso information provided, err: invalid UUID length: 0
2024-01-30T21:19:28+01:00 INFO client/internal/login.go:130: peer has been successfully registered on Management Service
2024-01-30T21:19:28+01:00 INFO client/internal/connect.go:94: starting NetBird client version 0.25.5
2024-01-30T21:19:28+01:00 INFO client/internal/wgproxy/proxy_ebpf.go:79: local wg proxy listening on: 3128
2024-01-30T21:19:28+01:00 INFO signal/client/grpc.go:157: connected to the Signal Service stream
2024-01-30T21:19:28+01:00 INFO client/internal/connect.go:231: Netbird engine started, my IP is: 100.76.141.7/16
2024-01-30T21:19:28+01:00 INFO management/client/grpc.go:143: connected to the Management Service stream
2024-01-30T21:19:28+01:00 INFO client/internal/dns/systemd_linux.go:135: adding 1 search domains and 0 match domains. Search list: [netbird.selfhosted] , Match list: []
2024-01-30T21:19:28+01:00 INFO client/internal/acl/manager.go:52: ACL rules processed in: 1.063256ms, total rules count: 2

in /var/log/netbird/netbird.out

turnc ERROR: 2024/01/30 05:13:58 Fail to refresh permissions: CreatePermission error response (error 400: Bad Request.)
turnc ERROR: 2024/01/30 05:15:58 Fail to refresh permissions: CreatePermission error response (error 400: Bad Request.)
turnc ERROR: 2024/01/30 05:17:58 Fail to refresh permissions: CreatePermission error response (error 400: Bad Request.)
turnc ERROR: 2024/01/30 05:19:58 Fail to refresh permissions: CreatePermission error response (error 400: Bad Request.)
turnc ERROR: 2024/01/30 05:21:58 Fail to refresh permissions: CreatePermission error response (error 400: Bad Request.)
turnc ERROR: 2024/01/30 05:23:58 Fail to refresh permissions: CreatePermission error response (error 400: Bad Request.)

wg show shows no peer

interface: wt0
  public key: yRX1w9l1h0NXwrZcHBmbxs9utdQ0KmVVZ3NoPBZ5Niw=
  private key: (hidden)
  listening port: 51820

I'm out of ideas, please point me somewhere

PavelNiedoba commented 5 months ago

I cannot get netbird network (the one starting with 100.) to work with nodes without public IP. I have multiple private networks. Each one have router with public IP where I can configure port forward. Inside those networks I have clients which I want to be available within netbird network. I cannot find documentation how to do that. Is there paid support?

PavelNiedoba commented 5 months ago

I have little progress. It's not client problem. Restarting server containers fixed the client problems. I did not expect it was server problem, because other peers worked fine. Message ping: sendmsg: Required key not available appears when there is no direct wireguard connection. There is working tunnel which should be used instead as printed from netbird status -d

Daemon version: 0.25.5
CLI version: 0.25.5
Management: Connected to https://xxx:443
Signal: Connected to https://vpn.xxx:443
Relays: 
  [stun:vpn.xxx:3478] is Available
  [turn:vpn.xxx:3478?transport=udp] is Available
FQDN: xxx
NetBird IP: 100.76.131.206/16
Interface type: Kernel
Peers count: 6/6 Connected

Restart helped, but one peer stopped working again after 5 hours of operation. I hope this will help somebody with similar problems.

bcmmbaga commented 5 months ago

Hello @PavelNiedoba, If you're looking to route your traffic to your LAN, we've provided comprehensive guidelines in our Documentation. These guidelines explain how to effectively use a routing peer to direct your traffic to your LAN. If you have any further questions or need assistance, feel free to reach out!

PavelNiedoba commented 5 months ago

@bcmmbaga Thank you for pointing me to the guidelines. I red it again and there's nothing what would be helpful mitigating ping: sendmsg: Required key not available which occurs periodically after some some time or changes made. I'm missing any explanation how tunnels work between peer behind NAT. I have situation when I have 2 nodes, each can ping netbird server but they can not ping each other. I can see thousands of errors in netbird.out turnc ERROR: 2024/02/05 13:19:51 Fail to refresh permissions: CreatePermission error response (error 400: Bad Request.)

On node behind nat I did netbird service stop, deleted logs and started it again. Client log has this errors:

2024-02-05T13:26:20+01:00 INFO client/cmd/service_controller.go:23: starting Netbird service
2024-02-05T13:26:20+01:00 INFO client/cmd/service_controller.go:63: started daemon server: /var/run/netbird.sock
2024-02-05T13:26:20+01:00 INFO client/internal/connect.go:94: starting NetBird client version 0.25.5
2024-02-05T13:26:20+01:00 INFO client/internal/wgproxy/proxy_ebpf.go:79: local wg proxy listening on: 3128
2024-02-05T13:26:20+01:00 INFO signal/client/grpc.go:157: connected to the Signal Service stream
2024-02-05T13:26:20+01:00 INFO client/internal/connect.go:231: Netbird engine started, my IP is: 100.76.195.37/16
2024-02-05T13:26:20+01:00 INFO management/client/grpc.go:143: connected to the Management Service stream
2024-02-05T13:26:20+01:00 WARN client/internal/routemanager/client.go:121: the network 10.1.1.0/24 has not been assigned a routing peer as no peers from the list [MIuSE46MWGqdtTfg4BpSb1kEP1SeZtQUN3p1KT7CiAI
2024-02-05T13:26:20+01:00 WARN client/internal/routemanager/client.go:121: the network 10.2.0.0/24 has not been assigned a routing peer as no peers from the list [mTlN3RCRl84uN3UPzpfVIQpcjEfnEeUuHNvby7CUKBo
2024-02-05T13:26:20+01:00 WARN client/internal/routemanager/client.go:121: the network 10.13.1.0/24 has not been assigned a routing peer as no peers from the list [RXW+rsqK1bIQxtzVvCYJF+U7TT8t3kFil2/LPQ6Gly
2024-02-05T13:26:20+01:00 ERRO client/internal/dns/server.go:292: unable to configure DNS for this peer using file manager without a nameserver group with all domains configured
2024-02-05T13:26:20+01:00 WARN client/internal/dns/upstream.go:169: probing upstream nameserver 100.76.252.88:53: write udp 100.76.195.37:46206->100.76.252.88:53: write: operation not permitted
2024-02-05T13:26:20+01:00 WARN client/internal/dns/upstream.go:169: probing upstream nameserver 100.76.252.88:53: write udp 100.76.195.37:44464->100.76.252.88:53: write: operation not permitted
2024-02-05T13:26:20+01:00 INFO client/internal/acl/manager.go:52: ACL rules processed in: 4.36098ms, total rules count: 2
2024-02-05T13:26:22+01:00 INFO client/internal/peer/conn.go:357: connected to peer RXW+rsqK1bIQxtzVvCYJF+U7TT8t3kFil2/LPQ6GlyY=, endpoint address: 10.13.0.1:57105
2024-02-05T13:26:22+01:00 INFO client/internal/routemanager/client.go:124: new chosen route is cmtcfn0bv9rc73brudl0 with peer RXW+rsqK1bIQxtzVvCYJF+U7TT8t3kFil2/LPQ6GlyY= with score 2 for network 10.13.1.0/
2024-02-05T13:26:23+01:00 INFO client/internal/peer/conn.go:357: connected to peer MIuSE46MWGqdtTfg4BpSb1kEP1SeZtQUN3p1KT7CiAI=, endpoint address: 77.240.106.227:51820
2024-02-05T13:26:23+01:00 INFO client/internal/routemanager/client.go:124: new chosen route is cmtbs9gbv9rc73brudj0 with peer MIuSE46MWGqdtTfg4BpSb1kEP1SeZtQUN3p1KT7CiAI= with score 2 for network 10.1.1.0/2
2024-02-05T13:26:23+01:00 INFO client/internal/wgproxy/proxy_ebpf.go:91: turn conn added to wg proxy store: 10.13.0.101:51820, endpoint port: :1
2024-02-05T13:26:23+01:00 INFO client/internal/peer/conn.go:357: connected to peer xwe4hYuRkSo9zKBkm8T/EQ2cjf+i/8KKd+RlLPrOdxA=, endpoint address: 127.0.0.1:1
2024-02-05T13:26:30+01:00 WARN client/internal/routemanager/client.go:121: the network 10.13.1.0/24 has not been assigned a routing peer as no peers from the list [RXW+rsqK1bIQxtzVvCYJF+U7TT8t3kFil2/LPQ6Gly
2024-02-05T13:26:37+01:00 INFO client/internal/peer/conn.go:357: connected to peer RXW+rsqK1bIQxtzVvCYJF+U7TT8t3kFil2/LPQ6GlyY=, endpoint address: 10.13.0.1:57105
2024-02-05T13:26:37+01:00 INFO client/internal/routemanager/client.go:124: new chosen route is cmtcfn0bv9rc73brudl0 with peer RXW+rsqK1bIQxtzVvCYJF+U7TT8t3kFil2/LPQ6GlyY= with score 2 for network 10.13.1.0/
2024-02-05T13:26:43+01:00 WARN client/internal/routemanager/client.go:121: the network 10.13.1.0/24 has not been assigned a routing peer as no peers from the list [RXW+rsqK1bIQxtzVvCYJF+U7TT8t3kFil2/LPQ6Gly
2024-02-05T13:26:51+01:00 INFO client/internal/peer/conn.go:357: connected to peer RXW+rsqK1bIQxtzVvCYJF+U7TT8t3kFil2/LPQ6GlyY=, endpoint address: 10.13.0.1:57105
2024-02-05T13:26:51+01:00 INFO client/internal/routemanager/client.go:124: new chosen route is cmtcfn0bv9rc73brudl0 with peer RXW+rsqK1bIQxtzVvCYJF+U7TT8t3kFil2/LPQ6GlyY= with score 2 for network 10.13.1.0/
2024-02-05T13:26:57+01:00 WARN client/internal/routemanager/client.go:121: the network 10.13.1.0/24 has not been assigned a routing peer as no peers from the list [RXW+rsqK1bIQxtzVvCYJF+U7TT8t3kFil2/LPQ6Gly
2024-02-05T13:27:07+01:00 INFO client/internal/wgproxy/proxy_ebpf.go:91: turn conn added to wg proxy store: 10.13.1.2:51820, endpoint port: :2
2024-02-05T13:27:07+01:00 INFO client/internal/peer/conn.go:357: connected to peer RXW+rsqK1bIQxtzVvCYJF+U7TT8t3kFil2/LPQ6GlyY=, endpoint address: 127.0.0.1:2
2024-02-05T13:27:07+01:00 INFO client/internal/routemanager/client.go:124: new chosen route is cmtcfn0bv9rc73brudl0 with peer RXW+rsqK1bIQxtzVvCYJF+U7TT8t3kFil2/LPQ6GlyY= with score 0 for network 10.13.1.0/