netbirdio / netbird

Connect your devices into a secure WireGuard®-based overlay network with SSO, MFA and granular access controls.
https://netbird.io
BSD 3-Clause "New" or "Revised" License
11.3k stars 518 forks source link

netbird doesn't seem to recover from network outage #1361

Closed TheRouce closed 11 months ago

TheRouce commented 11 months ago

Describe the problem I've had two network outages recently and both times I had to manually setup each device again. An outage happened today and the same thing has happened, the network is back up but all 10 of my nodes have not reconnected.

To Reproduce Steps to reproduce the behavior:

  1. Disconnect internet for at least a few hours, maybe up to 24 hours
  2. Reconnect internet

Expected behavior Netbird should automatically reconnect on the last network it was authorized

NetBird status -d output: I can comment the output when I am back at the server

Additional context If any tests need to be performed I can simulate an outage using proxmox or similar and try to retrieve logs

mlsmaycon commented 11 months ago

Hello @TheRouce, the peers would actively attempt to reconnect after network outages. It keeps retrying for 3 months.

What may cause an issue is a dependency on private DNS routed via the wireguard tunnel to resolve the domain name of your management services. If that is the case for you, you can add a nameserver configuration with a matching domain for your management services pointing to a public or a resolver independent from the tunnel.

If that is not the case, please follow the steps below to enable debug logs on one of the peers:

sudo netbird service stop
sudo netbird service uninstall
sudo netbird service install --log-level debug
sudo netbird service start

Then reproduce the outage on the proxmox host where this peers is running.

Once that is done, can you share the logs from the peer? they are located at /var/log/netbird

TheRouce commented 11 months ago

Sorry, forgot to close this. This ended up being related to an issue with our VPN accidentally being blocked. Not related to netbird.

mlsmaycon commented 11 months ago

Thanks for the feedback @TheRouce