netbirdio / netbird

Connect your devices into a secure WireGuard®-based overlay network with SSO, MFA and granular access controls.
https://netbird.io
BSD 3-Clause "New" or "Revised" License
10.61k stars 474 forks source link

NetBird fails to forward traffic from Oracle Cloud VM to internal peer, works fine with Tailscale #2451

Open fedeiglesias opened 3 weeks ago

fedeiglesias commented 3 weeks ago

Describe the problem

I'm facing an issue where NetBird does not properly forward traffic from an Oracle Cloud VM to a peer in my internal network. I followed the exact same steps with Tailscale, and everything works fine under the same conditions. However, with NetBird, the traffic does not reach its destination.

I have an Oracle Cloud VM running Ubuntu (Ampere ARM64, 4 cores, 24 GB RAM) with a static public IP. On this VM, I installed NetBird from scratch, logged into my cloud account, and have a policy that allows all peers to connect to each other.

In my homelab, I have an LXC container running uptimekuma service on port 3001. The internal NetBird IP of this peer is 100.88.192.94. If I run a curl command from the Oracle VM, I can retrieve the HTML response from the NGINX service, so the internal connection between peers is working.

The issue arises when I try to forward the traffic coming to the VM on port 80 to the internal NetBird peer. I configured traffic forwarding using iptables, but it doesn't seem to work with NetBird, whereas with Tailscale, the traffic is correctly forwarded without issues.

To Reproduce

Steps to reproduce the behavior:

  1. Create an Oracle Cloud VM (Ampere ARM64, Ubuntu minimal).
  2. Install NetBird and set up a peer in the internal network (in this case, an LXC container running an NGINX service).
  3. On the Oracle VM, enable traffic forwarding and configure iptables to forward traffic on port 80 to the NetBird peer on port 3001.
  4. Try to access the web service through the Oracle Cloud public IP.

Expected behavior

I expected the incoming traffic on port 80 of the Oracle Cloud VM to be correctly forwarded to the internal NetBird peer, as it happens with Tailscale under the same conditions.

Are you using NetBird Cloud?

Yes, I am using NetBird Cloud.

NetBird version

0.28.7

NetBird status -dA output

NetBird status -dA output ubuntu@oracle:~$ netbird status -dA output Peers detail: iphone-fede.netbird.cloud: NetBird IP: 100.88.5.103 Public key: WKZknsTPHEZQREt75Kmaz/HV2jwcNehnpt0S91PKH1c= Status: Connected -- detail -- Connection type: P2P Direct: true ICE candidate (Local/Remote): srflx/srflx ICE candidate endpoints (Local/Remote): 198.51.100.0:51820/198.51.100.1:57557 Last connection update: 22 minutes, 50 seconds ago Last WireGuard handshake: 1 minute, 48 seconds ago Transfer status (received/sent) 3.8 KiB/1.6 KiB Quantum resistance: false Routes: - Latency: 22.578577ms

oracle.netbird.cloud: NetBird IP: 100.88.61.146 Public key: PGT03R1EDb1cAf+uZymgvGYFXKHnnOP3/0ccKPInfBI= Status: Disconnected -- detail -- Connection type: Direct: false ICE candidate (Local/Remote): -/- ICE candidate endpoints (Local/Remote): -/- Last connection update: - Last WireGuard handshake: - Transfer status (received/sent) 0 B/0 B Quantum resistance: false Routes: - Latency: 0s

4da8810ea346.netbird.cloud: NetBird IP: 100.88.110.131 Public key: +AawwZqEzKesGbPISiXgeU0yfTfmtkGKXGs0v7U152s= Status: Disconnected -- detail -- Connection type: Direct: false ICE candidate (Local/Remote): -/- ICE candidate endpoints (Local/Remote): -/- Last connection update: - Last WireGuard handshake: - Transfer status (received/sent) 0 B/0 B Quantum resistance: false Routes: - Latency: 0s

macbook-fede.netbird.cloud: NetBird IP: 100.88.180.38 Public key: bQZuQnpveGtcU45nr7DTJtlWbhqi6O7rj/UwkNcB1iA= Status: Connected -- detail -- Connection type: P2P Direct: true ICE candidate (Local/Remote): srflx/srflx ICE candidate endpoints (Local/Remote): 198.51.100.0:51820/198.51.100.1:57431 Last connection update: 22 minutes, 53 seconds ago Last WireGuard handshake: 2 minutes, 3 seconds ago Transfer status (received/sent) 3.6 KiB/1.2 KiB Quantum resistance: false Routes: - Latency: 14.533849ms

uptimekuma.netbird.cloud: NetBird IP: 100.88.192.94 Public key: hA3DLsvCah9Gz6YeWWWRGEBtCusBhadDopDOom0a2Qs= Status: Connected -- detail -- Connection type: P2P Direct: true ICE candidate (Local/Remote): srflx/srflx ICE candidate endpoints (Local/Remote): 198.51.100.0:51820/198.51.100.1:58213 Last connection update: 22 minutes, 53 seconds ago Last WireGuard handshake: 2 minutes, 25 seconds ago Transfer status (received/sent) 3.0 KiB/2.1 KiB Quantum resistance: false Routes: - Latency: 20.015482ms

nginx.netbird.cloud: NetBird IP: 100.88.211.126 Public key: 9kpbqwyghDtjVPX92Ds0EF054+RzOMMw8/+efConO2I= Status: Disconnected -- detail -- Connection type: P2P Direct: false ICE candidate (Local/Remote): srflx/srflx ICE candidate endpoints (Local/Remote): 198.51.100.0:51820/198.51.100.1:57104 Last connection update: - Last WireGuard handshake: 1 minute, 48 seconds ago Transfer status (received/sent) 3.8 KiB/1.3 KiB Quantum resistance: false Routes: - Latency: 0s

pihole.netbird.cloud: NetBird IP: 100.88.212.19 Public key: gmSdBrNEI2j5fNwAWHRGyYhMAJUjTizb2HVqHEhq20Y= Status: Connected -- detail -- Connection type: P2P Direct: true ICE candidate (Local/Remote): srflx/srflx ICE candidate endpoints (Local/Remote): 198.51.100.0:51820/198.51.100.1:57104 Last connection update: 22 minutes, 53 seconds ago Last WireGuard handshake: 1 minute, 48 seconds ago Transfer status (received/sent) 3.8 KiB/1.3 KiB Quantum resistance: false Routes: - Latency: 9.531658ms

OS: linux/arm64 Daemon version: 0.28.7 CLI version: 0.28.7 Management: Connected to https://api.netbird.io:443 Signal: Connected to https://signal.netbird.io:443 Relays: [stun:stun.netbird.io:5555] is Available [turns:turn.netbird.io:443?transport=tcp] is Available Nameservers: [100.88.212.19:53] for [local.anon-are26.domain] is Available FQDN: oracle-1.netbird.cloud NetBird IP: 100.88.223.175/16 Interface type: Kernel Quantum resistance: false Routes: - Peers count: 4/7 Connected

Do you face any client issues on desktop?

No desktop client is involved, only the Oracle VM client and the peer in my homelab (LXC).

Screenshots

CleanShot 2024-08-19 at 11  29 13@2x

Additional context

Here are the commands I used to configure port forwarding:

I have ensured that port 80 is open and accessible from the Oracle Cloud dashboard. As I mentioned before, this exact setup works fine with Tailscale, which makes me think this could either be a bug in NetBird or a misconfiguration on my side.

lixmal commented 3 weeks ago

Hi @fedeiglesias,

could you test if this build fixes the issue? https://github.com/netbirdio/netbird/actions/runs/10419607061/artifacts/1836477728

You can replace /usr/bin/netbird with the new binary on the oracle VM

ghaisasadvait commented 3 weeks ago

I'm having the same issue. As soon as the peer is connected, i cannot access the web server hosted on it via the public IP

@fedeiglesias did you find any solution to this?