search

MassMesh / mm-cli

A CLI application for connecting to the mesh. Written in Python.
https://massmesh.org
GNU General Public License v3.0
1 stars 0 forks source link

Weird routing problem reported by @wfleurant #3

Closed wfleurant closed 4 years ago

wfleurant commented 4 years ago

Mesh traffic goes through peers, then gateway, then to the peer. It should just go straight to the peer instead.

Can test this with mtr -6 or traceroute -6

wfleurant commented 4 years ago

@JamesVorder I think this was fixed when we discovered double peering (with the Yggdrasil interface) resulted in packet/session loss.

This double peering issue is fixed in Yggdrasil routing today. We once had a case where nodes attempting to connect to the Virtual ISP at an internet IP address would gain a CKR IP address via the local multicast Yggdrasil network. Once the default route of our node switched the default route to the Crypto Key Routing address of the CKR tunnel, Internet traffic flowed back and forth from the vISP. Meanwhile this node is continuously attempting to peer with the VISP gateway over the internet. It does, but through the CKR tunnel (at the gateway!). Internet traffic becomes stop and go.

The 2nd iteration attempt at correcting this by providing null routes and careful direct-peering in a few situations. The cases below are for the addroute(dest, gateway)

  1. When the node WAN is not connected with a defgw and we are replacing the default route 0.0.0.0 to the CKR Gateway IP. (mesh only CKR internet)
  2. When we call the addroute() function we always add a null route with a high cost.
  3. Finally if a node primary WAN installs a default gateway with DHCP, we add a /32 route over that WAN Gateway IP.

CKR .. Crypto Key Routing .. IP meta-networking over Yggdrasil (10.42.0.0/16) WAN .. Wide Area Net .. Internet acquired with the WAN USB Ethernet cable
Null routes .. Prevents Leaks and Yggdrasil peering over a CKR Def. Gateway

https://github.com/MassMesh/mm-cli/blob/d22fd981ed23992ebeba4891b277cc44197a21c2/mm_cli/util.py#L171

Direct peering often means establishing a peer session using Internet because we are dealing with the routing table. So we can connect with Yggdrasil peers by adding direct routes or /32 routes. Once the client provisioning cycle completes the last call is to setdefaultroute(gateway). Provisioning is constant due because a WAN Gateway can be introduced or removed at any time so we currently use crontab to maintain the node routing table. CKR traffic destined to the VISP gateway never leaks over the WAN. The WAN (if available) is used only for direct Yggdrasil peering.

note: I don't recall documenting early boot leak tests, but we should have already addressed this with OpenWrt firewall zones (LAN to WAN vs. LAN to MESH)

https://github.com/MassMesh/mm-cli/blob/d22fd981ed23992ebeba4891b277cc44197a21c2/mm_cli/util.py#L207