search

tomeshnet / documents

Meeting notes and other important planning documents for Toronto Mesh
34 stars 13 forks source link

Tech - YGGDRASIL/CJDNS mulit hop test #116

Open darkdrgn2k opened 5 years ago

darkdrgn2k commented 5 years ago

Document Yggdrsaill vs CJDNS testing

Logical map of nodes (CJDNS addresses) connected for MESHPOINT and signal strengths between them image

Results

CJDNS/YGGDRASIL with 802.11s routeing disabled This is the "standard" mode for Prototype peering. CJDNS/YGGDRASIL will see multiple hops if peers are out of range and direct hops if they are in range.

image

CJDNS/YGGDRASIL with 802.11s routeing enabled

Enabliing 802.11s routing, the 802.11s protocol will shuttle the packets/frames around based on its routing protocol.

CJDNS/YGGRASILL see a flat 1 hop network to all nodes even though some paths may actually be multi hopped. 802.11s deasl with the multi hops

image

makew0rld commented 5 years ago

Could explain these a bit more, to help me understand what they mean?

darkdrgn2k commented 5 years ago

Its a brain dump of stats that need to be forumalted into a document

i added some context to the image above

Left side if CJDNS right side YGGDRASILL pi addresse in the same order (ie d45b is the same node as bec7)

One thing that stands out is that CJDNS suffered for peer c7eb because of the poor link quality. (in the map it was multi hop but at times it would be direct but over a verry poor connection) but yggdrasill did not. 80211s made cjdns work allot bettr with that peer

makew0rld commented 5 years ago

Thanks. And it looks like Yggdrasil did a bit better without the 802.11s routing?

darkdrgn2k commented 5 years ago

Thanks. And it looks like Yggdrasil did a bit better without the 802.11s routing?

I'd call it rounding error

second test was not as long as the first one

makew0rld commented 5 years ago

So it seems the conclusion is that we should always enable 802.11s routing on our nodes? And that Yggdrasil is probably a better choice for most things at the moment?

Arceliar commented 5 years ago

I would expect that turning on 802.11s forwarding should help a lot more often than it hurts. If I remember right, it's limited to ~32 nodes by default, so it shouldn't run into a protocol congestion wall the same way that other things could, as long as it handles the 33rd node gracefully (maybe it keeps the closest 32 nodes or something). And I would sincerely hope that 802.11s forwarding knows enough about the wireless state to do a better job of routing than ygg's simple feedback mechanism's (EDIT: if there's a difference, it may become more visible in a large network, but I guess that's even harder to test) or whatever state cjdns is tracking (it's not clear to me if a supernode is reachable to help with routing decisions, or if they're falling back to the old dht-based pathfinder code). I'm honestly surprised that ygg without 802.11s forwarding manages to do so well compared to when forwarding is enabled. Turning on 802.11s forwarding should lead to slightly more idle traffic, from more broadcast cjdns beacons and ygg multicast announcements, but I don't think this would matter except on devices running on battery, which may have issues regardless.

Some additional tests to consider and reasons why, roughly in increasing order of how annoying it would be to set up:

  1. ping over 802.11s+forwarding, just as a baseline for comparison against cjdns/ygg.
  2. iperf3 over these things, to see how bandwidth compares, how stable it is, etc; I imagine ygg will look less stable on the sender's side than on the receiver's, due to how some of the buffering stuff works internally, but cjdns and normal IP should both be pretty stable on both ends (+- whatever's happening when cjdns has 802.11s forwarding disabled). I guess there could be CPU bottlenecks on cjdns, depending on how the hardware and how good the wireless connections are, in which case this test would be less useful.
  3. ping with iperf3 running in the background, to see how much latency and packet loss are affected by congestion... this is a test where I expect yggdrasil to do appreciably worse than the others, since TCP buffers force some latency upon us when things are congested. There are different congestion control schemes you can try that should mitigate this (BBR especially), but that's probably beyond the scope of these tests.
  4. repeat all these things with batman-adv over 802.11s (a common freifunk setup, if I'm not mistaken)
  5. repeat with cjdns/ygg over batman-adv over 802.11s, I would expect this to look a lot like running them over 802.11s+forwarding
  6. repeat babel over 802.11s (should be a close approximation of Althea)
  7. repeat with cjdns/ygg over babel over 802.11s, I would (again) expect this to look like running them with 802.11s+forwarding