Wireless link test bed - Githubissues

This is a summary of a recent discussion on #deployment:tomesh.net involving @darkdrgn2k @Pedro-on-a-bike and @Shrinks99. The short term plan is to put up directional Ubiquiti antennas to form a stable 100+ Mbps point-to-point wireless link between two venues. Connect each one to a gigabit switch, so the assembly can serve as a test bed for plugging in single-board computer nodes running various mesh software (e.g. cjdns, althea, batman-adv, tomesh's prototype, mesh-orange, etc.) and they can run in parallel sharing the wireless link.

We want to think of setting up this initial link, and further extending this radio network, as a separate project from the ongoing work on node software development. This is a shared resource to be housed in donated space, and will serve as test bed for running our semi-permanent nodes in a monitored environment.

The basic test bed setup consists only of one or more Ubiquiti antenna and a gigabit switch. We would like this equipment to be accessible to Toronto Mesh members such that we can plug in other devices to test using this shared link.

The following should be discussed with potential partners donating space to host our test bed setups:

Roof or window space with access to power and other cables
Securing the equipment (from theft or accidentally getting unplugged)
Rules for individuals to access the equipment (if we need to plug something in what's the process?)
Possibly offering their venue Internet exit traffic (for private testing by Toronto Mesh members, not taking on the risk of being a public Internet exit)
On-premise advertising to let people know the space is hosting test equipment for Toronto Mesh

There was also an idea of running a monitoring screen at the donated space. This may be a great way to get tech people to notice that there is something exciting happening there and study the information provided on it.

After discussion with @Pedro-on-a-bike and @darkdrgn2k, I would like to recommend this set up for home user installations as well:

+-------------+
| LiteBeam AC |
| with AirOS  |
+--------+----+
         |                 [other radios]
         |                       |
+--------+-----------------------+---------------+
| 5-port Switch                                  |
+------+----------------+----------------+-------+
       |                |                |
 (for Internet)    (for cjdns)    [other routers]
       |                |
+------+------+ +-------+--------+
| EdgeRouterX | | Orange Pi Zero |
| with Althea | | with Prototype |
+-+-----------+ +-------+--------+
  |                     =
  |                    ===
+-+-------------+     =====
| Home Router   |  On-board WiFi
| Internet WiFi |  to cjdns Network
+---------------+

Devices:

Radio, one of:

Ubiquiti LiteBeam AC 16 dBi running AirOS $110
- Configured as AirMAX Access Point, capable of both P2P and P2MP
Ubiquiti Nanostation AC running AirOS $63
- Configured as AirMAX Station, connects to an AirMAX Access Point

Network switch:

D-Link GO-SW-5G switch $20
- Allow multiple radios and mesh devices to be plugged in

For internet-sharing (althea-babel):

Ubiquiti EdgeRouterX $70

For in-mesh peer-to-peer applications (ipfs, ssb over cjdns, yggdrasil):

Orange Pi Zero $10
Raspberry Pi 3 $60
[other single-board computers]

Notes:

Verified with @jkilpatr that Althea will work in this setup although traffic among switched radios will be unmetered as they will never pass through the EdgeRouterX
A setup here ranges from $150-$300 depending on use case, we hope this can be subsidized to < $100 through grants in the future
Costs of these hardware will drop to below $100 unsubsidized when projects like LibreRouter ship and SBC options open up

Organizational tasks:

[ ] Website to explain this setup and options for different use cases
[ ] At each monthly we demo each configuration
[ ] Have a go-to person for each option
- @Pedro-on-a-bike for Althea
- @darkdrgn2k for SBC setups

Please 👍 or 👎 to this proposal and sign up for the website task.

Original Problem: How do you share 2 antennas between two boards Original Proposed Solution: Connect all hardware to a switch to share resources Issue: This may work small scale but as it grows it will be a problem

New Problem: Both CJDNS and Althea find peers on their local networks (CJDNS via layer 2 beacons, Althea via layer 3 broadcast packets .

ISSUE only occurs when you need to share 2 or more antennas. It is not an issue with 1 antenna and a simple switch will be fine

Illustration of Problem:

Diagram shows different configurations under proposed solution Switch 1 and 4 - 1 antenna 2 boards shared via switch Switch 2 and 3 - 2 antennas 2 boards shared via switch diagram 1

The antennas are in bridge mode. This means that a simplified version of the diagram could remove the antennas and their links leaving only the switches behind. (There is still routing happening on layer 2 between switches but we will ignore that for now) diagram 2

Which could further be simplified into just one switch. diagram 3 Once the topology map is simplified we can see that the whole wireless network is one broadcast domain, and all devices seem to be directly connected.

If we review the original diagram we see the top left Althea node should be 3 hops away from the bottom right Althea node.

Issues:

Both Althea and CJDNS will see the whole network as 1 hop away
- CJDNS routes by best hop and will send directly to the requested node
- Althea SHOULD route best possible path, but it's unclear how this will work
On a small installation this would not really be a problem
Althea payment nodes can be bypassed
Larger broadcast domains will create larger overhead and possibly attack vector - broadcast flood

Solution to broadcast domain problem

By isolating each switch on its own broadcast domain, and using the non-bridged interfaces on the Boards, we can prevent the antennas from bridging over each other while still allowing to share the connection. This can be done in one of two ways

Option 1 - Separate physical switches

Connect a separate physical switch to each antenna and connecting and each board to each antenna using a separate port on the Boards

diagram 4

Pros

Unmanaged switches are cheaper
Less configuration required Cons
More items need to be plugged into the power
Antennas limited to the number of ports on the Board
Raspberry Pi only has 1 interface, so it may need USB network interfaces
More cabling
More space

Option 2 - Managed Switch A similar setup as Option 1 can be setup without using two physical separate switch using VLANS

VLAN on swithes - quick overview UNTAGGED PACKET - packet that has no knowledge of VLANs at all. UNTAGGED PORT - configured on the switch, may have only 1 vlan assigned to the port. Only packets tagged inside the switch as this port's configured vlan will be allowed to leave the switch through here (after being stripped of its tag to become untagged again). Likewise any packets coming in on this interface will be given the tag of the VLAN that is configured on this port. Each group of untagged ports with the same vlan defined can be though of as a separate physical switch TAGGED PACKET - packet that is labeled on which VLAN it is on TAGGED PORT (or trunk port) - This is a port that allows the packets to retain their tagging as it leaves the switch on this port. You can configure multiple VLANs to leave the switch through this ports (and as such limit which ones can). The device on the other end will require to deal with the tagged packets. Likewise packets coming into the switch must have be TAGGED by the correct vlan before being sent to the switch.

NOTE many devices have ability to "mix" both types of modes. Untagged packets are processed as Untagged and Tagged as Tagged.

Solution: Single managed switch. The antennas connect to it on an UNTAGGED VLAN

Board connect in one of to ways

separate UNTAGGED port on the switch for each antenna to a separate UNTAGGED port for each antenna the board
Single TRUNK port on the switch to single TRUNK port on the router in an with TAGGED packets

diagram 5

tomeshnet / deployment

Wireless link test bed #2