Some Questions

[HOGGL]

Good work! I especially appreciate the importance placed on incentives. The private room idea is excellent. Reading though your RFC, some questions came to mind.

1. Although Wireguard allows for mesh networks, adding a node to a network of size N increases the number of edges by N. This is exponential and therefore how can it be scalable?

2. If I understand the RFC correctly, VPNs are used to manage friends and for NAT punch-through. But if Pleroma and PeerTube already have blocking capabilities and their respective protocols do NAT punch-through. Are VPNs strictly required?

3. The network segregation mechanism seems to lend itself more to Scuttlebutt than Pleroma. What is the reasoning for choosing Pleroma?

4. PeerTube is able to make multiple resolutions available although the Raspberry Pi 4 is only able to encode video up to 1080p30. This can be done during off-peak times but live streaming will be infeasible. What resolutions should we support? I think a good middle-road would be to provide a presentation mode where only pictures and audio are used. This could be useful in scientific discussions.

[meathunter]

For 4 and live feeds, you would only be encoding the video at the source, the rest would just be packet re-direction and/or multicast and lower power hardware should be fine on this. It may be worth looking into integrated switching so that the switching/routing tasks and even VPN could be offloaded to dedicated hardware, it would probably be more efficient and faster inter-node communications increase.

[HOGGL]

For 4 and live feeds, you would only be encoding the video at the source, the rest would just be packet re-direction and/or multicast

I was more concerned with the time it takes to encode 1 minute of video. But I guess that and the propagation time could be solved by adding delay and maybe using something like OSPF over the mesh network. This can wait though as live video is not needed yet.

[HOGGL]

RE: No. 1: My concerns with mesh networking were confirmed in the wesher docs.

The wesher tool builds a cluster and manages the configuration of wireguard on each node to create peer-to-peer connections between all nodes, thus forming a full mesh VPN. This approach may not scale for hundreds of nodes

I guess this is the point with friend-to-friend networks but I don't think this would be scalable with popular accounts. While a full mesh network would give the most redundancy, maybe limit the active connections to 2–3?

[rynomad]

Thanks for the questions!

1) Wireguard is not a chatty protocol when you don't use keepalives, and I don't think wesher is all that chatty on it's gossip channel except when new nodes are being announced (we should confirm this). So the main scaling boundary that comes to my mind is just the size of the host/route table, hence no giant global network. However, we've had some recent conversations that might have us pivot towards a wesherless approach utilizing ipv6 and public key hashing, in which case there would be a global address space and we just need a point to point handshake protocol.

In either case, direct host to host connection is something we'd ideally like to abstract away from the data model. Different protocols (activitypub vs bittorrent for example) may play more or less nice with this plan, but the core premise is that broadcast content will replicate through cluster by cluster. Definitely needs more design thinking though.

2) There's some context missing in the RFC but we want some of the other guarantees of wireguard, plus virtually no modern router respects uPNP hole-punching on low numbered ports. This may or not be a problem for mvp services, but in anticipation of an extensible platform it's desirable to only have to open 1 port on the firewall for wireguard traffic and still be able to access standard low ports like 80 & 443 over the VPN connection.

That said, I can see cases where services might want raw WAN exposure (public bittorrent for example).

3) I don't actually have a good answer for that, it's been a while since I took a look at scuttlebut, will reacquaint myself thanks!

4) I think it's likely that more professional creators will wind up running more powerful hardware if they need the horsepower for things like video transcoding. Using the Balena suite allows us to support a variety of hardware, raspberry Pi is the reference implementation for a personal/consumer node.

[HOGGL]

Thank you @rynomad for your thorough responses to all of my questions! I can attest to balena's portability, I was able to get skunkworks running on my armv7 BeagleBone Black.

I'm not sure if I should close this issue now as it may serve as a place for more discussion. What do you think?

[rynomad]

I'm flying by the seat of my pants WRT project management. Probably want to optimize for single issue threads, so I'll close this one (and get a proper PM tool set up). I would like to capture some of this info into a wiki or similar though, but that's another issue as well.