search

dweb-camp-2019 / projects

Projects @ DWeb Camp 2019 💻
https://riot.im/app/#/room/#dweb-camp-2019-projects:matrix.org
15 stars 0 forks source link

Farm may be useful/interesting #1

Open pvh opened 5 years ago

pvh commented 5 years ago

We built farm, a p2p collaborative web development environment that builds offline-first real-time collaborative apps. It might be useful to the project and doesn't require internet but is an electron app you'll need to install.

It could be cool for building ad-hoc programs to organize people, capture notes, depict the camp, or pretty much anything else you could imagine you'd want real-time Docs-like collaboration for.

It uses discovery-swarm and can do peer discovery over mDNS & via the dat community's DNS-SD and using it requires installing an electron app as of this writing.

As of March it's transitioning from a research project to usable software and may-or-may-not actually work in the field. If someone wants it to, that increases the likelihood of us fixing the bugs.

Feel free to reach out to me here if you have questions about it.

If folks can find each other via mDNS, this'd be pretty easy, but I'm guessing the network will be too big for that to work?

benhylau commented 5 years ago

On the same Access Point, I'd imagine mDNS to work. Now across mesh node boundaries, if there are stable IPs, is there a way to do manual peering like ipfs bootstrap add?

pvh commented 5 years ago

If we have TCP routability the DHT will work.

On Tue, Mar 26, 2019 at 8:16 AM Benedict Lau notifications@github.com wrote:

On the same Access Point, I'd imagine mDNS to work. Now across mesh node boundaries, if there are stable IPs, is there a way to do manual peering like ipfs bootstrap add?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/dweb-camp-2019/meshnet/issues/14#issuecomment-476696830, or mute the thread https://github.com/notifications/unsubscribe-auth/AAALQKeb1f_4r2vDaQDtJCXTa9cIk8Keks5vajnJgaJpZM4cKr00 .

-- Peter van Hardenberg San Francisco, California "Everything was beautiful, and nothing hurt."—Kurt Vonnegut

RangerMauve commented 5 years ago

Yeah, if devices are routable then manual bootstrapping should be pretty straightforward. It'd be important to make it easy for people to actually configure the bootstrap nodes without recompiling everything.

benhylau commented 5 years ago

Yea the DHT will work after TCP peers are established, but what is the user experience of manually peering? Or is there any other discovery mechanism?

RangerMauve commented 5 years ago

Since farm is using discovery-swarm, if MDNS works, then it'll be enough to find peers. :D

hackergrrl commented 5 years ago

I'm not sure about dat, but cabal doesn't have any mechanisms for manually adding peers (yet). It's doable pre-camp, but since soooo many cool p2p apps already have mdns working well, that might be the path of least resistance.

RangerMauve commented 5 years ago

I raised an issue in the Dat CLI to add new flags to specify bootstrapping servers / DNS discovery servers.

benhylau commented 5 years ago

To clarify: Because we are planning to use Babel (or other Layer 3 mesh protocols), rather than Layer 2 protocols like batman-adv (think one big switch), each physical mesh node will present a Layer 3 boundary where without some sort of centralized gateway, the mDNS packets do not cross. So on the same Access Point (people within a ~100 m radius region) we'd peer automatically over mDNS, but trying to talk from one end of camp to another we'll have netsplit. Hence, manual peering / bootstrap.

RangerMauve commented 5 years ago

Can you still route across access points?

benhylau commented 5 years ago

Yes, just that mDNS packets don't naturally cross them. Data packets yes, but through NATs and stuff. @darkdrgn2k thoughts on discovery? Since you have been working on these a bunch :)

amark commented 5 years ago

I'm pretty dumb with actual networking hardware/standards/jargon... but let me see if I understand this?

Pretend there are 6 access points, and for sake of simulating network splits, it looks like this:

A ................ E
.. \ ........... /
... B --------- D
.. / ........... \
C ................ F

At least in the case of GUN using DAM (our daisy-chain ad-hoc mesh-network) and I'm pretty sure also true for SSB with EBT (epidemic-broadcast-trees), as long as a GUN peer is running on A,B,C,D,E,F and A-B, B-C, B-D, D-E, D-F know about each other and are connected...

THEN any message from a camper connected to A will automatically reach campers on B,C,D,E,F and vice-versa (or any other combination) efficiently and even ACKs will use the most optimal route.

Also, net-splits won't be a problem. Sure, while B-D is "down" there won't be sync, but as soon as connections come back up, all peers will automatically sync and repair themselves and converge (no more net split) even on conflicts. We have a fully automated net-split correctness tests with our PANIC tool (its kinda like Jepsen / Chaos Monkey).

So for clarity, when you say "mDNS packets do not cross boundaries" ... will that matter for me? Or does that only matter if people are needing A to discover F? If I do not need A to know about F (only about B. I mean hey, knowing about F is great too, but not necessary due to the robustness/resilience of the daisy-chaining) do I need to be worried?

RangerMauve commented 5 years ago

MDNS is useful for peer discovery, if you're able to connect peers together without it you should be fine. :D

benhylau commented 5 years ago

@amark adding to @RangerMauve's answer, what I was referring to is purely for discovery purposes. I understand that once peered with these links [ A-B, B-C, B-D, D-E, D-F ] messages will route across the network even if A-F doesn't exist. But when we spin up A and B, how do they form A-B in the first place?

If it's on the same switch, perhaps mdns peer discovery would allow them to automatically find each other and form A-B. However, if they are on two different switches, that's as if the A if plugged into my home, and B plugged into my office, behind their own NATs. How does GUN handle discovery / peering in this scenario?

amark commented 5 years ago

@benhylau @RangerMauve exactly, two laptops on my same home wireless router discover each other and start syncing.

But, IDK how this works with different WiFi routers - my friends' place with multiple routers, the repeater is still a different network that has to be connected to separately, and I have no clue what happens with multicast in those settings (???) can somebody inform/educate me?

My stupid naive thought is: Can a device plugged into A also wirelessly connect to B, and vice versa, a device plugged into B wirelessly connect with A? (I assume this isn't necessary) Now device has ethernet to A's WiFi connected peers, and is WiFi connected to B.

:P :P :P do OSes even let me use Ethernet + WiFi simultaneously? They should!!! I don't think I've ever seen/done this in actuality tho, and I honestly don't play around with this stuff much. But clearly + obviously need to and should.

Please teach me or link me to "WiFi Mesh Networks for Dummies 101" videos! :D :P

Assuming that WiFi A and WiFi B can also directly connect to each other, aren't they still going to be on some sort of same subnet that multicast would broadcast across?

amark commented 5 years ago

home <-> office is just usually IP addresses if available. If not, leaving it to WebRTC to NAT traversal and crossing fingers it'll work (WebRTC fails lots for me, only like 40% success rate). Final fallback is through websocket relay peer(s) with known publicly exposed IP/DNS addresses, but that is lame when if we have our own mesh network can't we be doing multicast!??

benhylau commented 5 years ago

@amark I feel like we are squatting the Farm thread :D but let me try to address the open questions, then move back to #2 for continued discussions heh

@benhylau @RangerMauve exactly, two laptops on my same home wireless router discover each other and start syncing.

But, IDK how this works with different WiFi routers - my friends' place with multiple routers, the repeater is still a different network that has to be connected to separately, and I have no clue what happens with multicast in those settings (???) can somebody inform/educate me?

My stupid naive thought is: Can a device plugged into A also wirelessly connect to B, and vice versa, a device plugged into B wirelessly connect with A? (I assume this isn't necessary) Now device has ethernet to A's WiFi connected peers, and is WiFi connected to B.

do OSes even let me use Ethernet + WiFi simultaneously?

Yup, you can have many many interfaces, they can be connected to the same or different networks. Which path traffic flows through will be determined by your routing table (e.g. route -n). If you bridge the interfaces, they'd behave as if they are plugged in to the same network switch.

Can a device plugged into A also wirelessly connect to B, and vice versa, a device plugged into B wirelessly connect with A? (I assume this isn't necessary) Now device has ethernet to A's WiFi connected peers, and is WiFi connected to B. [ . . . ] Assuming that WiFi A and WiFi B can also directly connect to each other, aren't they still going to be on some sort of same subnet that multicast would broadcast across?

I think if the device that is plugged into A and wirelessly connected to B has its two network interfaces bridged (e.g. eth0 and wlan0), then it will let the multicast flow through. Not 100% sure.

my friends' place with multiple routers

Depends on topology, I think if one is nested in the LAN of another then it will get all the broadcast.

repeater is still a different network

Depends on implementation of the repeater, they have all kinds of stuff out there!

benhylau commented 5 years ago

@pvh I want to loop back on the original discussion of this thread, and see if farm will be leading sessions at Camp. I don't think it's contingent on having ready / reliable software, perhaps this is exactly the right time to hold discussion sessions that could inform what the software will be!