richo / groundstation

Decentralised gossip driven data sharing for the 21st century.
http://groundstation.me
MIT License
59 stars 12 forks source link

broadcast syncing for reduction in redundant traffic #35

Open haxwithaxe opened 11 years ago

haxwithaxe commented 11 years ago

for layer 1's with a literal broadcast (adhoc wifi, amateur radio, etc), syncing with a broadcast reply to requests would let others in range receive data without needing to send a request. thus freeing up the collision domain for data to flow rather than sync requests and repeated retransmission of the same data.

virtadpt commented 11 years ago

+1

richo commented 11 years ago

I originally planned to implement this with Multicast on ip6 enabled hosts.

I'll have a look into how much work is involved in making this work. You guys will wind up with what amounts to your own stationd that just puts the pieces relevant to you together (Sync driver optimised for minimal throughput and prioritisation, only moving objects relevant to you, etc), and because of that including this should be reasonably easy.

virtadpt commented 11 years ago

We tried IPv6-over-AX.25 on Friday and Saturday nights - the kernel (v3.8.7 and v3.9.4) doesn't support it, just IPv4.

haxwithaxe commented 11 years ago

you could also use udp/sctp + ipv4 http://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol python lib http://epx.com.br/pysctp/ it has error correction but doesn't do the handshake of slowness :P

richo commented 11 years ago

Funny you should mention that- we were looking at sctp last night in the context of http/2 (which is a tempting candidate for a switch in the core driver).

I'm going to focus on the sync algorithm first anyway, because it doesn't matter how good your transport is if you're sending megs of unnecessary data. Thanks for the links though, being able to bench a working implementation ftw.

haxwithaxe commented 11 years ago

also sctp is available via the socket module though it isn't part of the variables made available in the python api. the C code is socket( AF_INET, SOCK_STREAM, IPPROTO_SCTP )

yeah, it should be close to a drop in replacement so no sense in jumping the gun just to fiddle with it.