Patch - Auto discovery of local nodes using Bonjour

[GoogleCodeExporter]

Attached is a patch that adds local discovery of nodes using mDNS/Bonjour. 
It uses pybonjour and requires Bonjour be present, i.e by installing 
Bonjour standalone or by having iTunes or Safari present on the machine. 

There are two parts - a bonjour 'server' that invokes a mDNS registration 
and a puller which is called from start() in core to search for nodes on 
the local (same broadcast domain/subnet) network.

This is separate from the dconfig puller module.

So far I've tested on Mac OS X, Windows XP and Windows 7. They have been 
able to start from a clean install to network sync within 2 minutes 
without any DNS ipcache help. 

Known issues:
1. Bridge/dconfigpusher has not been tested. We don't use bridge, and 
won't need to with this :)
2. I haven't taken much care programming wise - this is a days worth of 
coding, so there are probably better places to bolt on the server and 
puller modules - also, puller will block during the initial search, but 
this isn't much of a problem
3. If !udp is used, dtella needs to be restarted before the service 
registration is changed.
4. We don't check if bonjour is installed or not

In addition to the patch below, the following needs to be added to 
local_config:

use_bonjour = True

Original issue reported on code.google.com by MathewMc...@gmail.com on 4 Apr 2010 at 6:18

Attachments:

• dtella-bonjour.diff

[GoogleCodeExporter]

Revised patch
- Node searching is done in a separate daemon thread invoked from main where it 
can't 
interfere with Twisted. 
-- Requires Python 2.6 :(
- Spaces instead of tab indenting

Original comment by MathewMc...@gmail.com on 4 Apr 2010 at 11:01

Attachments:

• dtella-bonjour.diff

[GoogleCodeExporter]

Oops, a pair of import lines were missing from that patch, lets try again

Original comment by MathewMc...@gmail.com on 4 Apr 2010 at 11:12

Attachments:

• dtella-bonjour.diff

[GoogleCodeExporter]

in the first patch (non-thread), can you instead use select() with 
blocking=False,
and then just do a reactor.callLater() if the socket still isn't ready?

Original comment by infinity0x@gmail.com on 11 Apr 2010 at 6:58

[GoogleCodeExporter]

Try using "threads.deferToThread", because that integrates nicely with Twisted. 
 I 
used it in push_dnsupdate to call dnspython:

http://code.google.com/p/dtella/source/browse/trunk/dtella/modules/push_dnsupdat
e.py

Original comment by sparkm...@gmail.com on 11 Apr 2010 at 10:21

[GoogleCodeExporter]

Actually it seems we don't need to use select.select() as long as pybonjour can 
reach the callback functions, 
pybonjour has just been design to allow select.select() as its block mechanism. 
(Blocking was useful to make pybonjour do the search multiple times, but it 
doesn't seem to be needed now)

So the next iteration will be more like the first patch.

Original comment by MathewMc...@gmail.com on 11 Apr 2010 at 11:06

[GoogleCodeExporter]

I take that back, select.select() is needed so we can tell if bonjour has found 
any new nodes, blocking=False 
should be possible still.

Original comment by MathewMc...@gmail.com on 11 Apr 2010 at 11:15

[GoogleCodeExporter]

@sparkmaul i think solutions that don't involve any should be preferred.

if any thread uses threading._Condition.wait(), this is uninterruptible in 
python and
will block all signals to the process (ctrl-C etc). maybe the current code is 
OK, but
future coders might overlook this.

Original comment by infinity0x@gmail.com on 11 Apr 2010 at 8:21

[GoogleCodeExporter]

Have you seen this?:
http://www.indelible.org/ink/twisted-bonjour/

It appears to claim that you can make pybonjour work with Twisted's reactor 
loop.

Original comment by sparkm...@gmail.com on 12 Apr 2010 at 3:53

[GoogleCodeExporter]

hey all, has there been any progress on this? if not, i will pick it up in the 
next
month or so?

Original comment by infinity0x@gmail.com on 22 May 2010 at 4:27

[GoogleCodeExporter]

I guess I'm not very excited about Bonjour because it encourages the network to 
become 
fragmented into broadcast domains.  Do you have an example where something like 
this 
would be useful?

Original comment by sparkm...@gmail.com on 22 May 2010 at 5:59

[GoogleCodeExporter]

hmm, you're right - if the network is fragmented, then joining nodes will check
bonjour, find some peers, and not continue on to check the dconfig for peers 
outside
the broadcast domain.

i suppose one way to fix this would be, after initial sync, see if we have any 
peers
outside our broadcast domain, and if not, then check the dconfig[1]. (this 
could be
useful for general network fragmentation too.)

the main reason i like this is because it puts less dependency on the central
dconfig. although, i guess this is questionable - networks spanning multiple
broadcast domains will have to check the dconfig anyway.

in any case, it doesn't seem complicated to implement, i can't see it causing 
any
harm (if done right), so i could just have a go when i have a few hours to 
spare.

[1] maybe after a random time delay, so nodes don't all try to do this at once, 
which
is unnecessary (although not harmful).

Original comment by infinity0x@gmail.com on 22 May 2010 at 11:01

[GoogleCodeExporter]

I haven't done anything with the patch in regards to making it work without 
threading, you're welcome to have a go.

I don't care much about fragmentation because the site this is running at is 
trapped 
inside a single broadcast domain in private IP space. (Unless IPv6 is forced on 
us 
tomorrow, please) Not many people run behind their own routers. 

The way I structured the treaded patch was that Bonjour would start looking as 
soon 
as dtella starts up (even if client isn't working). dconfig is still hit when a 
client does connect, but now it has a nice list of global + local addresses to 
connect to. I actually set the bonjour enabled client to use a different 
dconfig 
record than the normal one, and once it starts up it simply gets non-bonjour 
nodes 
from the existing network.

Original comment by MathewMc...@gmail.com on 23 May 2010 at 1:24

[GoogleCodeExporter]

A single broadcast domain is certainly a degenerate case for Dtella, 
considering that 
Dtella's primary function is to simulate a broadcast domain.

It's analogous to using your car to cross the street because you don't have any 
shoes.  
It may work perfectly well, but it just seems wrong.

Original comment by sparkm...@gmail.com on 23 May 2010 at 2:23

[GoogleCodeExporter]

This is perfect for my university's network. It's segmented into a bunch of 
different
networks based on houses and schools; dtella doesn't work across them. I want to
start dtella for my school's local network, not university-wide. Going to test 
the
patch that works with python 2.5. I look forward to any changes regarding proper
twisted thread deferment. 

I don't see what's wrong with implementing this if it works alongside, and never
instead of, a configured dconfig object. 

Also, attached is a function for determining if bonjour is running on a windows
system. For OS X the script returns true, for Linux it returns false because I 
don't
know what the process would be called

Original comment by FiatPan...@gmail.com on 23 May 2010 at 4:50

Attachments:

• isBonjourRunning.py