Lookupd

[dkua]

This pull request allows for a Worker to connect to one or more NSQLookupd and NSQD nodes. A two-in-one solution for Issues #6 and #8

Some API changes:

• Worker now depends on config settings: QueueAddresses and LookupAddresses
• Worker no longer depends on the config setting QueueAddr as it is too limiting for this use case

[dkua]

An issue I've discovered while testing this out. If QMD is set with a Throughput = 5 and it connects to a NSQD node twice, once through NSQLookupd and once directly. Then only two in-flight messages get sent through each connection. The last spot seems to be in contention of some sort.

This also occurs when Throughput = 7. Only three in-flight messages are possible per connection.

If only one connection is allowed then max throughput is possible.

It seems there might be some issue with more than one connection to a NSQD node where in-flight messages are limited to floor( Throughput / Num of Connections ).

Will also need to test this with more than one NSQD node.

[dkua]

It seems it doesn't matter if there are connections directly through NSQD or through NSQLookupd. In the scenario below there exist two NSQD nodes and two QMD nodes. The two Servers are connected to the two NSQD nodes respectively. The two Workers each have a connection to both NSQD nodes through NSQLookupd. There are no direct connections and Throughput = 7.

Each connection maxes out at floor( Throughput / Num of Connections ) which in this case is 3. They respect the Throughput of the Worker and do not try to fight for the last spot.

So adding extra direct connections doesn't help squeeze more messages through. It only takes up precious resources. So I think it's probably best to have an either-or situation where QMD prefers to connect to NSQLookupds and if it can't find any, it tries to direct connect to NSQDs.