Brain Discovery not working

tmrobert8 commented 7 years ago

Win10, 64-bit

When I run ANY example script - it get's stuck (never times out or finishes) trying to discovery the first brain (ie discoverOneBrain). I know discovery works on the machine because I can find the brain with bonjour browser, with dns-sd and with my own java sdk. The only thing that doesn't discovery it is the SDK.

BTW - for testing, I simply hardcode the IP address in brainIp but would like to get this resolved

neophob commented 7 years ago

can you try to install a bonjour client on your windows machine? Itunes package the bonjour/zeroconf sdk, but https://support.apple.com/kb/DL999?locale=en_US might be shorter

This is noted in the sdk readme, see https://github.com/NEEOInc/neeo-sdk#windows-os

tmrobert8 commented 7 years ago

Already have it installed (that's how I got dns-sd)...

tmrobert8 commented 7 years ago

and I just reinstalled (in case it was corrupt) - same result, everything (even Microsoft Edge believe it or not) found the brain BUT the SDK ...

neophob commented 7 years ago

Ok, can you paste the output when the sdk starts? for example the output of env DEBUG="*" node device/simpleDevice/index.js

tmrobert8 commented 7 years ago

Not much to see (this is after 15 minutes of running):

NEEO SDK Example "simpleCustomDevice" adapter

discover one NEEO Brain...

From: Michael Vogt [mailto:notifications@github.com] Sent: Wednesday, June 7, 2017 3:10 PM To: NEEOInc/neeo-sdk neeo-sdk@noreply.github.com Cc: tmrobert8 troberts@bigfoot.com; Author author@noreply.github.com Subject: Re: [NEEOInc/neeo-sdk] Brain Discovery not working (#36)

http://ads.bigfoot.com/oasisc.php?s=514&w=728&h=90&iid=1496863589 Ads by Bigfoot http://www.bigfoot.net , to remove this ad please Upgrade http://ef.bigfoot.com to a premium account

Ok, can you paste the output when the sdk starts? for example the output of env DEBUG="*" node device/simpleDevice/index.js

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/NEEOInc/neeo-sdk/issues/36#issuecomment-306895302 , or mute the thread https://github.com/notifications/unsubscribe-auth/ATr3YjYRfFo-T5WYaF9ztRyL0x-13GEuks5sBvV5gaJpZM4NyqYv .

tmrobert8 commented 7 years ago

Of course you need to spell DEBUG correctly….

express:application set "x-powered-by" to true +0ms

express:application set "etag" to 'weak' +11ms

express:application set "etag fn" to [Function: wetag] +6ms

express:application set "env" to 'development' +2ms

express:application set "query parser" to 'extended' +0ms

express:application set "query parser fn" to [Function: parseExtendedQueryString] +1ms

express:application set "subdomain offset" to 2 +1ms

express:application set "trust proxy" to false +1ms

express:application set "trust proxy fn" to [Function: trustNone] +1ms

express:application booting in development mode +1ms

express:application set "view" to [Function: View] +1ms

express:application set "views" to 'C:\Users\Tim\Development\neeo\neeo-sdk-examples\views' +0ms

express:application set "jsonp callback name" to 'callback' +1ms

express:router:route new '/search' +8ms

express:router:layer new '/search' +1ms

express:router:route get '/search' +3ms

express:router:layer new '/' +1ms

express:router:route new '/:device_id' +1ms

express:router:layer new '/:device_id' +0ms

express:router:route get '/:device_id' +1ms

express:router:layer new '/' +3ms

express:router:route new '/:adapterid/discover' +14ms

express:router:layer new '/:adapterid/discover' +0ms

express:router:route get '/:adapterid/discover' +1ms

express:router:layer new '/' +0ms

express:router:route new '/:adapterid/:component/:deviceid' +1ms

express:router:layer new '/:adapterid/:component/:deviceid' +0ms

express:router:route get '/:adapterid/:component/:deviceid' +1ms

express:router:layer new '/' +0ms

express:router:route new '/:adapterid/:component/:deviceid/:value' +1ms

express:router:layer new '/:adapterid/:component/:deviceid/:value' +0ms

express:router:route get '/:adapterid/:component/:deviceid/:value' +1ms

express:router:layer new '/' +3ms

express:router:route new '/:adapterName/subscribe/:deviceId/:eventUriPrefix' +2ms

express:router:layer new '/:adapterName/subscribe/:deviceId/:eventUriPrefix' +0ms

express:router:route get '/:adapterName/subscribe/:deviceId/:eventUriPrefix' +1ms

express:router:layer new '/' +0ms

express:router:route new '/:adapterName/unsubscribe/:deviceId' +1ms

express:router:layer new '/:adapterName/unsubscribe/:deviceId' +2ms

express:router:route get '/:adapterName/unsubscribe/:deviceId' +0ms

express:router:layer new '/' +1ms

express:application set "x-powered-by" to false +0ms

express:router use '/' query +53ms

express:router:layer new '/' +1ms

express:router use '/' expressInit +2ms

express:router:layer new '/' +0ms

express:router use '/' jsonParser +5ms

express:router:layer new '/' +3ms

express:router use '/db' router +4ms

express:router:layer new '/db' +2ms

express:router use '/device' router +3ms

express:router:layer new '/device' +2ms

express:router use '/' router +7ms

express:router:layer new '/' +3ms

express:router use '/favicon.ico' +1ms

express:router:layer new '/favicon.ico' +8ms

express:router use '/' +2ms

express:router:layer new '/' +2ms

NEEO SDK Example "simpleCustomDevice" adapter

neeo:device:DeviceBuilder add button { name: 'button-a', label: 'Button A' } +0ms

neeo:device:DeviceBuilder add button { name: 'button-b', label: 'Button B' } +5ms

neeo:device:DeviceBuilder add buttonhandler +4ms

discover one NEEO Brain...

neeo:discover:mdns findFirstNeeoBrain +7ms

From: Michael Vogt [mailto:notifications@github.com] Sent: Wednesday, June 7, 2017 3:10 PM To: NEEOInc/neeo-sdk neeo-sdk@noreply.github.com Cc: tmrobert8 troberts@bigfoot.com; Author author@noreply.github.com Subject: Re: [NEEOInc/neeo-sdk] Brain Discovery not working (#36)

http://ads.bigfoot.com/oasisc.php?s=514&w=728&h=90&iid=1496863589 Ads by Bigfoot http://www.bigfoot.net , to remove this ad please Upgrade http://ef.bigfoot.com to a premium account

Ok, can you paste the output when the sdk starts? for example the output of env DEBUG="*" node device/simpleDevice/index.js

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/NEEOInc/neeo-sdk/issues/36#issuecomment-306895302 , or mute the thread https://github.com/notifications/unsubscribe-auth/ATr3YjYRfFo-T5WYaF9ztRyL0x-13GEuks5sBvV5gaJpZM4NyqYv . https://github.com/notifications/beacon/ATr3YluF6jg7qsD5x1rGnVI2YVJqOOrHks5sBvV5gaJpZM4NyqYv.gif

neophob commented 7 years ago

hint: you can use ``` to enter code or logs, that make it more readable.

however no further output is visible after the neeo:discover:mdns findFirstNeeoBrain +7ms line

tmrobert8 commented 7 years ago

Sorry - replied via email rather than the webpage and didn't think about it. No further output after that point (let it run about 15 minutes or so).

neophob commented 7 years ago

which exact windows version do you use?

tmrobert8 commented 7 years ago

Windows 10, 64-bit - version 1703 (build 15063.296)

tmrobert8 commented 7 years ago

Interesting - just discovered that my openhab mdns fails to see the brain after restarts as well. I'm thinking something may be funky in my bonjour install and I'm going to do a full clean and reinstall. I'll let you know how it goes..

neophob commented 7 years ago

thanks, let me know if you find out something

tmrobert8 commented 7 years ago

I completely uninstalled everything (itunes, bonjour, bonjour browser and anything else related to apple) based on their 'completely uninstall' docs. Then installed the bonjour printer services - no go. Then installed itunes - no go. Note: I was still able to find them with other services however (dns-sd, IE, etc)

neophob commented 7 years ago

Hmm hard to tell, we don't have any Windows machine here, but maybe @nklerk might have an idea?

nklerk commented 7 years ago

Please reboot your brain and try again.

If Its not bonjour or the brain then it might be caused by the packages. You could try to reinstall the SDK. The SDK depend on other packages

Also look if nothing else is already bound to port 5353 on your computer. As the used package liked to bind the port.

nklerk commented 7 years ago

Also be sure no firewall or anti virus utility is blocking the package to use MDNS. A packetcapture would be helpful, use wireshark to capture MDNS traffic.

tmrobert8 commented 7 years ago

Already have done all that - MDNS packets definitely are being received and all other tools (on the same machine) find it BUT the SDK (I've reinstalled it multiple times).

nklerk commented 7 years ago

Did you see a DNS query being sent out and can you see it's answer. Can you capture both and post a screenshot so I can have a look at them?

tmrobert8 commented 7 years ago

I know exactly what's going wrong and it's an issue with the underlying bonjour library they use.

My machine has not only openVPN on it but also VMWare - both of which create their own network interfaces. When the SDK starts up - it must be binding to one of those interfaces and will never see the MDNS broadcasts on the true interface.

Just to prove it - I disabled all those interfaces and tada - it found the brain with no problems. Reenabling the interfaces and the SDK no longer finds the brain. Seems kinda related to https://github.com/watson/bonjour/issues/14 - but not exactly.

Note: the VMWare interface is eth0 - betcha that's causing the issue with the binding if you ask me.

You might be able to recreate by installing openVPN and making sure it's eth0.

neophob commented 7 years ago

ahh your working in a vm. mdns works only in the local subnet, so you need make sure your vm is in the same network as the brain

tmrobert8 commented 7 years ago

Ack - didn't explain myself well - no it's not in a VM. The machine I'm running on ALSO acts as a VM host (for other stuff that I use). The SDK is running directly on the machine. What my point was - it's likely picking up on either the openVPN interface or those interfaces defined by VMWare (host) and listening on those (which won't work since it's not being broadcast there).

Again - I want to reiterate - everything else (on the machine) bonjour based finds the brain. The only thing that doesn't is the SDK. If I disable the other interfaces - the SDK suddenly starts to find the brain (and stops finding it if I enable the other interfaces). This is likely a bug in what interface the bonjour project (which the SDK uses to find stuff) listens on (or is a bug in how it deals with multiple interfaces)...

nklerk commented 7 years ago

I think that MDNS binds to the wrong interface on his computer. I have to say my laptop doesn't seem to have that issue, I have quite a collection of network interfaces (8) and never encoumtered the issue. Aso I think Its strange that you see broadcasts. You should see a multicast query and then a reply.

tmrobert8 commented 7 years ago

I don't actually think it's in the binding either because wireshark doesn't show the query on any of the other interfaces.

When everything BUT the real interface is disabled. I get the following query/response from the SDK

Multicast Domain Name System (response)
    Transaction ID: 0x0000
    Flags: 0x8400 Standard query response, No error
    Questions: 0
    Answer RRs: 4
    Authority RRs: 0
    Additional RRs: 0
    Answers
        _neeo._tcp.local: type PTR, class IN, NEEO Attic._neeo._tcp.local
        NEEO Attic._neeo._tcp.local: type TXT, class IN, cache flush
        NEEO Attic._neeo._tcp.local: type SRV, class IN, cache flush, priority 0, weight 0, port 3000, target NEEO-4f150397.local
        NEEO-4f150397.local: type A, class IN, cache flush, addr 192.168.1.29

However, if I enable say the openVPN interface - I get no MDNS query on the real interface and no MDNS query on the openVPN interface either. Kinda points to an issue in handling of multiple interfaces within the bonjour project...

neophob commented 7 years ago

@tmrobert8 did you make any progress here? if not, can you describe your local setup including the network settings

tmrobert8 commented 7 years ago

Yes - it was binding to my openVPN interface (which the brain isn't on). When an interface isn't defined, the system 'chooses' one (according to doc) - just happen to choose the wrong one. I made a patch to two of the files (index.js and mdns.js):

In index.js - I replaced the findFirstNeeoBrain with the following:

function findFirstNeeoBrain() {
    return new BluePromise((resolve, reject) => {
        const nets = os.networkInterfaces();
        for (const netName in nets) {
            const max = nets[netName].length;
            for (var idx = 0; idx < max; idx++) {
                const netInfo = nets[netName][idx];
                if (netInfo.internal === false && netInfo.family == "IPv4") {
                    mdns.findFirstNeeoBrain(netInfo.address)
                        .then(buildNEEOBrainModel)
                        .then(serverInfo => resolve(serverInfo));
                }
            }
        }
    });
}

That will loop through all the non-loopback IP4 interfaces and attempt to find the brain using that interface calling...

The mdns.js patch for the interface:

function findFirstNeeoBrain(netIf) {
  debug('findFirstNeeoBrain');
  return new BluePromise((resolve, reject) => {

    function serviceUpListener(service) {
      if (!service || !service.txt) {
        debug('invalid entry ignored');
        reject(new Error('INVALID_SERVICE_FOUND'));
      }
      debug('found a NEEO Brain: [%s]', service.name);
      resolve(service);
    }
    var bonjour = require('bonjour')({ interface: netIf });
    const mdnsBrowser = bonjour.findOne({ type: MDNS_NAME }, serviceUpListener);
    mdnsBrowser.start();
  });
}

Pretty self explanatory - basically create bonjour with the specified interface.

I don't have two brains (yet) so I can't really test to see if this works when each brain is on a different interface - but it looks like it should.

Note: let me know if you want me to submit an official PR with this fix...

neophob commented 7 years ago

Thanks @tmrobert8 that make sense. A PR would be awesome!

In fact all what's needed is a optional definition of the interface to use. then you would start the application like MDNS_IFACE_NAME=whatever node ....

tmrobert8 commented 7 years ago

Originally I did (although via an opts - not environment variable) but then I got to thinking that I'd likely have brains on different subnets and made the solution above to allow for that (covers both situations and is all automatic without any additional setup).

I'll put together a PR in a few days...

tmrobert8 commented 7 years ago

Just submitted the PR (changed the code to only discover the first brain - which is what the method's contract is). Resubmitted a few times for syntax type stuff - but now the PR is failing on something I have no idea on (something about failing an invalid brain?).

neophob commented 7 years ago

Thanks @tmrobert8 for your PR.

the unit test fails on travis, you can test that locally by running npm t. Take a look at the test/unit/lib/discover/index_test.js file, I guess this test fails because the parameters of this function changed and the stub method does not match anymore

tmrobert8 commented 7 years ago

stub method didn't change but the issue was that the error changed (promise.any throws an aggregateerror and your test method was expecting a single error [which makes sense given that the method returns a single brain instance anyway]). I changed the code to consume the aggregateerror and rethrow the first error - which allows your test to pass and the PR to be clean.

cro13 commented 6 years ago

I believe this should now be fixed in the SDK. Please use the multiInterface flag! Thanks for your help!

grumpyengineer commented 6 years ago

I think I've found a minor problem with the multiInterface implementation.

Win 7 with two NICs. 192.168.1.x is my main network, 192.168.0.x is my test network. Neeo brain is on 192.168.1.x. Brain is found just fine after setting multiInterface to true BUT the json in the registerSdkDeviceAdapter POST has the wrong IP so nothing works.

getAnyIpAddress in generateBaseUrl returns the first valid NIC IP address from os.networkInterfaces(). Which in my case is .0.x.

I have tried a quick fix using https://github.com/rs/node-netmask to check if the brain IP is in the same subnet as the NIC IP. And it works.

Not tried on Linux. I think Avahi is playing up for me.

NEEOInc / neeo-sdk

Brain Discovery not working #36