search

IoTone / huenim

A nim-based set of language bindings for Philips HUE lighting. Clean room implementation.
MIT License
2 stars 0 forks source link

UPnP SSDP discovery not working using UDP multicast #3

Open truedat101 opened 6 years ago

truedat101 commented 6 years ago

I've implemented this same logic in several other languages, but couldn't get it working in nim. I'm assuming it is my lack of understanding of the libraries or the low-level networking.

Method in question: https://github.com/IoTone/huenim/blob/master/src/huenim.nim#L136 findHueHubsViaUPnP()

To reproduce:

Call findHueHubsViaUPnP() It should return some response for any other devices on the network that match the criteria. However, no devices respond, not a single response. I can confirm that the request goes out (using network tools to monitor UDP multicast messages).

truedat101 commented 6 years ago

Assumption is I have some problems in the use of the nim apis. If it helps I can post some C code that does the same thing.

dom96 commented 6 years ago

Yeah, please post the C code and also the output that this code gives.

truedat101 commented 6 years ago

Will dig up the code and post that output.

truedat101 commented 6 years ago

I have a bad example I put together in C here: https://pastebin.com/CvmqvV2j

The bad example doesn't use the query to do a sendto first before listening for a response. I noticed that regardless of my query, I will get a response from my network which I can easily filter out the key tell-tale signs this is a Hue Hub.

Here are some responses from this code below. The giveaway that this is a HUE hub is the presence of a LOCATION value that ends with discovery.xml, for example: LOCATION: http://10.0.0.222:80/description.xml

One of the approaches people take to find their hub(s) is to enumerate all ip addresses in the range of the local network, and try to hit port 80/description.xml. However that's sort of a waste. If the UPnP approach works, it should yield a narrow set of devices that will respond, and from those you filter based on the LOCATION. It's ugly either way, but the multicast approach should be more efficient.

Opening datagram socket....OK.
Setting SO_REUSEADDR...OK.
Binding datagram socket...OK.
Adding multicast group...OK.
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://10.0.0.222:80/description.xml
SERVER: FreeRTOS/7.4.2 UPnP/1.0 IpBridge/1.16.0
NTS: ssdp:alive
hue-bridgeid: 001788FFFE1A73B5
NT: upnp:rootdevice
USN: uuid:2f402f80-da50-11e1-9b23-0017881a73b5::upnp:rootdevice

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://10.0.0.222:80/description.xml
SERVER: FreeRTOS/7.4.2 UPnP/1.0 IpBridge/1.16.0
NTS: ssdp:alive
hue-bridgeid: 001788FFFE1A73B5
NT: upnp:rootdevice
USN: uuid:2f402f80-da50-11e1-9b23-0017881a73b5::upnp:rootdevice

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://10.0.0.222:80/description.xml
SERVER: FreeRTOS/7.4.2 UPnP/1.0 IpBridge/1.16.0
NTS: ssdp:alive
hue-bridgeid: 001788FFFE1A73B5
NT: uuid:2f402f80-da50-11e1-9b23-0017881a73b5
USN: uuid:2f402f80-da50-11e1-9b23-0017881a73b5

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://10.0.0.222:80/description.xml
SERVER: FreeRTOS/7.4.2 UPnP/1.0 IpBridge/1.16.0
NTS: ssdp:alive
hue-bridgeid: 001788FFFE1A73B5
NT: uuid:2f402f80-da50-11e1-9b23-0017881a73b5
USN: uuid:2f402f80-da50-11e1-9b23-0017881a73b5

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://10.0.0.222:80/description.xml
SERVER: FreeRTOS/7.4.2 UPnP/1.0 IpBridge/1.16.0
NTS: ssdp:alive
hue-bridgeid: 001788FFFE1A73B5
NT: urn:schemas-upnp-org:device:basic:1
USN: uuid:2f402f80-da50-11e1-9b23-0017881a73b5

b5

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=100
LOCATION: http://10.0.0.222:80/description.xml
SERVER: FreeRTOS/7.4.2 UPnP/1.0 IpBridge/1.16.0
NTS: ssdp:alive
hue-bridgeid: 001788FFFE1A73B5
NT: urn:schemas-upnp-org:device:basic:1
USN: uuid:2f402f80-da50-11e1-9b23-0017881a73b5

b5

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=120
LOCATION: http://10.0.0.1:47509/rootDesc.xml
SERVER: AsusWRT/380.66 UPnP/1.1 MiniUPnPd/2.0
NT: upnp:rootdevice
USN: uuid:c88b4917-3f2e-4982-93cb-8ae49467d857::upnp:rootdevice
NTS: ssdp:alive
OPT: "http://schemas.upnp.org/upnp/1/0/"; ns=01
01-NLS: 1505133119
BOOTID.UPNP.ORG: 1505133119
CONFIGID.UPNP.ORG: 1337

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=120
LOCATION: http://10.0.0.1:47509/rootDesc.xml
SERVER: AsusWRT/380.66 UPnP/1.1 MiniUPnPd/2.0
NT: urn:schemas-upnp-org:device:InternetGatewayDevice:1
USN: uuid:c88b4917-3f2e-4982-93cb-8ae49467d857::urn:schemas-upnp-org:device:InternetGatewayDevice:1
NTS: ssdp:alive
OPT: "http://schemas.upnp.org/upnp/1/0/"; ns=01
01-NLS: 1505133119
BOOTID.UPNP.ORG: 1505133119
CONFIGID.UPNP.ORG: 1337

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=120
LOCATION: http://10.0.0.1:47509/rootDesc.xml
SERVER: AsusWRT/380.66 UPnP/1.1 MiniUPnPd/2.0
NT: uuid:c88b4917-3f2e-4982-93cb-8ae49467d857
USN: uuid:c88b4917-3f2e-4982-93cb-8ae49467d857
NTS: ssdp:alive
OPT: "http://schemas.upnp.org/upnp/1/0/"; ns=01
01-NLS: 1505133119
BOOTID.UPNP.ORG: 1505133119
CONFIGID.UPNP.ORG: 1337

33119
BOOTID.UPNP.ORG: 1505133119
CONFIGID.UPNP.ORG: 1337

"
Reading datagram message...OK.
The message from multicast server is: "NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=120
LOCATION: http://10.0.0.1:47509/rootDesc.xml
SERVER: AsusWRT/380.66 UPnP/1.1 MiniUPnPd/2.0
NT: urn:schemas-upnp-org:device:WANConnectionDevice:1
USN: uuid:c88b4917-3f2e-4982-93cb-8ae49467d859::urn:schemas-upnp-org:device:WANConnectionDevice:1
NTS: ssdp:alive
OPT: "http://schemas.upnp.org/upnp/1/0/"; ns=01
01-NLS: 1505133119
BOOTID.UPNP.ORG: 1505133119
CONFIGID.UPNP.ORG: 1337

"
truedat101 commented 6 years ago

In the Java version of this, it worked like a charm. The nim version I would never get a response.

dom96 commented 6 years ago

Hrm, all I can is guess, but this part is missing from your Nim code (isn't it?):

if (setsockopt(sd, IPPROTO_IP, IP_ADD_MEMBERSHIP, (char *)&group, sizeof(group)) < 0)
truedat101 commented 6 years ago

Thanks. I'll revisit this and try that, spent some time on different variations of the discover initially, and eventually switched to the nupnp implementation. UPnP has a bad rap because of security concerns, but this is still useful for a number of IoT type devices that are out there.

enthus1ast commented 6 years ago

@truedat101 if this is still relevant you could have a look at https://github.com/enthus1ast/nimMulticast

truedat101 commented 6 years ago

@enthus1ast actually yeah, awesome. I spent a fair amount of time chasing down what I was doing wrong , so will have a look at your lib.

truedat101 commented 5 years ago

Will have a look at this module nimMulticast.

truedat101 commented 5 years ago

Finally have some time to look at this.

enthus1ast commented 5 years ago

just skimmed through your code: bindAddr(socket, Port(1900), "239.255.255.250") afaik for multicast there is no need to bind to the multicast address. The multicast enabled socket will receive every datagram send to the multicast group.

truedat101 commented 5 years ago

Good point. Need to get back into this.

truedat101 commented 5 years ago

Hmm, tried socket.bindAddr(Port(1900)) , but still just get back empty response. I have to remember how I was snooping the network for UPnP discovery. I've been through the code several times and didn't find a solution.

Need to try the suggestion https://github.com/IoTone/huenim/issues/3#issuecomment-332238458 from Dominik.

enthus1ast commented 5 years ago

this principally is what multicast joinGroup does. So to receive multicast datagrams you have to:

  1. create a socket
  2. bindAddr (bind to your interface address not the multicast group!)
  3. socket.joinGroup("239.255.255.250") when joinGroup returns true, the socket should be able to receive multicast. you can test this by sending to the multicast group (the sender socket does not need to be in the multicast group) so sending a datagram with ncat should be enough:
[david@eb ~]$ ncat -4 --udp 239.255.255.250 1900
this should be visible in your testprogram
enthus1ast commented 5 years ago

Something like this should work:

bindAddr(socket, Port(1900))
let group = "239.255.255.250"
assert true == socket.joinGroup(group)
let res = socket.sendTo(group, Port(1900),cstring(upnpquery),upnpquery.len.cint)
# [...]
let bytesReceived = recvFrom(socket, data, recvlen, recvaddr, recvport)
truedat101 commented 5 years ago

Great, thank you for the tips. I'll jump onto this tonight. As mentioned, I have this all working reliably in my java code and also C code version, but really wanted to make this a no-brainer thing to work with UPnP w/ NIM, and eliminate the need to do discovery using the Hue's funky nupnp solution that requires a proper internet connection.

truedat101 commented 5 years ago

So I've added 0.1.2 version of multicast to the project. Strange error when I compile.

multicast-0.1.2/multicast.nim(54, 6) Error: undeclared identifier: 'IPv4'

This should be present in IPAddressFamily. Anyway, is this something obvious I've missed, or should I file a ticket on multicast.

enthus1ast commented 5 years ago

@truedat101 strange that this works for me, since this is a pure enum mhh. Nonetheless i pushed a new version, please try if it now compiles for you.

truedat101 commented 5 years ago

@enthus1ast Thanks I'll give this a look. I guess the only thing I can think of that might be different is I'm on Mac OS X for this particular project and using num 0.17.0.

enthus1ast commented 5 years ago

i bet this issue is because of 0.17 but i see another issue comeing: it might be that IP_ADD_MEMBERSHIP and IP_DROP_MEMBERSHIP int values differs from posix (they differ from old windows to new windows and linux. I might have to add these values for mac. Unfortunately i have no access to a mac. So please report if its working :)

truedat101 commented 5 years ago

Ahh,ok, that does make sense. I can continue testing on the mac. At least that can get resolved through testing. I can move to a later release as well.

truedat101 commented 5 years ago

Great upgraded to multicast module 0.1.3. Build is fine.

On the run, I get: 

Traceback (most recent call last)
runTests.nim(12)         runTests
huenim.nim(155)          findHueHubsViaUPnP
system.nim(3613)         failedAssertImpl
system.nim(3605)         raiseAssert
system.nim(2724)         sysFatal

    Unhandled exception: true == joinGroup(socket, group, 255)
assert true == socket.joinGroup(group)
enthus1ast commented 5 years ago

@truedat101 i've pushed a new branch of multicast: https://github.com/enthus1ast/nimMulticast/tree/freebsdAndMacos

i still have no access to macos but i've installed a freebsd and tested the branch on it. I've also fixed the example (edit: i know the example was working with an older nim version, so it might be, if you still use 0.17 that i've just broke the example for you) so to test if multicast works on macos you could just run the tests/example.nim on two different machines.

Please report if macos works now, then i'll merge it into master :)

truedat101 commented 5 years ago

Apologies for the delay. I am sad to report my mac died. Awaiting its return from the shop and then I'll jump back onto testing that.

truedat101 commented 5 years ago

@enthus1ast mac is back. Going to give this a go.