ISY 404 errors on REST add node API

[mjwesterhof]

With 5.0.2 firmware, the ISY responds with a "404" to requests to add a node:

ERROR [04-08-2016 10:45:56] polyglot.element_manager.isy: ISY: ( 0.04) [404] (OK) http://192.168.232.79:80/rest/ns/1/nodes/n001_hub/add/HUB?primary=n001_hub&name=Hue+Hub Discussions on the UDI forum have suggested that this is because the node is already added on the ISY, but because of a missing response from the ISY when the node was originally added, Polyglot attempts to re-add the node. (The node in the example above does indeed exist on the ISY)

This issue is opened to capture this issue. We need to figure out what happened when the node was added that causes Polyglot to think the node is missing. We also need to take a look at the HTTP status code - a 404 usually means that the API doesn't exist, or perhaps that the resource named in the API call doesn't exist... perhaps there's a better response we can provide from the ISY that would indicate that the call in improperly formed, or invalid? (Or, perhaps if the resource exists with the same name and primary, perhaps we just return a 200 status, which would be more compatible with a stateless protocol like REST, especially if we implement retries in Polyglot in the future...)

[mkohanim]

Hi Mike,

How many times does Polyglot reattempt adding the node?

With kind regards,

---

Michel Kohanim CEO

(p) 818.631.0333 (f) 818.436.0702 http://www.universal-devices.comhttp://www.universal-devices.com/

---

From: Mike Westerhof [mailto:notifications@github.com] Sent: Friday, April 8, 2016 9:54 AM To: UniversalDevicesInc/Polyglot Polyglot@noreply.github.com Subject: [UniversalDevicesInc/Polyglot] ISY 404 errors on REST add node API (#41)

With 5.0.2 firmware, the ISY responds with a "404" to requests to add a node:

ERROR [04-08-2016 10:45:56] polyglot.element_manager.isy: ISY: ( 0.04) 404 http://192.168.232.79:80/rest/ns/1/nodes/n001_hub/add/HUB?primary=n001_hub&name=Hue+Hub

Discussions on the UDI forum have suggested that this is because the node is already added on the ISY, but because of a missing response from the ISY when the node was originally added, Polyglot attempts to re-add the node. (The node in the example above does indeed exist on the ISY)

This issue is opened to capture this issue. We need to figure out what happened when the node was added that causes Polyglot to think the node is missing. We also need to take a look at the HTTP status code - a 404 usually means that the API doesn't exist, or perhaps that the resource named in the API call doesn't exist... perhaps there's a better response we can provide from the ISY that would indicate that the call in improperly formed, or invalid? (Or, perhaps if the resource exists with the same name and primary, perhaps we just return a 200 status, which would be more compatible with a stateless protocol like REST, especially if we implement retries in Polyglot in the future...)

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHubhttps://github.com/UniversalDevicesInc/Polyglot/issues/41

[mjwesterhof]

Each time a given node server starts or restarts it will issue the node add REST call.

My vague understanding of this is that Polyglot consults the configuration for the current node server, and determines that the node has not been added -- and that's why it issues the node add REST call.

[mkohanim]

Thanks Mike. This is a strange decision. Polyglot should already keep all the configuration information so that it does not have to keep sending an add request. But, for now, this can wait.

Thanks again.

With kind regards,

---

Michel Kohanim CEO

(p) 818.631.0333 (f) 818.436.0702 http://www.universal-devices.comhttp://www.universal-devices.com/

---

From: Mike Westerhof [mailto:notifications@github.com] Sent: Friday, April 8, 2016 10:14 AM To: UniversalDevicesInc/Polyglot Polyglot@noreply.github.com Cc: Michel Kohanim michel@universal-devices.com Subject: Re: [UniversalDevicesInc/Polyglot] ISY 404 errors on REST add node API (#41)

Each time a given node server starts or restarts it will issue the node add REST call.

My vague understanding of this is that Polyglot consults the configuration for the current node server, and determines that the node has not been added -- and that's why it issues the node add REST call.

— You are receiving this because you commented. Reply to this email directly or view it on GitHubhttps://github.com/UniversalDevicesInc/Polyglot/issues/41#issuecomment-207521355

[mjwesterhof]

The root cause of this problem is that the ISY, at least at v5.0.2, does not respond to the original add REST API call with the expected report REST call back to Polyglot.

This sequence illustrates the "node add" issue.

Initial conditions: A Polyglot node server has been defined, started, and has run for some time. An external event occurs that creates a new child node (e.g. a new Kodi server is found on the network, a new Philips Hue bulb is plugged in and added to the Hue hub, etc.). The node server calls the node server API on the ISY to add the new node. However, the ISY does not respond with the REST call back to the node server to tell it that the node has been added.

This first log snippet shows what happens currently when the above scenario happens. Line 1 is the startup of the node server. Line 2 can be ignored for the moment - the 404 shouldn't happen, and is an instance of this issue, but we're focused on the child node, not the parent right now. Line 3 shows the add of the newly-found child node (a kodi server named "Stinky") - the 200 return is what we expect. Line 4 shows the node server immediately updating the new child node's status. All is well... except, we didn't see the REST call from the ISY back to Polyglot to tell it that the new node has been added. Thus, the Polyglot configuration indicates that the child node has not yet been added to the ISY.

(1) INFO     [04-10-2016 12:41:10] polyglot.nodeserver_manager: Started Node Server: kodi:kodiak (23080)
(2) ERROR    [04-10-2016 12:41:11] polyglot.element_manager.isy: ISY:        ( 0.03) 404 ERR: http://192.168.232.79:80/rest/ns/4/nodes/n004_disco/add/KODIDISCO?primary=n004_disco&name=Kodi+Discovery
(3) INFO     [04-10-2016 12:41:22] polyglot.element_manager.isy: ISY:        ( 0.26) 200  OK: http://192.168.232.79:80/rest/ns/4/nodes/n004_906b6ed65d96e3/add/KODI?primary=n004_disco&name=Stinky
(4) INFO     [04-10-2016 12:41:22] polyglot.element_manager.isy: ISY:        ( 0.14) 200  OK: http://192.168.232.79:80/rest/ns/4/nodes/n004_906b6ed65d96e3/report/status/ST/3/25

The next snippet shows what happens when the node server is restarted for any reason. Line 5 shows the restart. Line 6 shows the same 404 error as before, just like line 5 -- this is another facet of the aforementioned problem. Because the ISY didn't tell us that it added the node server node, Polyglot assumes that it is not yet added, and attempts to add it again. The ISY apparently responds with a 404 to indicate that the node already exists, or perhaps the 404 status is a generic indicator of some sort.

(5) INFO     [04-10-2016 12:42:53] polyglot.nodeserver_manager: Started Node Server: kodi:kodiak (23127)
(6) ERROR    [04-10-2016 12:42:54] polyglot.element_manager.isy: ISY:        ( 0.04) 404 ERR: http://192.168.232.79:80/rest/ns/4/nodes/n004_disco/add/KODIDISCO?primary=n004_disco&name=Kodi+Discovery
(7) ERROR    [04-10-2016 12:43:06] polyglot.element_manager.isy: ISY:        ( 0.04) 404 ERR: http://192.168.232.79:80/rest/ns/4/nodes/n004_906b6ed65d96e3/add/KODI?primary=n004_disco&name=Stinky
(8) INFO     [04-10-2016 12:43:06] polyglot.element_manager.isy: ISY:        ( 0.03) 200  OK: http://192.168.232.79:80/rest/ns/4/nodes/n004_906b6ed65d96e3/report/status/ST/3/25

Note that this isn't fatal - line 8 shows that the subsequent status update to the same node happens just fine. So this is mainly log clutter for right now.

So, problem a) The ISY isn't sending back the expected message to indicate that it added a node.

And, problem b) A 404 error may not be the one to send back; it tends to indicate that the resource doesn't exist, rather than that it exists already. Perhaps a 200 OK is better if the node already exists and matches the information sent in the REST call? Or maybe there's another status code that would be better to use to indicate that the resource (URI) is fine, but the action cannot be completed?

[mjwesterhof]

Just committed the start of a general diagnostic/node-management mechanism, which includes node synchronization. So the questions above remain -- what IS the best way to determine node state changes on the ISY? -- but this mechanism will ensure that the nodes in Polyglot will reflect the state of those nodes in the ISY, albeit perhaps with a multi-minute delay.