Map doesn't update

[Notrial]

Since 0.89 I noticed that map doesn't update when I click on the update button (on the panel). Maybe it has something to do with the new component folder rules?

update: it updates the map when service is called but not via click on the map link in the panel

[pixeldoc2000]

I have the same issus with HA 0.89.1+ .

Update link does not work but service call works.

[rgruebel]

I'm not on 0.89 yet. Maybe something has changed in the webhooks, but I didn't see anything in the release notes.

[rgruebel]

I updated yesterday to 0.90 and couldn't reproduce the issue. Have you activated webhooks in the configuration?

[ikkentim]

I'm on 0.89, "update now" does not work:

Uncaught (in promise) ReferenceError: webhook is not defined
    at updateSource (map.html:88)
    at HTMLAnchorElement.onclick (map.html:49)

Also:

map.html:31 GET http://localhost:8123/local/zigbee2mqtt_networkmap/source.js?v=1553761906039 net::ERR_ABORTED 404 (Not Found)

EDIT: Never mind, my home assistant didn't properly restart.

[pixeldoc2000]

yes, webhook: is present in my HA config like the docs say: https://github.com/rgruebel/ha_zigbee2mqtt_networkmap

HA does not send a mqtt request for the map to zigbee2mqtt.

Same issus after HA Update to 0.90.2.

[rgruebel]

Sorry about the late answer. Were you able to solve the problem? Unfortunately I have no idea what it could be. Do you see error messages in the developer tools of your browser?

[Notrial]

I still can't update through the link. Automation works perfectly though and updates every X minutes.

[antst]

I have the same issue. Button "Update now" redraws map, and if maps was indeed updated by service in the mean time, it will be different. But there is no message with "zigbee2mqtt/bridge/networkmap graphviz" topic issued.

[Tche333]

Same issue for me, the log clearly indicates it is related to webhook. From log: [homeassistant.components.webhook] Received message for unregistered webhook f32e8f4e907fd...

despite the webhook is configured as per doc

[CodeFinder2]

I have the same problem.

Is this repo / component EOL?

[rgruebel]

No, if I can fix something, I'll do it. If the webhook does not work it can only be caused by the registration of the webhook or the URL when it is called. Do you have any error messages in your log files or in the developer console when you click the update button?

[codewise-nicolas]

I had the same problem about "webhook is not defined". Short answer: You must run the automation at least once (the 10 minute one in the example setup) in order to properly setup the source.js file. (Or try doing a graph request manually via mqtt publish)

It turns out there is code in the map.html that relies on a variable called "webhook". This variable is setup/assigned in the source.js file. The source.js file does not get created until the "message_received" function of the custom component is run (see line 46 of init.py https://github.com/rgruebel/ha_zigbee2mqtt_networkmap/blob/master/custom_components/zigbee2mqtt_networkmap/__init__.py) This doesn't run until a graph request result mqtt message comes thru.

Thus until the automation runs, or an mqtt graph message doesnt come thru the source.js file wont be setup and the webhook var wont be initialized and manually pressing on the "update now" button wont work.

Possible solution?

• Have the init.py setup/make a "bare bones" source.js that is empty.
• Check for webhook being defined in the calling code (map.html), if its undefined then present a message to the user.

[CodeFinder2]

No, if I can fix something, I'll do it. If the webhook does not work it can only be caused by the registration of the webhook or the URL when it is called. Do you have any error messages in your log files or in the developer console when you click the update button?

Great to hear that! IIRC, I had a warning in my log but forget to paste it here. I'll report back here if it occurs again. I'll later test, if there's any output in my browser's console and report back as well (unable to do that now).

Suprisingly, I tested it again using the "Update now" button after you commented here and it worked! I think it just doesn't work somewhat "reliably" (i.e., in some cases) and that's why people report the issue here (like me). Anyway, I know this is insufficient for you to debug / solve it so I'll try to provide more info ASAP. ;-)

[codewise-nicolas]

I just ran into the "Received message for unregistered webhook " problem as well. The problem again is how the source.js is setup/used and its tight integration with the 10minute automation.

The webhook url is setup in the source.js file via the incoming graph mqtt message as I noted in my previous post (above). However the url changes when HA is restarted. If the automation does not run/no graph mqtt message comes in, the source.js wont get updated with the new webhook url - and thus the web gui's update now link wont work.

How to test:

• Disable the 10 minute automation
• Restart HA
• Run "update now", it should fail to do anything and if you see the log you will see the error.
• Now run the automation (or trigger a graph mqtt message to come in from zigbee2mqtt)
• Run "update now" (as many times as you want). It will now work.

Solution:

• Better error handling of the ajax call to the webhook url, if 404 (or error) then dont proceed with code and let the user know
• Setup the updatenetwork map automation to run on homeassistant start event (if like me i dont have the automation as I dont need a map every 10 minutes, so this one is cleanest)
• or manually run the automation

hope this helps.

[CodeFinder2]

@codewise-nicolas Great insights! (I am not a JS / web programmer, so not an expert in this area.) I wonder if running on startup of HA is a good idea because I am not sure whether the Z2M addon will always be available already?

I think using (or even requiring) an automation at all is a very bad idea. In the Z2M docs, it is even recommended to only manually trigger an update due to the network load of the scan and in particular, slower coordinators like the CC2531 are unable to do much in parallel.

I would also suggest to not recommend / require that automation at all (also not in the README.md of this repo) for the given reason. If anyone still wants to use an automation, it would surely be still possible.

@rgruebel what do you think about this? Does it help to fix this issue?

[codewise-nicolas]

In my setup I did not want a network map every 10 minutes, I made an automation at HA start to trigger the update and thus setup the source.js file with the proper webhook url - and yes it will trigger a zigbee network map which could take some time in some cases but it works in my setup (only a few devices).

It would be nice to know if the other people's problems in this issue are solved/caused by my two issues I identified or if its something else; in which case I would stop posting here and start a new issue to not "steal" the issue.

I think the "proper" fix would be to have the source.js (or just the webhook var) setup when HA starts (or more specifically when the custom component is inited/started when ha starts) - no call to update the graph, just setup of the webhook var. Then when the user clicks the update now link/button the webhook call will work and trigger a real network map. Also some more error handling to the JS side for errors in the webhook call.

Ill see if I have some time tonight to work on improvements to the code, and post back with some updates; hopefully.

Went and edited the python and html to better support my ideas above.

• Made a new settings.js file to hold the webhook url
• The settings.js file is created when the plugin is setup (ie when HA starts)
• Added error handling to the map.html and visuals (ugly i know, but better than nothing) Find the updated files, give them a try and let me know. updates.zip

[rgruebel]

@codewise-nicolas Good point with the missing source.js In the README I wrote that the service should be called once, but probably this was not clear enough. I like your changes and they seem to work. Are you planning to create a Pull Request?

@CodeFinder2 i agree with you, i would not recommend automation anymore, especially not that often. I will change the README in the next time

[codewise-nicolas]

@codewise-nicolas Good point with the missing source.js In the README I wrote that the service should be called once, but probably this was not clear enough. I like your changes and they seem to work. Are you planning to create a Pull Request?

Ah yes I see it now, #5 on your list. I had assumed that was somewhat optional since it didnt mention it had to be run at least once, or mention that it needs to be run every time HA is restarted.

I wasn't planning on doing a PR, (dont hate me, but ive never been a fan of git, too complex of a workflow just to fix/submit a simple fix - I have to pull a whole repo, edit the files, commit, then push it, and whatnot. Maybe thats an idea for github, some way of editing repo files and submitting changes right on their website without having to clone). Hopefully someone can grab the update.zip files and test them and do a proper PR, its by no means complete/proper code it was a quick fix.

Thanks for the great custom component by the way. Simple and straightforward.

[CodeFinder2]

I observed another problem which may simply be related to this one: when clicking the "Update" button, sometimes the map gets updated properly and sometimes not. According to my observations, it seems to depend on how long z2m requires to do the network scan. If, for instance, there's a device not replying to the LQI ("ping") immediately or at all, the entire scan takes longer (and z2m internally waits due to timeouts being considered for such devices).

I don't know how this is implemented in this component but it seems there's just a fix amount of time which is far away from ideal regarding my just described situation.

@rgruebel what are your thoughts on this? Generally, are you planning to incorporate the fix provided by @codewise-nicolas ?

[codewise-nicolas]

I don't know how this is implemented in this component but it seems there's just a fix amount of time which is far away from ideal regarding my just described situation.

You hit it on the nail. The current code simply has a pre-set amount of time to wait before reloading. This may not be sufficient for the delays in the network scan. I personally dont know how we could make it dynamic (ie wait for the mqtt message to arrive with the response before reloading). It may take more ajax and fancier python code to do polling of some sort or events.

[CodeFinder2]

@rgruebel any plans on this? 🙈

[codewise-nicolas]

Thanks to your comment CodeFinder2 I decided to try tackling the problem again. Ive now added a check routine to the code so that once the request is sent, it will check every 5 seconds for the response. So far locally testing seems to work everytime.

Here is a copy of my custom component with all the changes. You should be able to just drop in and reload HASS and it should work. Please test it and let me know of any issues. As before, Im to lazy to try and do a proper PR so if someone else wants to tackle that part please do so rgruebel can update the repo for everyones benefit.

Updates since this message originally posted:

• Updated to use ajax for loading of the graph data instead of reloading the page
• More error handling

zigbee2mqtt_networkmap.zip

[CodeFinder2]

@codewise-nicolas I am short on time these days but I have just downloaded your files, applied them in my setup and will check if that works for me, too! I will then later create a Pull Request against this repo so that @rgruebel will hopefully update it (?).

Thank you very much for taking a second look on this!

EDIT: unfortunately, this does not seem to work / do any updates anymore... :see_no_evil: I tested on Linux with Chromium Browser, my console showed:

GET http://hassio:8123/local/community/zigbee2mqtt_networkmap/settings.js?v=.... net::ERR_ABORTED 404 (Not Found)
(anonymous) @ map.html:36
DevTools failed to parse SourceMap: chrome-extension://..../include.postload.js.map

Firefox' console shows (sorry, this is partially German):

Ein iframe, der sowohl "allow-scripts" als auch "allow-same-origin" für sein "sandbox"-Attribut gesetzt hat, kann seine Sandbox entfernen. networkmap
Laden fehlgeschlagen für das <script> mit der Quelle "http://hassio:8123/local/community/zigbee2mqtt_networkmap/settings.js?v=....". map.html:36:1
Successfully compiled asm.js code (total compilation time 157ms)

...when reloading the page (F5) and nothing is printed when I click the "Update and check" button on the page. (Hopefully, this helps ... I am not a web developer / expert in this regard, sorry :see_no_evil: ).

EDIT: let me know if I can provide more information. What type of browser did you use for testing, by the way?

[codewise-nicolas]

Ive tested in firefox and chrome on macos.

The problem is that "settings.js" is missing. Its possible HASS doesnt have permission to write the file f = open(hass.config.path('www', 'community', 'zigbee2mqtt_networkmap', 'settings.js'), "w") Can you check the permissions on that folder/file path.

Depending how you run HASS, check the log on startup for any errors related to the plugin. This is where you would see any problems with writing to that file.

There was originally a problem with this plugin where the settings.js was only written when step #5 was run, but i reworked things so that the file is generated when HASS starts and initializes the plugin. The settings.js file is loaded via a script tag, however I have error checking for the vars its supposed to setup when you click the button. Im surprised you dont get an alert.

Let me know what you find from the permissions and log.

[rgruebel]

Hi guys, sorry for the long delay, I'm also very busy at the moment. I tested the changes from @codewise-nicolas and everything works as expected. Thank you very much. I have already committed the changes. Thanks for your patience.

[CodeFinder2]

@codewise-nicolas reinstalled, flushed my browser cache and now it's working, great fix, thank you! :-)

And thanks for the merge!