Closed NightMean closed 1 month ago
Thanks for the report!
I normalized the string, e.g. ý
-> y
before passing it to the broker.
This should be available in the next release. It would be great if you could test this. Can you please switch to the netalertx-dev
docker image (backup everything at first), in about 15 minutes (or after the last action finishes) from now.
Make sure you refresh your browser cache - and click the 🔄 refresh button in the top right corner.
Thanks in advance, j
Normalization works as well :) Thanks for the quick fix! I was able to see it straight away when I switched from prod to dev branch.
However after doing some further testing with the dev branch from scratch, there seems to be an issue with the mqtt publisher. I'm able to see all devices after couple of minutes in HomeAssistant in prod but if I delete the topic, move to dev branch (different volumes), the mqtt data is almost never picked up by HomeAssistant. I was able to sometimes see 25/36 devices but I'm not able to reproduce it consistently. The devices do show up in MQTT Explorer but not in HomeAssistant. I think, it might be related to fix from b7b2e0bc6572fc66c9c13079b71c6aa9abbd7bac but I can't confirm that.
Will do some more testing and then create a new issue for this when I get to the bottom of it :)
Thanks for testing - let me know how you go.
FYI @ingoratsdorf - just so you know there might be an issue with the latest MQTT changes - let's see what @NightMean comes back with.
The latest MQTT changes only pertain to connecting up with the broker properly. None of the publishing routines have been touched. If stuff shows up in MQTT Explorer but not in HA, then this may be an issue with the HA config in MQTT, not the actual topic publishing.
Just to remember, HA relies on the actual published data (/system-sensors/#
) and the config section (/homeassistant/#
).
We'd need to know where in MQTT Explorer things are sitting.
@NightMean, could you please check system-sensors/sensor/mac_xy_yz_ee_bb_cc_aa/state
in MQTT Explorer
vs the actual sensor config in homeassistant in MQTT Explorer
homeassistant/sensor/mac_xy_yz_ee_bb_cc_aa/mac_address/config
Please note if you want to delete a sensor in HA, you have to delete the HA config, ie either through HA itself or delete the config section in MQTT Explorer in Homeassistant. Deleting a publish top will not delete the HA config and the sensor changes will not be picked up.
It also looks like if nothing on the device changes, the device config is NOT getting published as it is in the NetAlertX database. I deleted for a test my 'Bonny' server config in HA and it is not getting published again from Netalertx. NetalertX is publishing the topic, but not the HA config again, which I deleted via HA.
14:07:30 [MQTT] Existing, skip Device Name : Bonny on Proxmox1
14:07:30 [MQTT] Publishing sensor number 0
14:07:30 [MQTT] Existing, skip Device Name : Bonny on Proxmox1
14:07:30 [MQTT] Publishing sensor number 0
14:07:30 [MQTT] Existing, skip Device Name : Bonny on Proxmox1
14:07:30 [MQTT] Publishing sensor number 0
14:07:30 [MQTT] Existing, skip Device Name : Bonny on Proxmox1
14:07:30 [MQTT] Publishing sensor number 0
14:07:30 [MQTT] Existing, skip Device Name : Bonny on Proxmox1
14:07:30 [MQTT] Publishing sensor number 0
14:07:30 [MQTT] Existing, skip Device Name : Bonny on Proxmox1
14:07:30 [MQTT] Publishing sensor number 0
14:07:30 [MQTT] Sending MQTT topic: system-sensors/sensor/mac_bc_24_11_8e_f8_00/state
14:07:30 [MQTT] Sending MQTT message: {"last_ip": "192.168.100.2", "is_new": "0", "vendor": "Proxmox Server Solutions GmbH", "mac_address": "bc:24:11:8e:f8:00", "last_connection": "2024-09-22 18:06:36", "first_connection": "2024-09-02 06:55:55+12:00"}
14:07:30 [MQTT] Existing, skip Device Name : Bonny on Proxmox1
14:07:30 [MQTT] Publishing sensor number 0
14:07:30 [MQTT] Sending MQTT topic: system-sensors/binary_sensor/mac_bc_24_11_8e_f8_00/state
14:07:30 [MQTT] Sending MQTT message: {"is_present": "OFF"}
14:07:30 [MQTT] Existing, skip Device Name : Bonny on Proxmox1
14:07:30 [MQTT] Publishing sensor number 0
14:07:30 [MQTT] Sending MQTT topic: system-sensors/device_tracker/mac_bc_24_11_8e_f8_00/state
14:07:30 [MQTT] Sending MQTT message: away
14:07:30 [MQTT] Sending MQTT topic: system-sensors/device_tracker/mac_bc_24_11_8e_f8_00/attributes
14:07:30 [MQTT] Sending MQTT message: {"last_ip": "192.168.100.2", "is_new": "0", "vendor": "Proxmox Server Solutions GmbH", "mac_address": "bc:24:11:8e:f8:00", "last_connection": "2024-09-22 18:06:36", "first_connection": "2024-09-02 06:55:55+12:00"}
As a workaround, one could be checking if the config is still there before publishing the topic, but that would take more calls and logic than simply publishing the config again every time (which would in turn add quite some overhead).
Ideas?
On that matter, I changed the location field for my bonny server, but it still get's skipped in the config publish. Looks like we might also update the HA config when any device info changes?
Looking at the code in mqtt.py, the sensor is only getting updated on changes to id, name, type and icon, not anything else like location etc.
# Define your input string
input_string = str(self.deviceId) + str(self.deviceName) + str(self.sensorType) + str(self.sensorName) + str(self.icon)
# Hash the input string and convert the hash to a string
# Update the hash object with the bytes of the input string
md5_hash.update(input_string.encode('utf-8'))
# Get the hexadecimal representation of the MD5 hash
md5_hash_hex = md5_hash.hexdigest()
hash_value = str(md5_hash_hex)
self.hash = hash_value
plugObj = getPluginObject({"Plugin":"MQTT", "Watched_Value3":hash_value})
# mylog('verbose', [f"[{pluginName}] Previous plugin object entry: {json.dumps(plugObj)}"])
if plugObj == {}:
self.isNew = True
mylog('verbose', [f"[{pluginName}] New sensor entry name : {self.deviceName}"])
mylog('verbose', [f"[{pluginName}] New sensor entry mac : {self.mac}"])
mylog('verbose', [f"[{pluginName}] New sensor entry hash_value : {hash_value}"])
else:
device_name = plugObj.get("Watched_Value1", "Unknown")
mylog('verbose', [f"[{pluginName}] Existing, skip Device Name : {device_name}"])
self.isNew = False
So I changed the icon and name and voila, the new config is geeting published to HA:
14:36:44 [MQTT] New sensor entry name : Bonny on Proxmox1_
14:36:44 [MQTT] Sending MQTT message: {"name": "last_ip", "state_topic": "system-sensors/sensor/mac_bc_24_11_8e_f8_00/state", "value_template": "{{value_json.last_ip}}", "unique_id": "mac_bc_24_11_8e_f8_00_sensor_last_ip", "device": {"identifiers": ["mac_bc_24_11_8e_f8_00_sensor"], "manufacturer": "NetAlertX", "name": "Bonny on Proxmox1_"}, "icon": "mdi:ip-network"}
14:36:45 [MQTT] New sensor entry name : Bonny on Proxmox1_
14:36:45 [MQTT] Sending MQTT message: {"name": "mac_address", "state_topic": "system-sensors/sensor/mac_bc_24_11_8e_f8_00/state", "value_template": "{{value_json.mac_address}}", "unique_id": "mac_bc_24_11_8e_f8_00_sensor_mac_address", "device": {"identifiers": ["mac_bc_24_11_8e_f8_00_sensor"], "manufacturer": "NetAlertX", "name": "Bonny on Proxmox1_"}, "icon": "mdi:folder-key-network"}
14:36:46 [MQTT] New sensor entry name : Bonny on Proxmox1_
14:36:46 [MQTT] Sending MQTT message: {"name": "is_new", "state_topic": "system-sensors/sensor/mac_bc_24_11_8e_f8_00/state", "value_template": "{{value_json.is_new}}", "unique_id": "mac_bc_24_11_8e_f8_00_sensor_is_new", "device": {"identifiers": ["mac_bc_24_11_8e_f8_00_sensor"], "manufacturer": "NetAlertX", "name": "Bonny on Proxmox1_"}, "icon": "mdi:bell-alert-outline"}
14:36:47 [MQTT] New sensor entry name : Bonny on Proxmox1_
14:36:47 [MQTT] Sending MQTT message: {"name": "vendor", "state_topic": "system-sensors/sensor/mac_bc_24_11_8e_f8_00/state", "value_template": "{{value_json.vendor}}", "unique_id": "mac_bc_24_11_8e_f8_00_sensor_vendor", "device": {"identifiers": ["mac_bc_24_11_8e_f8_00_sensor"], "manufacturer": "NetAlertX", "name": "Bonny on Proxmox1_"}, "icon": "mdi:cog"}
14:36:48 [MQTT] New sensor entry name : Bonny on Proxmox1_
14:36:48 [MQTT] Sending MQTT message: {"name": "first_connection", "state_topic": "system-sensors/sensor/mac_bc_24_11_8e_f8_00/state", "value_template": "{{value_json.first_connection}}", "unique_id": "mac_bc_24_11_8e_f8_00_sensor_first_connection", "device": {"identifiers": ["mac_bc_24_11_8e_f8_00_sensor"], "manufacturer": "NetAlertX", "name": "Bonny on Proxmox1_"}, "icon": "mdi:calendar-start"}
14:36:49 [MQTT] New sensor entry name : Bonny on Proxmox1_
14:36:49 [MQTT] Sending MQTT message: {"name": "last_connection", "state_topic": "system-sensors/sensor/mac_bc_24_11_8e_f8_00/state", "value_template": "{{value_json.last_connection}}", "unique_id": "mac_bc_24_11_8e_f8_00_sensor_last_connection", "device": {"identifiers": ["mac_bc_24_11_8e_f8_00_sensor"], "manufacturer": "NetAlertX", "name": "Bonny on Proxmox1_"}, "icon": "mdi:calendar-end"}
14:36:50 [MQTT] New sensor entry name : Bonny on Proxmox1_
14:36:50 [MQTT] Sending MQTT message: {"name": "is_present", "state_topic": "system-sensors/binary_sensor/mac_bc_24_11_8e_f8_00/state", "value_template": "{{value_json.is_present}}", "unique_id": "mac_bc_24_11_8e_f8_00_sensor_is_present", "device": {"identifiers": ["mac_bc_24_11_8e_f8_00_sensor"], "manufacturer": "NetAlertX", "name": "Bonny on Proxmox1_"}, "icon": "mdi:wifi"}
14:36:51 [MQTT] New sensor entry name : Bonny on Proxmox1_
14:36:51 [MQTT] Sending MQTT message: {"state_topic": "system-sensors/device_tracker/mac_bc_24_11_8e_f8_00/state", "json_attributes_topic": "system-sensors/device_tracker/mac_bc_24_11_8e_f8_00/attributes", "name": "is_home", "payload_home": "home", "payload_not_home": "away", "unique_id": "mac_bc_24_11_8e_f8_00_device_tracker_is_home", "icon": "mdi:home", "device": {"identifiers": ["mac_bc_24_11_8e_f8_00_sensor", "mac_bc_24_11_8e_f8_00_device_tracker_is_home"], "manufacturer": "NetAlertX", "name": "Bonny on Proxmox1_"}}
If stuff shows up in MQTT Explorer but not in HA, then this may be an issue with the HA config in MQTT, not the actual topic publishing.
I'm not sure about this. The data in MQTT explorer looked the same for me from prod to dev branch but for some reason, HA refused to show those devices. It was very strange when I was playing with it yesterday.
Please note if you want to delete a sensor in HA, you have to delete the HA config, ie either through HA itself or delete the config section in MQTT Explorer in Homeassistant. Deleting a publish top will not delete the HA config and the sensor changes will not be picked up.
Honestly, I was restoring my HA backup in Proxmox everytime I cleaned it, just to be sure :) Regardless of that, the results were very different almost each time. Although it worked more times in prod than in dev.
It also looks like if nothing on the device changes, the device config is NOT getting published as it is in the NetAlertX database.
I also noticed that. Afterwards instead of long restore process in Proxmox I just deleted the MQTT integration and added it again. That seemed to work most of the times. Now that you mentioned this, this is actually what led me to play with this as I deleted the entity in HASS for the test above and it was not getting published anymore. At least I was not hallucinating :)
Will be playing with this deeper today/tomorrow.
homeassistant/sensor/mac_xy_yz_ee_bb_cc_aa/mac_address/config
.
I can see all devices in system-sensors/sensor/mac_xy_yz_ee_bb_cc_aa/state
Other device that shows as expected:
From what I noticed from the logs above, NetAlertX thinks that the devices already exist but in fact, HA never saw them. Maybe it's somewhere in the NetAlertX MQTT database as I'm using the same stack which I used since day 1 with all of the data.
system-sensors
but nothing really happened other than that NetAlertX started populating the topic again. Portainer
to Portainers
and after couple of seconds, I had 46 out of 47 devices discovered in HA. Renamed existing device that was previously imported in NetAlertX, devices started appearing in HA.
Internet
was not imported, same issue as previously mentioned with other devices.
After renaming, the Internet
device was also imported. In conclusion, I think that yesterday I got confused when all devices did not appear so I was switching from one docker stack to another, I might have done some mistake during that. Today, I can only confirm that the MQTT does not update the device unless it's removed which I'd consider as a bug. Oh and also, the string normalization works as expected :) Thanks!
Hi,
This behavior is by design to speed up notification processing. I don't consider changing secondary fields and updating them as an issue. The devices can be force updated by removing the Plugin objects associated with the MQTT plugin. I clarified the documentation on the MQTT plugin describing in detail how to force an update on all devices. Hoipe this works:
https://github.com/jokob-sk/NetAlertX/tree/main/front/plugins/_publisher_mqtt#forcing-an-update
Let me know if you have any feedback, Thanks, j
Seems fine to me. Thanks! I'd just modify the readme for HomeAssistant as well to include the information that if a device is deleted from HA, it requires you to force an update of MQTT in NetAlertX. Screenshot just for reference, in case you are not running HA.
Good idea - docs updated. 👍
Looks good to me, thanks!
Is there an existing issue for this?
Current Behavior
Devices which have diacritics in their name (e.g. ľ,š,č,ť,ž,ý,á,í,é,ô, etc..) are not correctly published into MQTT. I have a device name called
Peter-NTB - Firemný) which is correctly shown in the NetAlertX UI but when the device is sent to MQTT, the
ý` is stripped out.Home Assistant:
Expected Behavior
MQTT should send the full name even with the diacritics
Steps To Reproduce
app.conf
No response
docker-compose.yml
No response
What branch are you running?
Production
app.log
23:18:18 [MQTT] Sending MQTT message: {"last_ip": "192.168.1.178", "is_new": "1", "vendor": "OnePlus Tech Shenzhen Ltd", "mac_address": "xy:yz:ee:bb:cc:aa", "last_connection": "2024-09-28 21:05:36+02:00", "first_connection": "2024-09-28 21:05:36+02:00"} 23:18:18 [MQTT] Existing, skip Device Name : Michal-NTB - Firemn 23:18:18 [MQTT] Publishing sensor number 0 23:18:18 [MQTT] Existing, skip Device Name : Michal-NTB - Firemn 23:18:18 [MQTT] Publishing sensor number 0 23:18:18 [MQTT] Existing, skip Device Name : Michal-NTB - Firemn 23:18:18 [MQTT] Publishing sensor number 0 23:18:18 [MQTT] Existing, skip Device Name : Michal-NTB - Firemn 23:18:18 [MQTT] Publishing sensor number 0 23:18:18 [MQTT] Existing, skip Device Name : Michal-NTB - Firemn 23:18:18 [MQTT] Publishing sensor number 0 23:18:18 [MQTT] Existing, skip Device Name : Michal-NTB - Firemn 23:18:18 [MQTT] Publishing sensor number 0 23:18:18 [MQTT] Sending MQTT topic: system-sensors/sensor/mac_xy_yz_ee_bb_cc_aa/state 23:18:18 [MQTT] Sending MQTT message: {"last_ip": "192.168.1.210", "is_new": "1", "vendor": "Liteon Technology Corporation", "mac_address": "xy_yz_ee_bb_cc_aa", "last_connection": "2024-09-28 21:10:42+02:00", "first_connection": "2024-09-28 21:10:42+02:00"} 23:18:18 [MQTT] Existing, skip Device Name : Michal-NTB - Firemn 23:18:18 [MQTT] Publishing sensor number 0 23:18:18 [MQTT] Sending MQTT topic: system-sensors/binary_sensor/mac_xy_yz_ee_bb_cc_aa/state 23:18:18 [MQTT] Sending MQTT message: {"is_present": "OFF"} 23:18:18 [MQTT] Existing, skip Device Name : Michal-NTB - Firemn 23:18:18 [MQTT] Publishing sensor number 0 23:18:18 [MQTT] Sending MQTT topic: system-sensors/device_tracker/mac_xy_yz_ee_bb_cc_aa/state 23:18:18 [MQTT] Sending MQTT message: away 23:18:18 [MQTT] Sending MQTT topic: system-sensors/device_tracker/mac_xy_yz_ee_bb_cc_aa/attributes 23:18:18 [MQTT] Sending MQTT message: {"last_ip": "192.168.1.210", "is_new": "1", "vendor": "Liteon Technology Corporation", "mac_address": "xy_yz_ee_bb_cc_aa", "last_connection": "2024-09-28 21:10:42+02:00", "first_connection": "2024-09-28 21:10:42+02:00"}
Debug enabled