Problem sending values over MQTT from nodered [not original title]

[dersch81]

VenusOS v3.20 is now released and after 4 month beta FlashMQ is the new broker.

As i can see there is still no support for it by dbus-mqtt-devices?

[freakent]

@dersch81 Please see the announcement I posted yesterday : https://github.com/freakent/dbus-mqtt-devices/discussions/85#discussion-6215729 A similar announecment was also posted on the Victron Modifications forum.

How are you coming to the conculsion the this driver still doesn't support FlashMQ?

(I have renamed the title of this issue to avoid confusion for other users)

[dersch81]

Oh sorry, didn't notice that in the changelog of 0.6.4. My conclusion was because i have not my custom mqtt devices in place as before. I'm registering the devices with nodered like before but nothing happens in venus dbus.

I can see them here:

@4000000065ce113616496fcc INFO:device_manager:Received device status message {'clientId': 'shellyht1E6D29', 'connected': 1, 'services': {'t1': 'temperature'}, 'version': '0.14'}
@4000000065ce113616c8cd44 INFO:device_manager:publish {'portalId': 'b827eb909369', 'deviceInstance': {'t1': 1}} to device/shellyht1E6D29/DBus, status is 0
@4000000065ce117216f2c75c INFO:device_manager:Received device status message {'clientId': 'hygrometer0776', 'connected': 1, 'services': {'t1': 'temperature'}, 'version': '1.0'}
@4000000065ce117217447d54 INFO:device_manager:publish {'portalId': 'b827eb909369', 'deviceInstance': {'t1': 3}} to device/hygrometer0776/DBus, status is 0
@4000000065ce117217677e44 INFO:device_manager:Received device status message {'clientId': 'shellyht1E6D29', 'connected': 1, 'services': {'t1': 'temperature'}, 'version': '0.14'}
@4000000065ce117217c9e994 INFO:device_manager:publish {'portalId': 'b827eb909369', 'deviceInstance': {'t1': 1}} to device/shellyht1E6D29/DBus, status is 0
@4000000065ce11ae221bfcac INFO:device_manager:Received device status message {'clientId': 'hygrometer0776', 'connected': 1, 'services': {'t1': 'temperature'}, 'version': '1.0'}
@4000000065ce11ae223be88c INFO:device_manager:publish {'portalId': 'b827eb909369', 'deviceInstance': {'t1': 3}} to device/hygrometer0776/DBus, status is 0
@4000000065ce11ae2267ced4 INFO:device_manager:Received device status message {'clientId': 'shellyht1E6D29', 'connected': 1, 'services': {'t1': 'temperature'}, 'version': '0.14'}
@4000000065ce11ae229d1b5c INFO:device_manager:publish {'portalId': 'b827eb909369', 'deviceInstance': {'t1': 1}} to device/shellyht1E6D29/DBus, status is 0
@4000000065ce11ea149c2944 INFO:device_manager:Received device status message {'clientId': 'hygrometer0776', 'connected': 1, 'services': {'t1': 'temperature'}, 'version': '1.0'}
@4000000065ce11ea14d45814 INFO:device_manager:publish {'portalId': 'b827eb909369', 'deviceInstance': {'t1': 3}} to device/hygrometer0776/DBus, status is 0
@4000000065ce11ea15012ca4 INFO:device_manager:Received device status message {'clientId': 'shellyht1E6D29', 'connected': 1, 'services': {'t1': 'temperature'}, 'version': '0.14'}
@4000000065ce11ea15356f8c INFO:device_manager:publish {'portalId': 'b827eb909369', 'deviceInstance': {'t1': 1}} to device/shellyht1E6D29/DBus, status is 0

I can see the W Topic

The devices are registered in dbus but no values are visible.

[freakent]

So what version of the driver do you have installed?

What do you see under the N/portalid>/temperature ? Something similar to below? You will need to issue a keepalive every 60 seconds. There is a python script in the samples directory that can help you with that.

[dersch81]

i'm running 0.6.4

nothing of my custom mqtt devices are at N. The keep alive is included in my node red flow. I'm running this since more than one year.

[freakent]

Please can you reboot the GX device then upload the whole fresh log file. Don't cut and paste a few lines it's not enough to see what is happening.

[dersch81]

@4000000065ce186619e5d83c *** starting dbus-mqtt-devices ***
@4000000065ce186d22d601c4 dbus_mqtt_devices v{} 0.6.4
@4000000065ce186d22d61934 INFO:logger:Loglevel set to INFO
@4000000065ce186d22f84734 INFO:dbus_mqtt_devices:-------- dbus_mqtt_devices, v0.6.4 is starting up --------
@4000000065ce186f316a332c INFO:device_manager:Using portalId xxx
@4000000065ce1872070bfe44 INFO:mqtt_gobject_bridge:[Init] Connecting to local broker
@4000000065ce18720f23f80c INFO:device_manager:[Connected] Result code 0
@4000000065ce187b1b0430ec INFO:device_manager:Received device status message {'clientId': 'hygrometer0776', 'connected': 1, 'services': {'t1': 'temperature'}, 'version': '1.0'}
@4000000065ce187b1b31345c INFO:device:**** Registering device: hygrometer0776, services: {'t1': 'temperature'} ****
@4000000065ce187b1b3f1edc INFO:device:Registering Service temperature for client hygrometer0776
@4000000065ce187b1b4e805c INFO:device_service_config:About to open config file
@4000000065ce187d09199cdc INFO:device_service:Registering service temperature for client hygrometer0776 at path com.victronenergy.temperature.mqtt_hygrometer0776_t1
@4000000065ce187d0d3bfa44 INFO:settingsdevice:Setting /Settings/Devices/mqtt_hygrometer0776_t1/ClassAndVrmInstance does not exist yet or must be adjusted
@4000000065ce187d1abe88e4 INFO:vedbus:registered ourselves on D-Bus as com.victronenergy.temperature.mqtt_hygrometer0776_t1
@4000000065ce187d1b3602d4 INFO:device_service:Registered Service com.victronenergy.temperature.mqtt_hygrometer0776_t1 under DeviceInstance 3
@4000000065ce187d1b6150c4 INFO:device_manager:publish {'portalId': 'xxx', 'deviceInstance': {'t1': 3}} to device/hygrometer0776/DBus, status is 0
@4000000065ce187d1da6242c INFO:device_manager:Received device status message {'clientId': 'shellyht1E6D29', 'connected': 1, 'services': {'t1': 'temperature'}, 'version': '0.14'}
@4000000065ce187d1dafc8ec INFO:device:**** Registering device: shellyht1E6D29, services: {'t1': 'temperature'} ****
@4000000065ce187d1dc92904 INFO:device:Registering Service temperature for client shellyht1E6D29
@4000000065ce187d1f410b94 INFO:device_service_config:About to open config file
@4000000065ce187f381ad8ac INFO:device_service:Registering service temperature for client shellyht1E6D29 at path com.victronenergy.temperature.mqtt_shellyht1E6D29_t1
@4000000065ce18800afd974c INFO:settingsdevice:Setting /Settings/Devices/mqtt_shellyht1E6D29_t1/ClassAndVrmInstance does not exist yet or must be adjusted
@4000000065ce188015e844cc INFO:vedbus:registered ourselves on D-Bus as com.victronenergy.temperature.mqtt_shellyht1E6D29_t1
@4000000065ce188016b299ac INFO:device_service:Registered Service com.victronenergy.temperature.mqtt_shellyht1E6D29_t1 under DeviceInstance 1
@4000000065ce188016e98444 INFO:device_manager:publish {'portalId': 'xxx', 'deviceInstance': {'t1': 1}} to device/shellyht1E6D29/DBus, status is 0
@4000000065ce18b70761a024 INFO:device_manager:Received device status message {'clientId': 'hygrometer0776', 'connected': 1, 'services': {'t1': 'temperature'}, 'version': '1.0'}
@4000000065ce18b707ac8404 INFO:device_manager:publish {'portalId': 'xxx', 'deviceInstance': {'t1': 3}} to device/hygrometer0776/DBus, status is 0
@4000000065ce18b707dbaa54 INFO:device_manager:Received device status message {'clientId': 'shellyht1E6D29', 'connected': 1, 'services': {'t1': 'temperature'}, 'version': '0.14'}
@4000000065ce18b7081056b4 INFO:device_manager:publish {'portalId': 'xxx', 'deviceInstance': {'t1': 1}} to device/shellyht1E6D29/DBus, status is 0
@4000000065ce18f30156e34c INFO:device_manager:Received device status message {'clientId': 'hygrometer0776', 'connected': 1, 'services': {'t1': 'temperature'}, 'version': '1.0'}
@4000000065ce18f3021e3a8c INFO:device_manager:publish {'portalId': 'xxx', 'deviceInstance': {'t1': 3}} to device/hygrometer0776/DBus, status is 0
@4000000065ce18f302412fc4 INFO:device_manager:Received device status message {'clientId': 'shellyht1E6D29', 'connected': 1, 'services': {'t1': 'temperature'}, 'version': '0.14'}
@4000000065ce18f3026c913c INFO:device_manager:publish {'portalId': 'xxx', 'deviceInstance': {'t1': 1}} to device/shellyht1E6D29/DBus, status is 0
@4000000065ce192f136683bc INFO:device_manager:Received device status message {'clientId': 'hygrometer0776', 'connected': 1, 'services': {'t1': 'temperature'}, 'version': '1.0'}
@4000000065ce192f13913184 INFO:device_manager:publish {'portalId': 'xxx', 'deviceInstance': {'t1': 3}} to device/hygrometer0776/DBus, status is 0
@4000000065ce192f13b451b4 INFO:device_manager:Received device status message {'clientId': 'shellyht1E6D29', 'connected': 1, 'services': {'t1': 'temperature'}, 'version': '0.14'}
@4000000065ce192f13f133a4 INFO:device_manager:publish {'portalId': 'xxx', 'deviceInstance': {'t1': 1}} to device/shellyht1E6D29/DBus, status is 0

But also this:

@4000000065ce184701da8684 *** starting dbus-mqtt-devices ***
@4000000065ce185e0ecf408c dbus_mqtt_devices v{} 0.6.4
@4000000065ce185e0ecf63b4 INFO:logger:Loglevel set to INFO
@4000000065ce185e0ecf6f6c INFO:dbus_mqtt_devices:-------- dbus_mqtt_devices, v0.6.4 is starting up --------
@4000000065ce185e1c142dd4 Traceback (most recent call last):
@4000000065ce185e1c144544   File "/usr/lib/python3.8/site-packages/dbus/bus.py", line 177, in activate_name_owner
@4000000065ce185e1c145cb4     return self.get_name_owner(bus_name)
@4000000065ce185e1c14686c   File "/usr/lib/python3.8/site-packages/dbus/bus.py", line 361, in get_name_owner
@4000000065ce185e1c147bf4     return self.call_blocking(BUS_DAEMON_NAME, BUS_DAEMON_PATH,
@4000000065ce185e1c148b94   File "/usr/lib/python3.8/site-packages/dbus/connection.py", line 666, in call_blocking
@4000000065ce185e1c169aec     reply_message = self.send_message_with_reply_and_block(
@4000000065ce185e1c16aa8c dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NameHasNoOwner: Could not get owner of name 'com.victronenergy.system': no such name
@4000000065ce185e1c16c5e4
@4000000065ce185e1c16c9cc During handling of the above exception, another exception occurred:
@4000000065ce185e1c16d96c
@4000000065ce185e1c173efc Traceback (most recent call last):
@4000000065ce185e1c174ab4   File "/data/drivers/dbus-mqtt-devices-0.6.4/dbus_mqtt_devices.py", line 83, in <module>
@4000000065ce185e1c175e3c     main()
@4000000065ce185e1c17660c   File "/data/drivers/dbus-mqtt-devices-0.6.4/dbus_mqtt_devices.py", line 65, in main
@4000000065ce185e1c177994     handler = MQTTDeviceManager(
@4000000065ce185e1c178164   File "/data/drivers/dbus-mqtt-devices-0.6.4/device_manager.py", line 37, in __init__
@4000000065ce185e1c1936e4     self.portalId = self._lookup_portalId()
@4000000065ce185e1c1a05ec   File "/data/drivers/dbus-mqtt-devices-0.6.4/device_manager.py", line 116, in _lookup_portalId
@4000000065ce185e1c1a1974     portalId = VeDbusItemImport(self._dbus_conn, "com.victronenergy.system", "/Serial").get_value()
@4000000065ce185e1c1a30e4   File "/data/drivers/dbus-mqtt-devices-0.6.4/ext/velib_python/vedbus.py", line 284, in __init__
@4000000065ce185e1c1a446c     self._proxy = bus.get_object(serviceName, path, introspect=False)
@4000000065ce185e1c1aa9fc   File "/usr/lib/python3.8/site-packages/dbus/bus.py", line 241, in get_object
@4000000065ce185e1c1abd84     return self.ProxyObjectClass(self, bus_name, object_path,
@4000000065ce185e1c1acd24   File "/usr/lib/python3.8/site-packages/dbus/proxies.py", line 250, in __init__
@4000000065ce185e1c1ae0ac     self._named_service = conn.activate_name_owner(bus_name)
@4000000065ce185e1c1af04c   File "/usr/lib/python3.8/site-packages/dbus/bus.py", line 182, in activate_name_owner
@4000000065ce185e1c1b55dc     self.start_service_by_name(bus_name)
@4000000065ce185e1c1b657c   File "/usr/lib/python3.8/site-packages/dbus/bus.py", line 277, in start_service_by_name
@4000000065ce185e1c1b7904     return (True, self.call_blocking(BUS_DAEMON_NAME, BUS_DAEMON_PATH,
@4000000065ce185e1c1b88a4   File "/usr/lib/python3.8/site-packages/dbus/connection.py", line 666, in call_blocking
@4000000065ce185e1c1c4bf4     reply_message = self.send_message_with_reply_and_block(
@4000000065ce185e1c1c5b94 dbus.exceptions.DBusException: org.freedesktop.DBus.Error.ServiceUnknown: The name com.victronenergy.system was not provided by any .service files
@4000000065ce185f1412735c *** starting dbus-mqtt-devices ***
@4000000065ce18650e588604 dbus_mqtt_devices v{} 0.6.4
@4000000065ce18650e58a15c INFO:logger:Loglevel set to INFO
@4000000065ce18650ee898d4 INFO:dbus_mqtt_devices:-------- dbus_mqtt_devices, v0.6.4 is starting up --------
@4000000065ce1865208a19dc Traceback (most recent call last):
@4000000065ce1865208a3534   File "/usr/lib/python3.8/site-packages/dbus/bus.py", line 177, in activate_name_owner
@4000000065ce1865208a48bc     return self.get_name_owner(bus_name)
@4000000065ce1865208a5474   File "/usr/lib/python3.8/site-packages/dbus/bus.py", line 361, in get_name_owner
@4000000065ce1865208a67fc     return self.call_blocking(BUS_DAEMON_NAME, BUS_DAEMON_PATH,
@4000000065ce1865208a779c   File "/usr/lib/python3.8/site-packages/dbus/connection.py", line 666, in call_blocking
@4000000065ce1865208bfa54     reply_message = self.send_message_with_reply_and_block(
@4000000065ce1865208c09f4 dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NameHasNoOwner: Could not get owner of name 'com.victronenergy.system': no such name
@4000000065ce1865208c254c
@4000000065ce1865208c2934 During handling of the above exception, another exception occurred:
@4000000065ce1865208c38d4
@4000000065ce1865208c9694 Traceback (most recent call last):
@4000000065ce1865208ca24c   File "/data/drivers/dbus-mqtt-devices-0.6.4/dbus_mqtt_devices.py", line 83, in <module>
@4000000065ce1865208cb5d4     main()
@4000000065ce1865208cbda4   File "/data/drivers/dbus-mqtt-devices-0.6.4/dbus_mqtt_devices.py", line 65, in main
@4000000065ce1865208cd12c     handler = MQTTDeviceManager(
@4000000065ce1865208cd8fc   File "/data/drivers/dbus-mqtt-devices-0.6.4/device_manager.py", line 37, in __init__
@4000000065ce1865208d4274     self.portalId = self._lookup_portalId()
@4000000065ce1865208dfa0c   File "/data/drivers/dbus-mqtt-devices-0.6.4/device_manager.py", line 116, in _lookup_portalId
@4000000065ce1865208e0d94     portalId = VeDbusItemImport(self._dbus_conn, "com.victronenergy.system", "/Serial").get_value()
@4000000065ce1865208e2504   File "/data/drivers/dbus-mqtt-devices-0.6.4/ext/velib_python/vedbus.py", line 284, in __init__
@4000000065ce1865208e388c     self._proxy = bus.get_object(serviceName, path, introspect=False)
@4000000065ce1865208e9e1c   File "/usr/lib/python3.8/site-packages/dbus/bus.py", line 241, in get_object
@4000000065ce1865208eb1a4     return self.ProxyObjectClass(self, bus_name, object_path,
@4000000065ce1865208ec144   File "/usr/lib/python3.8/site-packages/dbus/proxies.py", line 250, in __init__
@4000000065ce1865208ed0e4     self._named_service = conn.activate_name_owner(bus_name)
@4000000065ce1865208ee084   File "/usr/lib/python3.8/site-packages/dbus/bus.py", line 182, in activate_name_owner
@4000000065ce186520905b6c     self.start_service_by_name(bus_name)
@4000000065ce186520906724   File "/usr/lib/python3.8/site-packages/dbus/bus.py", line 277, in start_service_by_name
@4000000065ce186520907aac     return (True, self.call_blocking(BUS_DAEMON_NAME, BUS_DAEMON_PATH,
@4000000065ce186520908e34   File "/usr/lib/python3.8/site-packages/dbus/connection.py", line 666, in call_blocking
@4000000065ce186520914d9c     reply_message = self.send_message_with_reply_and_block(
@4000000065ce186520915d3c dbus.exceptions.DBusException: org.freedesktop.DBus.Error.ServiceUnknown: The name com.victronenergy.system was not provided by any .service files

[freakent]

That looks like a stability issue with the dbus itself. The driver was unable to connect to the dbus.

In the first log file you seem to be re-registering the device multiple times, is there a reason for that?

[dersch81]

Its pretty strange. Now everything is also at Topic N, all values are frequently written to W but not at N.

In the first log i manually registered the items with the hope it will start to show values.

[freakent]

Do you understand what the W, N and R topic branches are used for? W = Write, N = Notification, R = read request You should publish your values to W and the dbus will publish it's values to N. That is why you should monitor N.

[dersch81]

Yes i know that

[freakent]

Can you try running the combo-simulator.py script from your local computer (not on the Venus device)? It's in the dbus-mqtt-devices/samples directory and you will need Python installed.

[dersch81]

It is working again as before. Cause was my PoE HAT of the Pi and the additional connected USB powered touchscreen....