switch over to MQTT instead of HTTP ?

[arrow1800]

hi all,

I am thinking about switching over to MQTT instead of http. the http protocol is slow and has a lot of overhead and is therefore less suitable for requests every second.

MQTT on the other hand is; advantages would be;

• more updates per second.
• more reliable ?

disadvantage would be;

• the need of a MQTT server. however VenusOS already provides this. you can enable this somewhere in the settings.

Any ideas or thoughts?

[morres83]

I also thought about this, and I have a script for the EV wall charger based on MQTT already running. I think it is no problem that I adapt the virtual battery script to MQTT. However for the wall charger script I used MQTT just with standard options and I am not expert on QoS, LastWill and all those other things. Maybe we'll need to look deeper into those options to see what is best for the battery script.

I think I will make a branch in my github trying to adapt it, what do you think?

[teefixx]

Hi Even if I cannot support you, I would appreciate it. As I run Venus large, the MQTT server is already installed. To be honest, I don't think, that we need a higher update rate than 1s. Anyway, it would bundle all date coming in (at least on my system) to MQTT

[arrow1800]

I also thought about this, and I have a script for the EV wall charger based on MQTT already running. I think it is no problem that I adapt the virtual battery script to MQTT. However for the wall charger script I used MQTT just with standard options and I am not expert on QoS, LastWill and all those other things. Maybe we'll need to look deeper into those options to see what is best for the battery script.

I think I will make a branch in my github trying to adapt it, what do you think?

great suggestion. i think i will do the same. lets share our knowledge here when we know more.

[arrow1800]

@morres83, i would like to know how you got that screenshot. what did you do in node-red to get that overview of available topics ?

[morres83]

I checked it by using the MQTT Explorer. You'll need to publish a keepalive message first and then Victron will publish everything, or what you tell it to publish in the payload.

Check here for the syntax: https://github.com/victronenergy/dbus-mqtt

I didn't do anything in Node Red so far, this will be second step after I updated the driver. (and I am also trying to figure out which BMS is which one, I cannot find anything delivered by serialbattery letting me distinguishing them)

[morres83]

@arrow1800 I have uploaded a MQTT version into a branch of my GitHub and will keep it running for tonight and tomorrow, to see if it is stable.

I did not yet take care in NodeRed about the data collection via MQTT (it is still based on the blue Victron blocks).

[arrow1800]

@arrow1800 I have uploaded a MQTT version into a branch of my GitHub and will keep it running for tonight and tomorrow, to see if it is stable.

I did not yet take care in NodeRed about the data collection via MQTT (it is still based on the blue Victron blocks).

nice! i've looked at your implementation. one question:

i see you are effectively still polling your mqtt bus. -> Lib.timeout_add(1000, self._update)

i was hoping that the python part could be more event driven, like: when mqtt receives a message it could be pushed directly to the dbus. In my own implementation this is a problem as well, didnt found a solution yet.

any suggestions maybe?

[morres83]

That's not completely true, I am not polling the MQTT bus. This is how often the dbus gets updated. The MQTT topics are subscribed and received by on_message I guess you can also trigger an dbus update there, if you like.

I am not sure what is the minimum update rate until Victron thinks the connection is dead, but for me I won't change that (at least for now). I think 1s is reasonable. How often you provide data to the MQTT you'll have to decide in NodeRed. If you want to go completely event-driven, MQTT message could arrive very fast I found (multiple messages in one second), so I limit to 1s both in NodeRed and in this script. Multiplus anyhow cannot react that fast.

[arrow1800]

maybe, but it minimizes the added value of mqtt if the dbus is still updated at a fixed interval. i would rather remove that interval from python and determine my update rate from node-red itself only.

not found a working version yet though, when i remove the fixed interval from the dbus code, the dbus doesnt get updated anymore.

[morres83]

I know how to do it and will update and upload it when I have time. In terms of traffic it will not spare anything, I am sure, unless to restrict the data rate in NodeRed to more than 1s. For me there is always flickering a value, leading to a new MQTT message.

[morres83]

I updated my repo for event-based + timeout when no message arrives within one minute (precautions have to be taken in NodeRed)

[arrow1800]

thank you for your help and inspiration! i've refactored the code a lot to make it look a bit prettier. let me know what you think of it