Support configuration updates through MQTT /config/set

[mathieucarbou]

I know this PR will bring up some discussions, but I am still opening it.

It's been months that I am using OpenEVSE, always up to date with the latest changes on master and always appending this commit on top (so custom build) because I need this feature and it really frustrates me that it is not included in the default firmware.

HTTP API allows to update the gateway configuration, so this PR aims at bringing also this ability through MQTT. The idea is to allow a better integration with Home Assistant, Jeedom and others.

There are several use cases, and I know at least one person also using my branch with these changes to allow a config change through MQTT.

Some non exhaustive possible usages:

• expose some configuration settings with their corresponding (and ore dynamic) widgets (sliders, etc) and not just labels

• ability to update the divert settings based on Tempo Days through some automation (here in France some electricity providers have plans where some random picked days are so expensive that we want to switch from a no waste strategy to a no import strategy).

• compatibility with solar routers: someone having a solar router might be interested in using a no waste strategy with a high ratio to prioritize the solar router (heating water) with the few solar production of the morning before it reaches an interesting threshold where openevse can be triggered), and then after, move the ratio closer to 0.3 or 0.2 to prioritize EV charging.

Here is my gist with thisHA / MQTT integration: https://gist.github.com/mathieucarbou/92a3d5e0dc38d6b68aa1bdaf153a80c5

[jeremypoulter]

I am really not convinced that config is best managed by MQTT, it is really not the best tool for the job, but people really don't want to use HTTP/OCPP for various reasons so I guess we will allow it.

Apart from that you have only implemented the set side of things. For 'propper' implementation you also need to event config changes, this is the only way you can get the current config (via MQTT) and validate that the config changes you have just set have been applied.

[mathieucarbou]

For 'propper' implementation you also need to event config changes, this is the only way you can get the current config (via MQTT) and validate that the config changes you have just set have been applied.

Hi @jeremypoulter , I think this part is already done as part of my previous refactoring of the config layer in PR #634.

All the places calling config_deserialize(doc) and config_commit() (i.e. web server, divert code, mqtt, etc) are all calling these same functions to update and save the config the same way. The increment and event sending of the new version is done in this function.

[jeremypoulter]

Oh you are correct, I forgot about that, I think this is ok then.

[Mathopenevse]

@jeremypoulter hi, i don't understand why you say that MQTT isn't a good tool compare HTTP? Could you give me a example? Personnaly i find mqtt config simpler than HTML protocol. Moreover, with jeedom, i can integrate config directly on the plugin without build script.

[jeremypoulter]

For broadcast scenarios where the device is just sending out data MQTT is great, but for request/response scenarios, like setting config, you end up doing a lot of additional work (and I am talking about network/EVSE here rather than any connection application).

In HTTP you send a request and get a response on the same socket, with the success/failure of the request. On MQTT you send a request into the ether, hope something is listening, if there is you might be lucky and it will send back a response on an unrelated resource (from a protocol POV) at some point in the future. If there is an error you probably are never going to get to know about it. To get the initial state you have to wait for something to change and/or depend on the broker to have an up-to-date copy of the state.

Simply put MQTT is just not designed for it, that is not to say it can't do it, it is just a lot more complicated than HTTP. Likewise HTTP can be used to get 'broadcast' status updates from the device, but is just a lot more work.

[mathieucarbou]

@jeremypoulter : I agree but I do think both protocols avec their pros and cons depending of the system using it. 😃

You have listed the pros of a sync http connection and the cons of a mqtt connection, but we should also list the cons of http and pros of mqtt in consideration of a home assistant integration or other system 😃

Here are some I see:

• mqtt is a lightweight protocol compared to triggering http calls, coupled eventually with authentication.

• It can also overload a system or potentially disrupt it because the system under load is directly targeted by clients, thus impacted by their number and request load. This is not applicable to EVSE but this is nevertheless a fact.

• MQTT is using a third party to decouple the direct connection between a system and clients. Yes, it breaks the request/response scheme, but this constraint allows a more flexible integration because the system does not have to think or pay attention to the number of directly connected clients (since there is just the mqtt broker), or the potential impact of a heavy http request / response (memory-wise). Also, the system does not have to answer, which brings an additional cost.

• In the case of home assistant for example (or any system where UI components can be bound to mqtt topics) where components can trigger mqtt calls and refresh themselves based on mqtt events, this is totally OK because a failure of doing an action will result of the components not being updated to the new value. If were were in the normal webapp, yes we would need a request/response to know the reason of the failure, but in the case of an integrated home assistant, I don't think it matters because the system status would be taken from another topic, eventually linked to the UI components (I am thinking for exemple if someone pushed a pilot value greater than the max soft limit). This kind of things work great with mqtt: I have my max soft limit set to 20A, if I set my pilot limit up to 32A, the UI will automatically refresh it to 20A, thanks to the components being bound to mqtt topics updated by the gateway.

• In HA, it is definitely easier to integrated a UI with MQTT than HTTP.

I do agree that from the gateway GUI perspective, the UI definitely benefit of http / websocket because the client-side (system) in javascript is more suited to use http calls and target directly its server.

But from an integration perspective, I strongly think that an HTTP API is not suited to be used everywhere and there are more efficient protocols, mqtt being one of them. The integration I did of OpenEVSE for HA, which is highly dynamic (the UI updates itself based on what is active or not) is more proof that we can achieve a powerful integration in HA without the need of a complex plugin doing HTTP calls ;-)

[jeremypoulter]

@mathieucarbou I am not going to block anyone trying to use MQTT for control (or contributions to support this), and indeed Home Assistant etc may hide a lot of the workarounds you have to do to get it working but my personal recommendation will be to use HTTP for any control use cases.

[chris1howell]

HTTP in the case of OpenEVSE control becomes complicated once you transverse a NAT firewall. MQTT does not have this issue.

It is simple to use a MQTT server on the internet or cloud service. All the major cloud services have standardized on MQTT for IoT applications likely for this reason.

To use HTTP for the same would require port forwarding (not secure) or a VPN.

Sure HTTP may be better suited if both OpenEVSE and the client are on the same network. This is likely only the case for home users. Commercial, fleet, vehicle OEM, etc. This will never be the case.

On Sat, Aug 19, 2023, 5:27 PM Jeremy Poulter @.***> wrote:

@mathieucarbou https://github.com/mathieucarbou I am not going to block anyone trying to use MQTT for control (or contributions to support this), and indeed Home Assistant etc may hide a lot of the workarounds you have to do to get it working but my personal recommendation will be to use HTTP for any control use cases.

— Reply to this email directly, view it on GitHub https://github.com/OpenEVSE/openevse_esp32_firmware/pull/714#issuecomment-1685115355, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAN5QHYQDERTOD6ICWLRBQ3XWEVUPANCNFSM6AAAAAA3UPMP6Y . You are receiving this because your review was requested.Message ID: @.***>

[jeremypoulter]

For fleet management you would use OCPP no?

[chris1howell]

Depends, OCPP would be recommended for an EV only system built from scratch. An existing cloud based system with a mix of data sources would likely already be built with IoT cloud tools and MQTT would better mesh with other non-EV data sources.

On Sun, Aug 20, 2023, 9:11 AM Jeremy Poulter @.***> wrote:

For fleet management you would use OCPP no?

— Reply to this email directly, view it on GitHub https://github.com/OpenEVSE/openevse_esp32_firmware/pull/714#issuecomment-1685282367, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAN5QH52PGS5OW4II4BYFETXWIEIPANCNFSM6AAAAAA3UPMP6Y . You are receiving this because your review was requested.Message ID: @.***>