serkri / SmartEVSE-3

Smart Electric Vehicle Charging Station (EVSE)
MIT License
68 stars 28 forks source link

add MQTT client #159

Closed arpiecodes closed 1 year ago

arpiecodes commented 1 year ago

This pull is my second attempt to add MQTT support (see https://github.com/serkri/SmartEVSE-3/pull/69 for first attempt). It was kept to the minimum to enable MQTT to function properly this time, and only contains minor refactors where it made sense to do so.

The pull request seems to contain a lot of added code, but this is due to the addition of a lightweight MQTT client library (I simply included the whole Git repo contents as it was not possible to directly use it through pio dependencies). The standard Arduino MQTTPubSubClient library would require a Ethernet/WiFiClient, and it seems we are not using that in this project with mixed Arduino/ESP IDF framework code. If someone figures out a better way to implement it properly using standard libraries, contributions to the code are very welcome. I also edited the library to support username/password authentication.

Code has been tested on my SmartEVSE for some time already albeit in another form. "It works on my machine" applies, YMMV, some testing would probably be wise.

The MQTT support can easily be disabled again by defining MQTT_DISABLED.

PS: Swagger will need an update after this is merged to document all the new MQTT setting params. I am willing to take a stab at that when the code is accepted and merged.

dingo35 commented 1 year ago

@arpiecodes First of all, love your coding style, looks like a really clean job!

I understand your struggle with the libraries, been there, done that. I understand you didn't want to include the library, but you had no choice. What I did not like is that you had to add set_credentials, so now we have an unmaintained, modified library. Still, I have no problem to do that as long as it is our last resort.

I did not understand the ethernet/wifi client remark (since we use that wifi.h library), so I fiddled around a bit with the MQTTPubSubClient library; in the zip file you find a .diff that shows how I think the MQTTPubSubClient lib could be used; the code compiles but is otherwise UNTESTED since I currently have no MQTT server running; it will not work since the callback code has to be moved and I didn't want to make the diff unreadable.

Please let me know what you think, would moving towards that lib be feasible? Pro's, cons?

mqtt_pubsub.zip

arpiecodes commented 1 year ago

Nice! I'll check this out and confirm whether it works in a 'MQTT production set-up'. It would definitely make things a lot more 'clean' and we incur less technical debt.

I think you found a more generic version of the same library that supports more platforms. The one I found was the original Arduino one. Any ways; it looks good. Let me report back soon.

arpiecodes commented 1 year ago

Also, good to note; this pull request is missing the 'EVState' specific Publish and Set topics since these variables are introduced in https://github.com/serkri/SmartEVSE-3/pull/160 respectively.

arpiecodes commented 1 year ago

So I applied your diff and refactored all the code to implement the MQTTPubSubClient library, but it seems to cause a bootloop. Flashing back the other version seems to fix it again. Any ideas?

PS: Those NVS messages are harmless - I also see them on the original firmware. But they did reveal a bug with the DelayedStartTime/StopTime feature. The keys used for those preferences being too long for NVS to save/read from/to flash. :)

bootloop

dingo35 commented 1 year ago

Ok thanks for the bug discovery!!

Could you upload your code (just zip total SmartEVSE dir or a diff to current master) so I can take a look?

dingo35 commented 1 year ago

This might explain your problem:

https://github.com/hideakitai/MQTTPubSubClient/issues/10

dingo35 commented 1 year ago

... you might also try this library, which is derived from the one above, so it shouldn't need much changes in your code: https://github.com/khoih-prog/MQTTPubSubClient_Generic

It is an archived library, but this khoih guy makes good stuff.....

arpiecodes commented 1 year ago

@dingo35 I found another library (https://github.com/256dpi/arduino-mqtt) that seems to be actively supported with a healthy number of stars. It's also easily included into the project with pio. So I went with that. It seems to work nice now!

Also performed some minor refactors, removed some stuff I was not sure of to include as of yet. Also tested some edge cases, all seems to go well. As far as I'm concerned, it's good to go.

dingo35 commented 1 year ago

@arpiecodes Ok I'm putting up a test environment; first mistake was to use the public test.mosquitto.org server, hundreds of devices appeared in my HA; so I setup a local mqtt server, it receives topics from SmartEVSE, I can subscribe to topics in HA, but how do I get the autodiscovery activated? It must be some setting in the mqtt server, but I cannot find it....

EDIT: Or are we missing discovery-topic code in SmartEVSE? Because now I am getting my Tasmota devices discovered on my LAN.....

arpiecodes commented 1 year ago

@arpiecodes Ok I'm putting up a test environment; first mistake was to use the public test.mosquitto.org server, hundreds of devices appeared in my HA; so I setup a local mqtt server, it receives topics from SmartEVSE, I can subscribe to topics in HA, but how do I get the autodiscovery activated? It must be some setting in the mqtt server, but I cannot find it....

EDIT: Or are we missing discovery-topic code in SmartEVSE? Because now I am getting my Tasmota devices discovered on my LAN.....

Yes that's correct, auto-discovery is not built-in yet (still on the to-do list). I will edit and push my current YAML for Home Assistant so you can try it out without having to set up everything by hand.

dingo35 commented 1 year ago

Ok great! Do you mind if I change ifndef MQTT_DISABLE to ifdef MQTT ? I find double negatives hard to read.... IMHO MQTT should be enabled by default, because it adds responsiveness to HA while not costing much in resources....

arpiecodes commented 1 year ago

I have some more changes coming up actually, which I will push later today. I think it's worth the wait before we merge.

dingo35 commented 1 year ago

O ok, Im currently doing some minor layout / editing changes directly in github, that Ok or you want me to await your push?

arpiecodes commented 1 year ago

O ok, Im currently doing some minor layout / editing changes directly in github, that Ok or you want me to await your push?

If you could add them as review comments in-line, I will be sure to handle them as I push later. 👍

dingo35 commented 1 year ago

Ok will do!

dingo35 commented 1 year ago

@arpiecodes Just pushed the discovery-code for the sensors ....

dingo35 commented 1 year ago

@arpiecodes any comment on PR #172 ? I'm considering committing it, but also to remove EVChargeCurrent topic since it is a ducplicate of ChargeCurrent....

What do you think?