[Feature] Data in JSON format

[nygma2004]

First of all thank you for all this work, in works great on my EB3A. Had some issues compiling the project in Arduino IDE, but it is all fixed now.

I know this could be personal preference, but I was thinking if it could be configured to send the data in separate topic like now, or just have a single data topic where all the values are send in JSON format (like the device topic) - or both at the same time. Data arriving in the same topic makes certain processes easier (e.g. saving data in CSV or database).

[gerab]

It would be good as an option. As an active user since like two days ago 👴, I'm terrified with only the idea of parsing JSON data array with Flux for Grafana. My Xeon is already not happy with data transformations

[mariolukas]

I think a webserver endpoint on the esp like

http://<ip>/josn

would be another interesting option. The disadvantage of this option is that you need some kind of polling mechanism on your consumer endpoint to get the data.

[nygma2004]

@gerab :) I was also planning to save data in Infux for testing, and having the data in JSON would make it easier to save all the fields together in the same record.

[gerab]

@nygma2004 it's the question of whether the decreased number of records to read will have benefits over JSON parsing. I'm interested in research. However, the most significant issue at the moment is with the transformation. The heaviest query is calculating time on battery, where I have to union tables with inputs summary, and outputs and then do some extra math on an extensive set. The other CPU-hungry task is to replace the substring with the cell number and then do the final regexp. Except those two, just regular data, even with a separate record, is impressively quick. I would be more concerned about moving that logic to the esp32. The calculation on a single parameter set will come at almost no cost.

[mariolukas]

I had a similar discussion in another project. We added a config parameter where the user was able to select the influx db format.

[nygma2004]

I have a server running Node-Red, so I usually all the data translation in Node-Red, therefore for me it is never a complicated task. I never ingest data to Influx directly from MQTT. I store of lot of sensor data, I was thinking adding a tag to identify the device like device = "Bluetti-EB3A" and next each value from MQTT goes into a different field. And I can also easily set up a filter in Node-Red to omit new record if there are no changes (e.g. all outputs are off and battery level is not changed). Btw, I heard that EB3A has overheating issues, it would be great if we could get temperature data as well, or maybe a flag when the unit detects overheating....