data missing in app/web while using mqtt

[harrhel]

Hi,

recently I started to use the mqtt feature; I observed afterwards that the data during the time of my experiments is not available in the enervu app/web - a zero line in that period.

I assume that using the mqtt integration the data is no longer transmitted to the outside (or maybe not even stored on the device anymore?).

Ideally I would be interested in having both - the data in app/web plus some own integration I wonder if this seems possible ..

and thanks for the work already done

[gluap]

@harhel Oh, this didn't happen on my side while testing. I see glitches in the app (losing connection frequently) but the app was crashing a lot on my phone already before I wrote the python wrapper. History data on the app works on my side while running mqtt. I never saw related issues with the webapp. Since i'm using the same interface as the smartphone app this definitely shouldn't interfere with the enervu web page.

Are you using the iPhone or Android app? Are we talking about the same web page (enervu.lg-ess.com)?

If you need the data from the web page I suggest to stop using the mqtt wrapper until this is resolved. My feeling is that the polling interval of 10 seconds may be too tight or that I might be using new connections where the app keeps its connection alive. Ill look into both and report back.

Were you running several copies of the mqtt wrapper at once? While developing on one day I had many scripts hammering the box without delay (severaltimes as fast as the mqtt wrapper) and on that day saw history data on the app (but not the web interface) was off. I had switched the ess on and off several times on the same day however so in the end I couldn't be sure what caused the issue and I was not able to reproduce it after.

[thetimmyventura]

hi!

I observed similar issues. When the python script was running, there was data missing in the webapp as well as in the app. Nevertheless the outage values between web page and app differ slightly.

Right after stopping the script, the data was updated, which resulted in high peaks (up to 25kw/p). So, I guess LG is updating the statistics to the absolute values, when connection is back.

[gluap]

@thetimmyventura @harrhel which model/firmware version of ESS are you two using?

I'm using the following version: ("system information" in the mobile app, best not post the registration number as it is the default password for the admin mode according to documentation) Model: D0005KD1N111 firmware version info: PMS SW Version: 10.05.6053 / 22.02.2019 R1340 PCS SW Version: LG P1 02.00.02.00 R1753 BMS SW Version: LGBMS.01.01.01.00

If you want to try you could experiment with setting a longer polling interval - I just uploaded a new version of essmqtt that accepts an extra parameter --interval_seconds -- default is 10 seconds but I suggest setting it to 60 seconds or so and checking whether that fixes the issue for you.

[thetimmyventura]

I have exactly the same model and version like you.

gluap notifications@github.com schrieb am So., 26. Apr. 2020, 13:17:

@thetimmyventura https://github.com/thetimmyventura @harrhel https://github.com/harrhel which model/firmware version of ESS are you two using?

I'm using the following version: ("system information" in the mobile app, best not post the registration number as it is the default password for the admin mode according to documentation) Model: D0005KD1N111 firmware version info: PMS SW Version: 10.05.6053 / 22.02.2019 R1340 PCS SW Version: LG P1 02.00.02.00 R1753 BMS SW Version: LGBMS.01.01.01.00

If you want to try you could experiment with setting a longer polling interval - I just uploaded a new version of essmqtt that accepts an extra parameter --interval_seconds -- default is 10 seconds but I suggest setting it to 60 seconds or so and checking whether that fixes the issue for you.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/gluap/pyess/issues/2#issuecomment-619532233, or unsubscribe https://github.com/notifications/unsubscribe-auth/APJ2XNY3A6JV6GJL3BHJKQDROQJ6LANCNFSM4MRCZQMQ .

[gluap]

@thetimmyventura One more thing: Is your ESS connected via WIFI or via network cable?

I agree with your suggestion that it seems that the website is not receiving data for some time and as a result then calculates a very high power when the daily charge values are finally uploaded. This seems to point to the ESS web server being too busy serving essmqtt and not uploading to enervu because of it -- either because the network connection has issues or because the polling frequency is too high. The 10 seconds are more or less what I saw the app do so I considered it should be safe.

To test that assumption as I said I just uploaded a new version of essmqtt that accepts an extra parameter --interval_seconds -- default is 10 seconds but I suggest setting it to 60 seconds (--interval_seconds 60) or so and checking whether that fixes the issue for you.

Furthermore I realized essmqtt shouldn't be used at the same time as the graphite integration and I added some info on that on the readme. At most one instance of essmqtt should run at any one time. If several clients run they will fight for the one login token available from the box, long-term I will merge them in one script that only fetches data once and can upload to both graphite and mqtt.

[harrhel]

@gluap

my model is D008KE1N211 PMS SW 10.05.7065 / 07.04.2020 R756 PCS SW LG P2 02.00.01.00 R49 1.45.3 BMS SW BMS 01.07.00.01 / DCDC 7K.4.8

I am using the Android App

the device is connected via cable to a switch that connects to the internet router.

I'll try with some different intervalls

[harrhel]

I tried now with 45, 54 and 59 seconds; still observing the issue of no data in the web - hoping this recovers

the intervals conflict a bit at least with the app in parallel

[gluap]

Conflicts with the app are unavoidable as far as I can tell because the ess is only expecting one user. I would bet that the enervu app running on several phones at the same time will also cause interference between the two phones.

I can now reproduce the issue with reports on the enervu web page. As far as I can tell the cumulative values on the web are correct at the end of the day, but the realtime view is behaving strangely. Now that I'm able to reproduce it I hope I can find a cure.

[thetimmyventura]

I do not see conflicts running the app on several phones at the same time.

So, I think there must be a difference between running the script or running two phones.

Am Mi., 29. Apr. 2020 um 10:03 Uhr schrieb gluap notifications@github.com:

Conflicts with the app are unavoidable as far as I can tell because the ess is only expecting one user. I would bet that the enervu app running on several phones at the same time will also cause interference between the two phones.

I can now reproduce the issue with reports on the enervu web page. As far as I can tell the cumulative values on the web are correct at the end of the day, but the realtime view is behaving strangely. Now that I'm able to reproduce it I hope I can find a cure.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/gluap/pyess/issues/2#issuecomment-621051272, or unsubscribe https://github.com/notifications/unsubscribe-auth/APJ2XNZS65UFLOZGRJJXCZLRO7NLZANCNFSM4MRCZQMQ .

[cbartetzko]

I've seen the same issues with instable App-Access and strange Results from Enervu-Website. So I've decided not to use anymore App+Web from LG.

I'm currently working on an independant docker solution with Telegraf/InfluxDB/Grafana (based on pyess-esscli) running on my Synology NAS (could also run on Raspi), which exports all current datapoints into an InfluxDB via cronjob every 10 seconds. Additionally it will export all historic graph data every 15 minutes into the InfluxDB.

Dashboards/Graphics can be easily done in Grafana (accessing InfluxDB) MQTT can be exported from InfluxDB via Telegraf (not tested yet)

[gluap]

@cbartetzko @harrhel @thetimmyventura I'm quite sure I found the issue. When connecting, the phone sends the current time in UTC to the ess to update its internal clock (don't ask me why ess doesn't just connect to a time server), pyess mimics this behaviour (because if I can avoid it I want to not have to use the app).

The machine I was developing on initially was using GMT for its time zone, formatting local time yielded the same time the phone was sending so no problems when I'm using that machine. However when running pyess on a non-utc system (for instance Berlin time), the time is two hours ahead of GMT. Now the following happens:

1. Smartphone app connects, sets time in UTC
2. Pyess connects, sets time in UTC+2
   • Now to ESS the sessionid of the phone seems two hours old and is timed out
3. Smartphone app reconnects sets time in UTC
   • Now the pyess sessionid is in the future and thus invalid
4. Ess reconnects
   • goto 3.

In my first tests with the now-updated version 0.1.2 does not interfere with the app any more. And I think it is plausible that it also shouldn't interfere with the web service.

[cbartetzko]

Wonderful. I'm just a bit confused with the release numbers. In your last comment you announced 0.1.2 Then 1.1.2 was launched Attached to 1.1.2 are files for 0.1.2 and 1.1.2 Which ones are now to be used ?

[gluap]

@cbartetzko thank you for pointing it out, I made a typo when creating the tag on GitHub, it was meant to be numbered 0.1.2 all all along. I fixed it by removing the misnumbered release tag -- the tag name was the only location where I mistyped the version number.

If you installed the version attached to the misnamed 1.1.2 no harm is done, it will even continue to be upgradable from pypi since it thinks knows its correct version number 0.1.2.

[thetimmyventura]

I tested it now for a few hours. Works now like a charm! Thx for your quick response and bugfixing.

gluap notifications@github.com schrieb am Do., 30. Apr. 2020, 08:47:

@cbartetzko https://github.com/cbartetzko thank you for pointing it out, I made a typo when creating the tag on GitHub, it was meant to be numbered 0.1.2 all all along. I fixed it by removing the misnumbered release tag -- the tag name was the only location where I mistyped the version number.

If you installed the version attached to the misnamed 1.1.2 no harm is done, it will even continue to be upgradable from pypi since it thinks knows its correct version number 0.1.2.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/gluap/pyess/issues/2#issuecomment-621646006, or unsubscribe https://github.com/notifications/unsubscribe-auth/APJ2XN7LBGXCGECVTXRCGNTRPENGTANCNFSM4MRCZQMQ .

[harrhel]

great - thanks for resolving

:+1: