Energy Data 1D, 1MIN, 1MO all read 0 since this week's update

[DrMik-1]

I'm relatively new to HA so apologies if I don't have all the information.

Since updating the integration this week. All energy data reads 0. Integration still works with no errors or warnings. I can also control all emporia switches.

I'm running a VM with the following: Emporia v0.9.2 Bump PyEmVue to v0.18.3 Core 2024.2.5 Supervisor 2024.02.1 Operating System 12.0 Frontend 20240207.1

2024-03-02 21:42:49.340 INFO (MainThread) [custom_components.emporia_vue] Got None usage for device 229784 channel 1,2,3 scale 1MIN and timestamp 2024-03-02T21:42:42-06:00. Instead using a value of 0 2024-03-02 21:42:49.340 INFO (MainThread) [custom_components.emporia_vue] Could not find usage info for device 229784 channel 1 2024-03-02 21:42:49.340 INFO (MainThread) [custom_components.emporia_vue] Could not find usage info for device 229784 channel 2 2024-03-02 21:42:49.340 INFO (MainThread) [custom_components.emporia_vue] Could not find usage info for device 229784 channel ................................................................ (ALL OTHER SENSORS WITH SAME MESSAGE) .................................................................... 2024-03-02 21:43:07.553 INFO (MainThread) [custom_components.emporia_vue] Integrating minute data into day sensors 2024-03-02 21:43:07.554 DEBUG (MainThread) [custom_components.emporia_vue] Finished fetching sensor data in 0.001 seconds (success: True) 2024-03-02 21:43:12.704 DEBUG (MainThread) [custom_components.emporia_vue.switch] Finished fetching switch data in 0.415 seconds (success: True)

[DabblerIOT]

I agree last 2 revs have issues...

I fell back to the rev I put the pull request in that resulted in 18.1 (core pyemvue project)... I put in bug report as well on that project today for 18.3, but didn't realize failure was broader than just balance readings.

Install file editor mod and change requirements line.. file is custom_components/emporia_vue/manifest.json

Change the version back to 18.2 or 18.1 if those worked better for you..

For me 18.1 was "ok", 18.2 and 18.3 not so much.

I fell back to the code I put in pull request to add retries until project tests their code better before releasing. (I know it works for me.)

"requirements": ["git+https://github.com/DabblerIOT/PyEmVue"],

You will have to restart after editing file to get new dependency to update.

Cheers.

[magico13]

Very odd that you're seeing it consistently not able to retrieve any info. Implies to me that it either didn't set up the devices correctly or that the usage response is not returning any data. There was a change to how devices were set up, previously it only looked at the first response from the usage request and now it builds it primarily from the device tree, so there's a chance there's a difference there.

Does the Emporia app also show 0s or missing devices when on the home page set to "min" and the time set to "now"? If it's missing there, it will likely be missing from Home Assistant. Since you mentioned it only being since the update that's not likely the be the issue but it's always a good check. A good number of reports of missing data I've seen have been the connection between the clamp and main device being flaky, usually because it got pinched or slightly pulled out when putting the cover back on the panel. That would usually be the case for individual channels, not generally all the data.

You could try deleting the integration, rebooting, then setting it up again. That would set up the devices fresh and the sensors should all reattach to the previous ones if the names haven't changed. That would avoid any mismatch of data from before the update and after.

You mention having switches, how is your hardware set up? I assume a single Vue2 with 16 individual circuit monitors and a handful of smart plugs, all nested under the one device?

You can try @DabblerIOT's suggestion, I'd be curious if you see any difference with that. The primary difference there is the number of retries and how long it waits between retries. If you're always getting no data back it might not make a difference, if you only get data back after many retries then it could help but there's something more serious going on since it's not normal to never get data back on the first call.

[csipu]

I have the same issue since the update (0.9.2). The HA does not get values, but 0. It seems the update broke something for me also. The Emporia web app shows the real value. I have removed the integration, reboot and make a fresh install. Same issue, no values.

2024-03-03 11:20:07.138 INFO (MainThread) [custom_components.emporia_vue] Found 2 Emporia devices with 2 total channels 2024-03-03 11:20:13.780 INFO (MainThread) [custom_components.emporia_vue] Got None usage for device 165911 channel 1,2,3 scale 1MIN and timestamp 2024-03-03T11:20:07-08:00. Instead using a value of 0 2024-03-03 11:20:13.780 INFO (MainThread) [custom_components.emporia_vue] Could not find usage info for device 281466 channel 1,2,3 2024-03-03 11:20:13.781 DEBUG (MainThread) [custom_components.emporia_vue] Finished fetching sensor data in 6.642 seconds (success: True) 2024-03-03 11:20:13.781 INFO (MainThread) [custom_components.emporia_vue] 1min Update data: {'165911-1,2,3-1MIN': {'device_gid': 165911, 'channel_num': '1,2,3', 'usage': 0, 'scale': '1MIN', 'info': <pyemvue.device.VueDevice object at 0xffff738b73e0>, 'reset': None, 'timestamp': datetime.datetime(2024, 3, 3, 11, 20, 7, tzinfo=tzfile('/usr/share/zoneinfo/America/Los_Angeles'))}, '281466-1,2,3-1MIN': {'device_gid': 281466, 'channel_num': '1,2,3', 'usage': 0, 'scale': '1MIN', 'info': <pyemvue.device.VueDevice object at 0xffff738b52b0>, 'reset': None, 'timestamp': datetime.datetime(2024, 3, 3, 11, 20, 7, tzinfo=tzfile('/usr/share/zoneinfo/America/Los_Angeles'))}} 2024-03-03 11:20:20.411 INFO (MainThread) [custom_components.emporia_vue] Got None usage for device 165911 channel 1,2,3 scale 1MON and timestamp 2024-03-03T11:20:13-08:00. Instead using a value of 0 2024-03-03 11:20:20.411 INFO (MainThread) [custom_components.emporia_vue] Could not find usage info for device 281466 channel 1,2,3 2024-03-03 11:20:20.411 DEBUG (MainThread) [custom_components.emporia_vue] Finished fetching sensor data in 6.630 seconds (success: True) 2024-03-03 11:20:20.412 INFO (MainThread) [custom_components.emporia_vue] 1mon Update data: {'165911-1,2,3-1MON': {'device_gid': 165911, 'channel_num': '1,2,3', 'usage': 0, 'scale': '1MON', 'info': <pyemvue.device.VueDevice object at 0xffff738b73e0>, 'reset': datetime.datetime(2024, 3, 1, 0, 0, tzinfo=tzfile('/usr/share/zoneinfo/America/Los_Angeles')), 'timestamp': datetime.datetime(2024, 3, 3, 11, 20, 13, tzinfo=tzfile('/usr/share/zoneinfo/America/Los_Angeles'))}, '281466-1,2,3-1MON': {'device_gid': 281466, 'channel_num': '1,2,3', 'usage': 0, 'scale': '1MON', 'info': <pyemvue.device.VueDevice object at 0xffff738b52b0>, 'reset': datetime.datetime(2024, 3, 1, 0, 0, tzinfo=tzfile('/usr/share/zoneinfo/America/Los_Angeles')), 'timestamp': datetime.datetime(2024, 3, 3, 11, 20, 13, tzinfo=tzfile('/usr/share/zoneinfo/America/Los_Angeles'))}}

[DrMik-1]

@magico13

Emproia App shows full data and trending. Not a single instance of loss in its data.

Tired deleting and re-integrating to HA but I am getting the same results.

I'm monitoring 2 homes. V1 with 8 and a V2 with 16. V1 has been offline on Emporia's app since October. I'm unable to troublehoot in person and suspect an internet issue. V1 hasn't caused any issues thus far with HA since its outage. Just shows offline. All smart plugs are nested under the V2 at my home. All smart plugs can be triggered by HA with no delay or error.

I'm going to roll it back soon and I'll let you know the results.

Appreciate the insight.

[DabblerIOT]

I think 18.1 and my test won't be happy with down devices.

So that may not be helpful.

On Sun, Mar 3, 2024 at 2:34 PM DrMik-1 @.***> wrote:

@magico13 https://github.com/magico13

Emproia App shows full data and trending. Not a single instance of loss in its data.

Tired deleting and re-integrating to HA but I am getting the same results.

I'm monitoring 2 homes. V1 with 8 and a V2 with 16. V1 has been offline on Emporia's app since October. I'm unable to troublehoot in person and suspect an internet issue. V1 hasn't caused any issues thus far with HA since its outage. Just shows offline. All smart plugs are nested under the V2 at my home. All smart plugs can be triggered by HA with no delay or error.

I'm going to roll it back soon and I'll let you know the results.

Appreciate the insight.

— Reply to this email directly, view it on GitHub https://github.com/magico13/ha-emporia-vue/issues/261#issuecomment-1975273725, or unsubscribe https://github.com/notifications/unsubscribe-auth/BGBTIU2OXTSHE3TXDPGWSQDYWN3LBAVCNFSM6AAAAABEDTXTMSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZVGI3TGNZSGU . You are receiving this because you were mentioned.Message ID: @.***>

[DrMik-1]

@magico13 @DabblerIOT .

I rolled it back to 18.1 using HACS. All devices are now reporting data.

Let me know if you want any additional info about my setup to help you troubleshoot this issue.

Thanks!

[magico13]

I rolled it back to 18.1 using HACS

Clarification question, did you roll back the version of the Emporia integration via HACS, and to which version? v0.8.1/v0.8.3? Or did you change the version of PyEmVue in use by the integration by modifying the manifest file?

The two separate Vues, one of which is offline, is definitely important information so thank you for sharing that. @csipu do you also have separate Vues or any offline devices? I may be able to replicate that setup, I have a Vue1 as a sub-device of my Vue2 and I can move that in the Emporia app and then unplug it to see what happens. Not sure what that will do to the longer term data but I suppose we'll find out.

[csipu]

I have 1 device: https://shop.emporiaenergy.com/collections/emporia-products/products/utility-connect

[magico13]

I have 1 device: https://shop.emporiaenergy.com/collections/emporia-products/products/utility-connect

Interesting, your account appears to say you have two devices on it, id 165911 and id 281466. The former is reporting None and the latter is not returning in the data at all. Still seems like it might be related to the two devices, one offline idea. I'm working on testing that situation now but running into issue with my development environment. I'll hopefully have something later today for you both to try.

[csipu]

I have an Emporia EV charger, it can be the 2nd device. Mostly it is turned off.

[csipu]

Just turned the EV charge on, disable/enable the service and now I get data! Probably the EV is the 1st device and the energy monitor would mapped to the EV charger in HA?

[csipu]

I tried to validate my concept, but the HA lost the Emporia integration as a service. I have removed the integration from the HACS and readded the Emporia. Now during the integration config I have added the user+pwd and I got an error. According to the logs the user is configured: Traceback (most recent call last): File "/config/custom_components/emporia_vue/config_flow.py", line 79, in async_step_user self._abort_if_unique_id_configured() File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1772, in _abort_if_unique_id_configured raise data_entry_flow.AbortFlow(error) homeassistant.data_entry_flow.AbortFlow: Flow aborted: already_configured

Is there a way to purge the previous user config to the Emporia?

[magico13]

Is there a way to purge the previous user config to the Emporia?

Generally a reboot will either make the configured one visible so you can remove it, or will complete the removal process. I recommend waiting a little bit, I'm pretty sure I found what the cause of this was and am about to push a fix.

[DabblerIOT]

@magico13 I was going to run my code with more retries as a test to see if no missing data with 10 retries... (lol)...

But I can switch to a pre release to help verify if you want some more testing.

[magico13]

I pushed something up to the master branch here that is working correctly on my dev environment and not working on my server, currently trying to figure out what is different between them. I believe the main source of the problem is that the offline device will always not report data and if we happen to find that first in the list then the retry code would stop looking and retry immediately. It would keep doing this every retry and never actually look at the data for the other device.

I am attempting to fix that by fully looking through the list, taking whatever good data we do find, but still retry if we find that data is missing. Each retry if we have good data for a device then we take it so by the end of the retries we should end up still having data for the online device and none for the offline. There's room for improvement there since we shouldn't retry for devices that we know are offline but that will require additional work, as a quick fix this seems like it should work.

If you go into HACS and choose "redownload" and then choose "master" for the version then that should pull the latest. Like I said though, it's working in my dev environment and doesn't seem to be behaving correctly in my actual server so I am not sure what's up with it.

Edit: I figured out my problem, I had previously pointed directly at a branch on github of PyEmVue so it wasn't actually downloading the new version from PyPi despite the code being different. After forcing it to install the new version of PyEmVue it started working correctly.

[CAHutchins]

Same here. I had to roll back to 0.8.3 to get it working again. I have two Vue 2 energy monitors with the integration showing 37 devices and 111 entities. Oddly, one of the base VUE002 devices has 12 copies sensor.main_123 works fine and sensor.main_5 thru sensor.main_16 just show unavailable.

[eastoncrafter]

I have a Gen 1 power monitor that recently stopped reporting only the minute entity for each monitored circuit. Reboots did nothing, but after downgrading from 0.8.3 to 0.8.1 in the manifest and rebooting everything was fixed. One thing to note was that the disabled entities were reported as "disabled by the integration" in home assistant, not reporting 0 values.

[DrMik-1]

Clarification question, did you roll back the version of the Emporia integration via HACS, and to which version? v0.8.1/v0.8.3? Or did you change the version of PyEmVue in use by the integration by modifying the manifest file?

v0.8.1 using HACS and selecting the version.

[magico13]

Can everyone who is seeing this please try installing the "master" version through HACS and report back? I believe that should fix it since it removes another remnant of the code that was throwing out the data if anything was missing, it should now fully use what data it retrieved instead of cutting out early.

[csipu]

I have update the integration and still 0 for me. I am new in this HA business, so I hope I did it correctly, just redownload the 0.9.2 w/o the beta flag and restart the system, but no luck. Yesterday I have restored the latest backup and it works.

[CAHutchins]

Can everyone who is seeing this please try installing the "master" version through HACS and report back?

I downloaded the 'Master' v7cec1a8 and it seems to be working fine. It seemed to take a long time to initialize but all the numbers are there and updating. Should I leave it on Master or switch to v0.9.2 now?

[magico13]

Leave it on master, v0.9.2 does not have this fix in it yet. I wanted to verify the issue was fixed before rolling that out to everyone. I will bump up to v0.9.3 with the fix likely later today.

The startup is definitely going to be slower in these situations because the retries are going to run and the total of the delays it uses is at least 30 seconds. I will look into making that smarter so it can ignore devices that have been offline for a while but from just the one API call in isolation it can't detect the difference between an offline device that's not likely to come back online and the API temporarily omitting data for a device. After startup it shouldn't be too noticeable, but it will add an additional 30 second delay to data being updated.

[DabblerIOT]

@magico13 Running test. All looks good right now. (Only 30 minutes run). :)

Do you have a text file of the json return with one meter online / one meter offline? Curious what it looks like.

Thanks again.

[magico13]

@DabblerIOT sure, here's the setup I was testing with yesterday. 7466 is a Vue1 usually nested under the 86116 Vue2 but for this I set it to standalone and then unplugged it.

I'll throw in a status response from around then too, I am already regularly calling that to make sure the outlets and chargers are in the correct state so I should be able to use that to determine which devices to exclude from the usage polling, just need to figure out the best way (or at least an acceptable way) to get that data to where it needs to be.

UsageResponse_MissingCoDevice.json StatusResponse_MissingCoDevice.json

[DabblerIOT]

2 retries hit status request? then use new isconnected data to omit retry / failure = true for that device? this way won't double api hits until we suspect a device is down

mainstream bms stuff normally does this on a minute basis.. but another request on a local bus is cheap and wont annoy anyone.

also my data and logs don't look very different polling every minute as opposed to every two minutes. Get same data back 5 out of 10 times.

Half as many chances to get bad responses?? lol i know sad.

thanks for sharing responses

On Mon, Mar 4, 2024 at 2:16 PM Mike Marvin @.***> wrote:

@DabblerIOT https://github.com/DabblerIOT sure, here's the setup I was testing with yesterday. 7466 is a Vue1 usually nested under the 86116 Vue2 but for this I set it to standalone and then unplugged it.

I'll throw in a status response from around then too, I am already regularly calling that to make sure the outlets and chargers are in the correct state so I should be able to use that to determine which devices to exclude from the usage polling, just need to figure out the best way (or at least an acceptable way) to get that data to where it needs to be.

UsageResponse_MissingCoDevice.json https://github.com/magico13/ha-emporia-vue/files/14486153/UsageResponse_MissingCoDevice.json StatusResponse_MissingCoDevice.json https://github.com/magico13/ha-emporia-vue/files/14486154/StatusResponse_MissingCoDevice.json

— Reply to this email directly, view it on GitHub https://github.com/magico13/ha-emporia-vue/issues/261#issuecomment-1977286590, or unsubscribe https://github.com/notifications/unsubscribe-auth/BGBTIU2NVVTLTLRMQCSCCKLYWTCBXAVCNFSM6AAAAABEDTXTMSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZXGI4DMNJZGA . You are receiving this because you were mentioned.Message ID: @.***>

[DabblerIOT]

Sorry that didn't make sense.. if you change to poll every 2 minutes (which is closer to guarantee you will have new data from api)...

every 2 minutes request status of devices poll only "online" devices with retry logic for api issues (think missing data should result in unknown with ha entity stuff)

You would be at.. 1440 request per day for minute data which would leave room for other requests to stay below the ask of 2000 or less a day from api maker.

This type of interval is fine for most things... 1 min vs 2 min not a big deal... if you are sorting out what circuit something is on, use the actual emporia app, and it updates faster somehow..

Or this may be odd thought, if we REALLY want new data each minute, request second data, and multiply correctly to get power. Hmmm... I just don't see a reason to request, what is duplicate data half the time at the 1 min setting.

Easy to mod local copy of ha code to play... :). Just thinking.

Cheers. Thanks again for hard work sorting this out. Hope most realize that reason for issues with 1 min data isn't this projects code, but unreliable api that we are reading, which isn't the norm in this type of thing.

[DabblerIOT]

Had issue with browser update killing my test code... updated it to plugs I added and on first request got bad data in.. Will rework my test code to keep stats on how bad api is with errors on requests like this... dead serious first request with 6 devices came back like this.

badresponsefirstattempttest.json

[DabblerIOT]

@magico13

Ran test doing 6 device get usages with counter... have data in javascript console

request:570 null:77

About 7 hours of run null count is any usage is null

Clicked clear instead of copy half asleep this morning. Will wait to get more responses with nulls and look and see what devices are null, if it is normally first, or any. From my earlier testing it is worse in the early am.

Data from 18.4 is clean (well at least both vue2 data is). Will look closer later.

So the retry and assembling valid data logic looks to be working very well.

[DabblerIOT]

Seen multiple with all plugs null and vue devices ok.

Counting plug nulls, and view nulls separately now, have not seen a vue error looking back through nulls for last couple hours.

errorallplugsnullvueok.json someplugsmnullvueok.json

If it turns out that the plugs are more problematic... I may ditch them and get another make of output. Really just using plugs now to turn off vampire loads (ifit treadmill doesn't turn off screen backlight), and stuff like that.

[CAHutchins]

Of course it seems that the Emporia web service is offline at the moment. No data being returned and unable to log into the web site. It's not just me, right?

[DabblerIOT]

From USA east coast..

api is still up (have test code that just updated web site up webapp up, app up

3:30 pm eastern time

time is UTC in json file

apiup.json

[asinla]

From USA, West Coast.

Tried the redownload Master and it worked! Seems to be fine at the moment.

Thanks for this.

[CAHutchins]

FWIW, it looks like the problem was XFinity DNS. Switched to OpenDNS and I can log in. Changed DNS on the WiFi and devices connected.

[DabblerIOT]

Think there will be another fix to not retry on known down devices in a better way in future.

@magico13 minute requests from my test code: request:237 plug:55 vue:0

API is worse in the morning for me, will let it run. Kind of crazy how many more errors on the plugs vs vue2 devices. 3 of the plugs that have null data on some requests are in same room as router and arn't having any ping fails or things like that, so I think connection is good.

So 23% of my 2 vue, 4 plug requests result in at least one plug with error response (returning null, when it's really not). api issue.

I am Comcast in the east, dns here working fine.. weird. Always forget that can cause all kind of issues if it is wonky.

my dns with Xfinity is 75.75.75.75 and 75.75.76.76 which seem to be fine here.

[DabblerIOT]

Vue2 still failing, had typo in my count logic. :(. Not with frequency of plugs. Test running overnight.

vue2badresponse.json

[DabblerIOT]

request:1141 plug:110 vue:4

just over 19 hours run.. plugs returned null 110 times, vue2 devices 4

6 devices polled, 2 vue2, 4 plugs

[multinet33]

Hello

can anyone share is home assistant automation that restart emporia vue intégration when 1min sensor goes unavailable ? Mine is not working

thanks

[DabblerIOT]

update it and pick the master repository.. with hacs. (Make sense?). there have been some problems with api giving bad data which is being worked through.

[CAHutchins]

can anyone share is home assistant automation that restart emporia vue intégration when 1min sensor goes unavailable ? Mine is not working

I don't have an automation, I just remember to re-start the Emporia integration a few minutes after every time I re-start Home Assistant. For some reason, the 1 minute sensors don't work directly after a re-start until I re-load the integration. It hasn't happened to me any other time.

This is may have been fixed in the new version which is still a work in progress. Under HACS Integrations, select 'Redownload' from the three dots menu in the top-right corner and select 'Master' at the bottom of the version drop-down list.

[DabblerIOT]

I am running on a vm image of home assistant with haos on a Synology 920+ nas, and I don't have any issues on startup.

Maybe I am on overpowered hardware compared to others?

[davezahn5656]

Same Issue HA reporting all O's Removed integration and reinstalled. Now it errors attempting to log the integration into Emporia

[mattlewis19]

Also - erroring out on attempting to login. (Tried changing DNS host since I'm on Comcast and that didn't change anything.) Looks like this broke when I updated the HA server/application earlier this month. Currently running HACS provided 9.3 tried rolling back to 8.1 and 8.3 without success. I've got a vue2 with 16 monitored circuits. Current HA is a HA VM on a VMWare hypervisor Core 2024.4.3 Supervisor 2024.04.0 Operating System 12.2 Frontend 20240404.2

Error: I'm new to HA and not a developer.