Minute entity keeps dropping

[rhaymg]

The Minute Entities frequently are "not available" and the only way to restore them is to go into the integration screen and select "reload" from the Emporia Vue. I've got several automations tied to the minute entities and they obviously don't work when they aren't available. Thanks.

[armdan]

same here, I wish we could flash the ESP chip and put tasmota or ESPHome in it

[JBoyKey]

Same here, have 6 - Min updates and only the 1st one total power updates, Joe.

[magico13]

For people with this issue, are there any messages in the logs for this integration? I unfortunately can't really reproduce this locally, it's working solidly for me.

[JBoyKey]

Wow, cleared a Growatt error in the yaml.cfg and it works now, thanks

From: Michael Marvin @.> Sent: Wednesday, August 25, 2021 4:00 PM To: magico13/ha-emporia-vue @.> Cc: JBoyKey @.>; Comment @.> Subject: Re: [magico13/ha-emporia-vue] Minute entity keeps dropping (#68)

For people with this issue, are there any messages in the logs for this integration? I unfortunately can't really reproduce this locally, it's working solidly for me.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/magico13/ha-emporia-vue/issues/68#issuecomment-905869896 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AVCJZ73Y46NKPSSWCIGSTZDT6VKVBANCNFSM5CSLTDBA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email . https://github.com/notifications/beacon/AVCJZ73SDSEVI4IHNMT3LTDT6VKVBA5CNFSM5CSLTDBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOGX7HUSA.gif

[rhaymg]

For people with this issue, are there any messages in the logs for this integration? I unfortunately can't really reproduce this locally, it's working solidly for me.

Where are the logs I need to check and what should I be looking for? This isn't a log but a screen shot of an entity that drops.

[magico13]

The logs are available at https://{your_HA_url}:8123/config/logs. You'd be looking for errors sort of like this, please copy and paste them fully (you can strip out any info that you are worried about being specific to you, like customer/device ids if present). If you see TypeError: unsupported operand type(s) for *: 'int' and 'NoneType' then make sure you update to v0.6.5 which I just released to fix that.

[rhaymg]

I'll update the integration and see how things work. But below are two error that show in the logs.

Here is the first error in the logs for the Emporia Vue:

This error originated from a custom integration.

Logger: custom_components.emporia_vue Source: custom_components/emporia_vue/init.py:152 Integration: Emporia Vue (documentation) First occurred: 1:04:19 AM (1 occurrences) Last logged: 1:04:19 AM

Unexpected error fetching sensor data: unsupported operand type(s) for +=: 'float' and 'NoneType' Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 187, in _async_refresh self.data = await self._async_update_data() File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 147, in _async_update_data return await self.update_method() File "/config/custom_components/emporia_vue/init.py", line 152, in async_update_day_sensors last_day_data[day_id]["usage"] += data[ TypeError: unsupported operand type(s) for +=: 'float' and 'NoneType'

The second error that shows up:

Logger: homeassistant.components.sensor Source: custom_components/emporia_vue/sensor.py:150 Integration: Sensor (documentation, issues) First occurred: 1:03:19 AM (2 occurrences) Last logged: 1:03:19 AM

Error adding entities for domain sensor with platform emporia_vue Error while setting up emporia_vue platform for sensor Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 382, in async_add_entities await asyncio.gather(tasks) File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 587, in _async_add_entity await entity.add_to_platform_finish() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 711, in add_to_platform_finish self.async_write_ha_state() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 464, in async_write_ha_state self._async_write_ha_state() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 498, in _async_write_ha_state state = self._stringify_state() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 470, in _stringify_state state = self.state File "/config/custom_components/emporia_vue/sensor.py", line 91, in state return self.scale_usage(usage) File "/config/custom_components/emporia_vue/sensor.py", line 150, in scale_usage usage = round(60 1000 usage) # convert from kwh to w rate TypeError: unsupported operand type(s) for : 'int' and 'NoneType'

[skynet01]

Happens to me as well, i think its related to #33 , i have set up an automation to reload integration that ussually fixes it. I posted how in that thread.

[mikedm139]

I also have repeated dropouts. I have an automation to reload the integration but it doesn't seem to work reliably. A full reboot of HA is usually required to bring it back. It seems to be an issue with API communication although at present (after a reboot) the API error is in the logs but the integration is working. Here is the error I keep seeing in the logs when the integration fails:

This error originated from a custom integration.

Logger: custom_components.emporia_vue
Source: custom_components/emporia_vue/__init__.py:263
Integration: Emporia Vue (documentation)
First occurred: 2:53:19 PM (1 occurrences)
Last logged: 2:53:19 PM

Error communicating with Emporia API: HTTPSConnectionPool(host='api.emporiaenergy.com', port=443): Max retries exceeded with url: /AppAPI?apiMethod=getDeviceListUsages&deviceGids={REDACTED}&instant=2021-11-05T20:53:14.003921Z&scale=1MIN&energyUnit=KilowattHours (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x153c440505b0>: Failed to establish a new connection: [Errno -3] Try again'))

I'll try to update with any additional logs when the integration next fails.

[skynet01]

I think it depends on the day, the past few days I had my script auto reload it about 7 times, usually its about once a day. I use NodeRed though, if anyone wants it here is the export of the flow, they work pretty reliably for me.

[{"id":"8a93ebd45796b533","type":"server-state-changed","z":"1169dafb.347525","name":"Emporio channel is down","server":"8a745406.799e98","version":3,"exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityidfilter":"sensor.power_av_closet_6","entityidfiltertype":"exact","outputinitially":true,"state_type":"str","haltifstate":"0","halt_if_type":"num","halt_if_compare":"gte","outputs":2,"output_only_on_state_change":false,"for":"","forType":"num","forUnits":"seconds","ignorePrevStateNull":false,"ignorePrevStateUnknown":false,"ignorePrevStateUnavailable":false,"ignoreCurrentStateUnknown":false,"ignoreCurrentStateUnavailable":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"true","valueType":"bool"},{"property":"data","propertyType":"msg","value":"","valueType":"date"}],"x":250,"y":1360,"wires":[[],["9b1037d5faa2f4f8"]]},{"id":"00fcf99a23deb608","type":"api-call-service","z":"1169dafb.347525","name":"Reload Emporio Config","server":"8a745406.799e98","version":3,"debugenabled":false,"service_domain":"homeassistant","service":"reload_config_entry","entityId":"sensor.power_pelican_123","data":"","dataType":"json","mergecontext":"","mustacheAltTags":false,"outputProperties":[],"queue":"first","x":710,"y":1380,"wires":[["c7b66907f042d2b7"]]},{"id":"2237f15227046410","type":"inject","z":"1169dafb.347525","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":380,"y":1420,"wires":[["00fcf99a23deb608"]]},{"id":"715ab5db5fde4670","type":"api-current-state","z":"1169dafb.347525","name":"Check if backup","server":"8a745406.799e98","version":2,"outputs":2,"halt_if":"0","halt_if_type":"num","halt_if_compare":"gte","entity_id":"sensor.power_av_closet_6","state_type":"num","blockInputOverrides":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"entityState"},{"property":"data","propertyType":"msg","value":"","valueType":"entity"}],"x":1140,"y":1320,"wires":[["c259a1816ef57d7f"],["00fcf99a23deb608"]]},{"id":"c259a1816ef57d7f","type":"telegrambot-notify","z":"1169dafb.347525","name":"Notify - fixed","bot":"3f37c1e5.83436e","chatId":"399776827","message":"✅ Emporia sensor recovered successfully!","parseMode":"","x":1370,"y":1340,"wires":[]},{"id":"c7b66907f042d2b7","type":"delay","z":"1169dafb.347525","name":"","pauseType":"delay","timeout":"20","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"x":940,"y":1360,"wires":[["715ab5db5fde4670"]]},{"id":"9b1037d5faa2f4f8","type":"delay","z":"1169dafb.347525","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"10","rateUnits":"minute","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"allowrate":false,"x":480,"y":1360,"wires":[["00fcf99a23deb608"]]},{"id":"8a745406.799e98","type":"server","name":"Skynet Home Assistant - Local","version":1,"addon":true,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true},{"id":"3f37c1e5.83436e","type":"telegrambot-config","botname":"PelicaTron_bot","usernames":"","chatIds":"","pollInterval":"300"}]

[mikedm139]

I'll try to update with any additional logs when the integration next fails.

After restarting HA yesterday, I found this morning that the integration had failed again. Based on the logs, it looks like it happened yesterday evening. Here are the log entries:

This error originated from a custom integration.

Logger: custom_components.emporia_vue
Source: custom_components/emporia_vue/__init__.py:263 
Integration: Emporia Vue (documentation) 
First occurred: November 5, 2021, 2:53:19 PM (3 occurrences) 
Last logged: November 5, 2021, 8:58:58 PM

Error communicating with Emporia API: HTTPSConnectionPool(host='api.emporiaenergy.com', port=443): Max retries exceeded with url: /AppAPI?apiMethod=getDeviceListUsages&deviceGids={REDACTED}&instant=2021-11-05T20:53:14.003921Z&scale=1MIN&energyUnit=KilowattHours (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x153c440505b0>: Failed to establish a new connection: [Errno -3] Try again'))
Error communicating with Emporia API: HTTPSConnectionPool(host='api.emporiaenergy.com', port=443): Max retries exceeded with url: /AppAPI?apiMethod=getDeviceListUsages&deviceGids={REDACTED}&instant=2021-11-05T21:08:22.002165Z&scale=1MIN&energyUnit=KilowattHours (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x153c4bd614f0>: Failed to establish a new connection: [Errno -3] Try again'))
Error communicating with Emporia API: HTTPSConnectionPool(host='api.emporiaenergy.com', port=443): Max retries exceeded with url: /AppAPI?apiMethod=getDeviceListUsages&deviceGids={REDACTED}&instant=2021-11-06T02:58:53.000691Z&scale=1MIN&energyUnit=KilowattHours (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x153c4088ff40>: Failed to establish a new connection: [Errno -3] Try again'))

This error originated from a custom integration.

Logger: custom_components.emporia_vue
Source: helpers/update_coordinator.py:219 
Integration: Emporia Vue (documentation) 
First occurred: November 5, 2021, 2:53:19 PM (3 occurrences) 
Last logged: November 5, 2021, 8:58:58 PM

Error fetching sensor data: Error communicating with Emporia API: HTTPSConnectionPool(host='api.emporiaenergy.com', port=443): Max retries exceeded with url: /AppAPI?apiMethod=getDeviceListUsages&deviceGids={REDACTED}&instant=2021-11-05T20:53:14.003921Z&scale=1MIN&energyUnit=KilowattHours (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x153c440505b0>: Failed to establish a new connection: [Errno -3] Try again'))
Error fetching sensor data: Error communicating with Emporia API: HTTPSConnectionPool(host='api.emporiaenergy.com', port=443): Max retries exceeded with url: /AppAPI?apiMethod=getDeviceListUsages&deviceGids={REDACTED}&instant=2021-11-05T21:08:22.002165Z&scale=1MIN&energyUnit=KilowattHours (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x153c4bd614f0>: Failed to establish a new connection: [Errno -3] Try again'))
Error fetching sensor data: Error communicating with Emporia API: HTTPSConnectionPool(host='api.emporiaenergy.com', port=443): Max retries exceeded with url: /AppAPI?apiMethod=getDeviceListUsages&deviceGids={REDACTED}&instant=2021-11-06T02:58:53.000691Z&scale=1MIN&energyUnit=KilowattHours (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x153c4088ff40>: Failed to establish a new connection: [Errno -3] Try again'))

This error originated from a custom integration.

Logger: custom_components.emporia_vue
Source: custom_components/emporia_vue/__init__.py:89 
Integration: Emporia Vue (documentation) 
First occurred: November 5, 2021, 8:59:06 PM (1 occurrences) 
Last logged: November 5, 2021, 8:59:06 PM

Could not authenticate with Emporia API

If there's other info that would be useful for troubleshooting, please let me know.

[Zyell]

I am encountering a failure across all 1 minute entities that is different than others on this thread but started this morning after an update to Home assistant. Reloading the integration actually causes all 1 minute entities to fail. The entity page shows all with a red circle and a state of "Restored". The only way to get the 1 minute entities back is to delete the integration and recreate it. Doing this works for a little bit and then I get the following errors showing up in the log:

This error originated from a custom integration.

Logger: homeassistant Source: custom_components/emporia_vue/sensor.py:90 Integration: Emporia Vue (documentation) First occurred: 1:41:12 PM (5 occurrences) Last logged: 1:45:12 PM

Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 134, in _handle_refresh_interval await self._async_refresh(log_failures=True, scheduled=True) File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 265, in _async_refresh update_callback() File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 325, in _handle_coordinator_update self.async_write_ha_state() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 486, in async_write_ha_state self._async_write_ha_state() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 519, in _async_write_ha_state state = self._stringify_state() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 492, in _stringify_state if (state := self.state) is None: File "/config/custom_components/emporia_vue/sensor.py", line 90, in state usage = self.coordinator.data[self._id]["usage"] KeyError: '42009-1-1MIN'

After this, I'm unable to restore the 1 minute entities without deleting the integration and creating it again. This only works temporarily though. If I just restart, there are no log entries at all, but the entities are in an "unavailable" state and never work again.

[Zyell]

In debugging this, I discovered that the API itself isn't reporting minute level entities. So I thought maybe the API has changed. However, I went to the Vue app on my phone and found that the second and minute data is not updating properly. I'm guessing something is wrong on Emporia's end...

[GitHubGoody]

@magico13 It seems there are a few of these issues related to 1min sensor unavailability.

My 1min sensors also continue to go Unavailable after HA restarts and then seemingly randomly while HA is running. As others have done, I have attempted to work around this with an automation set to trigger on the Unavailability state to call the homeassistant.reload_config_entry service on the same entity, but it seems to work only some of the time. Any ideas how to make 1min sensors more reliable? Would it help if I created another 1min sensor issue to consolidate all of them into one and request any data (e.g. logs, etc.) you might need to troubleshoot?

Thank you.

[skynet01]

Yep, that's exactly what I have to do as well. Maybe we can at least fix the issue when HA restarts as it happens 100% of the time for me. Maybe there is a check that checks if HA is still restarting and then waits some time before setting up the sensor?

Or maybe if the sensor is unavailable for X amount of time it gets reloaded automatically? I doubt everyone spends time setting up the automation like us that reloads this integration.

[magico13]

This one's a bit tough for me because I can't ever reproduce it. I constantly get 0's but the state doesn't go Unavailable for me, except every now and then for exactly one sensor. When I get 0's for the data the Emporia app has the same problem and lists the device as offline, despite it still sending data up that I can review later. The 0's are absolutely a result of the API having trouble and restarting when you see them will cause more server strain because startup requires a bunch of API calls. When mine go 0 they come back later with no intervention needed.

If the API doesn't report values for a particular sensor when the initial startup request is made then that one will fail to get set up and won't ever get data because it will never ask to be updated, since Home Assistant doesn't realize it should exist. I can and should change the startup to create the entities per the Get Devices call and not by parsing the Usage call like I do now, that would fix that issue in more cases.

What I really want to do but am not sure of how to do yet, is to load entities/sensors dynamically. So if something is missing from the initial call, or you add a new outlet, it will add the appropriate sensors automatically. I think that'd be the best long term solution but I need to dedicate a few days to trying to figure that out and I haven't had the time.

I'd like to condense all the issues into one issue but haven't taken the time to sort through all of the open issues and close out fixed ones and merge duplicates.

[skynet01]

I see... if you would like I can share my Emporio login details with you. I don't have any switches just 8 monitoring sensors and I also have solar.

[GitHubGoody]

@magico13 I'm happy to help "condense all the issues into one issue." If you're able to give me the necessary permission, I could also assist sorting and merging the remaining open issues.

For this one, if I started a new "1 Minute (1min) entity reliability troubleshooting" issue, what information and/or data would you like from people to assist your efforts?