Open peterus opened 2 years ago
bug also valid for version 1.0.0
Yes, that the reason this issue is not marked fixed!
I also have the same issue on 2022.6.3 using v1.0.0:
2022-06-07 09:30:34 ERROR (MainThread) [homeassistant.components.sensor] Error adding entities for domain sensor with platform shelly Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 382, in async_add_entities await asyncio.gather(tasks) File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 548, in _async_add_entity entity.add_to_platform_abort() File "/config/custom_components/shelly/device.py", line 52, in add_to_platform_abort super().add_to_platform_abort() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 800, in add_to_platform_abort self._call_on_remove_callbacks() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 793, in _call_on_remove_callbacks self._on_remove.pop()() File "/config/custom_components/shelly/device.py", line 62, in _remove_handler self._dev.cb_updated.remove(self._updated) ValueError: list.remove(x): x not in list 2022-06-07 09:30:34 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 382, in async_add_entities await asyncio.gather(tasks) File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 548, in _async_add_entity entity.add_to_platform_abort() File "/config/custom_components/shelly/device.py", line 52, in add_to_platform_abort super().add_to_platform_abort() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 800, in add_to_platform_abort self._call_on_remove_callbacks() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 793, in _call_on_remove_callbacks self._on_remove.pop()() File "/config/custom_components/shelly/device.py", line 62, in _remove_handler self._dev.cb_updated.remove(self._updated) ValueError: list.remove(x): x not in list
Can confirm:
Running ShellyForHASS v1.0.0 with Home Assistant Core 2022.6.4 - Home Assistant Supervisor 2022.05.3 + Home Assistant OS 8.1 and logs are flooding with errors.
Mostly these ones:
2022-06-10 07:55:26 ERROR (MQTT connection) [pyShelly] Error receiving MQTT message, [Errno 104] Connection reset by peer, Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/pyShelly/mqtt_server.py", line 29, in _loop head = b(self._connection.recv(1)) ConnectionResetError: [Errno 104] Connection reset by peer Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/pyShelly/mqtt_server.py", line 29, in _loop head = b(self._connection.recv(1)) ConnectionResetError: [Errno 104] Connection reset by peer
sometimes the ones from dwarfer166
Same issue after upgrade to 1.0.0. Make HA (2022.6.5) non usable.
2022-06-12 20:20:21 ERROR (MainThread) [homeassistant.components.binary_sensor] Error adding entities for domain binary_sensor with platform shelly
Traceback (most recent call last):
File "/srv/homeassistant/lib/python3.9/site-packages/homeassistant/helpers/entity_platform.py", line 382, in async_add_entities
await asyncio.gather(*tasks)
File "/srv/homeassistant/lib/python3.9/site-packages/homeassistant/helpers/entity_platform.py", line 548, in _async_add_entity
entity.add_to_platform_abort()
File "/home/homeassistant/.homeassistant/custom_components/shelly/device.py", line 52, in add_to_platform_abort
super().add_to_platform_abort()
File "/srv/homeassistant/lib/python3.9/site-packages/homeassistant/helpers/entity.py", line 800, in add_to_platform_abort
self._call_on_remove_callbacks()
File "/srv/homeassistant/lib/python3.9/site-packages/homeassistant/helpers/entity.py", line 793, in _call_on_remove_callbacks
self._on_remove.pop()()
File "/home/homeassistant/.homeassistant/custom_components/shelly/device.py", line 62, in _remove_handler
self._dev.cb_updated.remove(self._updated)
ValueError: list.remove(x): x not in list
2022-06-12 20:20:21 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/srv/homeassistant/lib/python3.9/site-packages/homeassistant/helpers/entity_platform.py", line 382, in async_add_entities
await asyncio.gather(*tasks)
File "/srv/homeassistant/lib/python3.9/site-packages/homeassistant/helpers/entity_platform.py", line 548, in _async_add_entity
entity.add_to_platform_abort()
File "/home/homeassistant/.homeassistant/custom_components/shelly/device.py", line 52, in add_to_platform_abort
super().add_to_platform_abort()
File "/srv/homeassistant/lib/python3.9/site-packages/homeassistant/helpers/entity.py", line 800, in add_to_platform_abort
self._call_on_remove_callbacks()
File "/srv/homeassistant/lib/python3.9/site-packages/homeassistant/helpers/entity.py", line 793, in _call_on_remove_callbacks
self._on_remove.pop()()
File "/home/homeassistant/.homeassistant/custom_components/shelly/device.py", line 62, in _remove_handler
self._dev.cb_updated.remove(self._updated)
ValueError: list.remove(x): x not in list
Same issue here. Any solutions?
Same issue here, adding Sentry traceback: https://sentry.io/share/issue/b658e6c9ee024954b301365aa185aa79/
The monkeypatch I used was to ignore exception raised:
# custom_components/shelly/device.py:60
def _remove_handler(self):
self._is_removed = True
try:
self._dev.cb_updated.remove(self._updated)
except ValueError:
pass
self._dev.lazy_load = True
It seems harmless to me to ignore failed de-registration of update handler in case it is not registered.
Maybe it could even be an official solution? But I am not confident enough to submit an official PR, because I have no context and I made no investigation about why _remove_handler
is called when it probably should not be.
I was doing a little bit of debugging and found out why this issue is popping up:
2022-06-20 20:01:57 DEBUG (MainThread) [homeassistant.components.binary_sensor] Not adding entity Technikraum-SW - MQTT connected because it's disabled
so it is because of a disabled binary sensor (in my case i don't care about mqtt connected. will investigate further to get a proper fix.
issue fixed in https://github.com/StyraHem/ShellyForHASS/pull/687
@hakana please check PR if this is a real fix for you, otherwise please give directions for a better way.
The monkeypatch I used was to ignore exception raised:
# custom_components/shelly/device.py:60 def _remove_handler(self): self._is_removed = True try: self._dev.cb_updated.remove(self._updated) except ValueError: pass self._dev.lazy_load = True
It seems harmless to me to ignore failed de-registration of update handler in case it is not registered.
Maybe it could even be an official solution? But I am not confident enough to submit an official PR, because I have no context and I made no investigation about why
_remove_handler
is called when it probably should not be.
Why don't you just check if the item is in the list or not?
It may be micro-optimization, but checking if an item is in a list and then removing it requires 2 times walkthrough the list = O(2N). While simply removing it and catching an exception is a single walkthrough = O(N). Still asymptotically O(N), but slightly more optimal. :) Also, it is more idiomatic in Python.
Hi
I have manually applied the fix for #687 in my setup and how my HA log is filling up with the following messages for every disabled shelly entity I have:
2022-07-09 07:32:01 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 504, in async_update_ha_state raise RuntimeError(f"Attribute hass is None for {self}") RuntimeError: Attribute hass is None for <Entity Shelly RGBW2 - 80AA1E-3: off>
I went back to version 0.3.5
It may be micro-optimization, but checking if an item is in a list and then removing it requires 2 times walkthrough the list = O(2N). While simply removing it and catching an exception is a single walkthrough = O(N). Still asymptotically O(N), but slightly more optimal. :)
Thanks for the explanation! Now it makes sense to me why you did it this way. I think your change makes more sense. Do you want to create a pull request with your change? Than I will close mine.
Hi I have manually applied the fix for #687 in my setup and how my HA log is filling up with the following messages for every disabled shelly entity I have:
2022-07-09 07:32:01 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 504, in async_update_ha_state raise RuntimeError(f"Attribute hass is None for {self}") RuntimeError: Attribute hass is None for <Entity Shelly RGBW2 - 80AA1E-3: off>
Hi, I've also manually applied the fix, and on top of the same message, it also seemed to fix my memory leak issue. It was quite significant, it quickly filled up my 4GB raspberry pi within a day before the fix, and had to perform scheduled core restart twice daily.
Glad to explain. :)
Do you want to create a pull request with your change?
No, I don't think is a good idea to suppress symptoms without looking for cause. That's what I already explained.
And I don't want to invest more time into digging deeper because I plan to migrate to Zigbee soon.
Hi
Is there any news about it ? Every day, the RAM is full :(
What could be done ?
Thanks :)
As I experienced this too recently due to a user mistake: I had to restore a backup before that user mistake. I fear that integrating new Shelly devices will result in the same issue.
On the long-term (maybe mid-term) I'm sad to say: migrating away from this custom to the default/official/core integrated one will probably be not a bad choice, especially when looking at the amount of open issues.
In fact, this and another one or two issues in this custom Shelly repo block me to finally update to HA 2022.7. Still stuck on 2022.6.
Nothing particular to add to solve this. Dev experts, the stage is yours...
Hello, is there any patch for this issue? Also a manual code fix. The log file is flooded with error messages at a rate of one per second making impossible to read/check the file itself. Thanks.
Hello, is there any patch for this issue? Also a manual code fix. The log file is flooded with error messages at a rate of one per second making impossible to read/check the file itself. Thanks.
I recently applied the fixed mentioned above in #687. Whilst not ideal it has cleared the logs of all the noise that made it impossible to troubleshoot any other issues. I would recommend applying.
Wasn't that fix (maybe I mix it up with one of the many other issues, in this case sorry) just silencing the log output?
So basically putting a plaster on the bleeding wound (fighting symptoms, not causes)?
If not but actually fixing root causes: merging and releasing would be great.
Wasn't that fix (maybe I mix it up with one of the many other issues, in this case sorry) just silencing the log output?
So basically putting a plaster on the bleeding wound (fighting symptoms, not causes)?
If not but actually fixing root causes: merging and releasing would be great.
Yes, absolutely. Which is why I hadn't done it up until now. I just couldn't get anything else done with the log being flooded so much (and it was causing crashes).
I will still monitoring this thread for a better and more permanent solution. Just saying it did the job...as a plaster :-)
People still seeing this might check if they maybe use multiple ShellyForHass integrations (like in https://github.com/StyraHem/ShellyForHASS/issues/696).
I just created a prerelease for 1.0.1 that should fix this issue.
You need to enable beta versions in HACS to see and install it.
Please let me know the result when testing so I can release it fully shortly.
I have installed the Beta version. It does seemed to have calmed the RAM usage down but I'm still getting the below in the logs. It appears for various Shelly devices:-
Logger: homeassistant Source: helpers/entity.py:504 First occurred: 17:15:25 (1 occurrences) Last logged: 17:15:25
Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 504, in async_update_ha_state raise RuntimeError(f"Attribute hass is None for {self}") RuntimeError: Attribute hass is None for <Entity Shelly Dimmer - 98F4ABD0EBE5-1 - Switch: off>
I have installed the Beta version. It does seemed to have calmed the RAM usage down but I'm still getting the below in the logs. It appears for various Shelly devices:-
This error is because of an earlier error in the log. Can you restart HA and look for the first error from Shelly in the log.
I have installed the Beta version. It does seemed to have calmed the RAM usage down but I'm still getting the below in the logs. It appears for various Shelly devices:-
This error is because of an earlier error in the log. Can you restart HA and look for the first error from Shelly in the log.
Thank you for looking at this.
After restarting HA the first Shelly warning in the log is:-
2022-08-03 21:54:37.865 WARNING (MainThread) [homeassistant.helpers.frame] Detected integration that uses deprecated async_get_registry
to access entity registry, use async_get instead. Please report issue to the custom integration author for shelly using this method at custom_components/shelly/init.py, line 356: await self.hass.helpers.entity_registry.async_get_registry()
@proxlx That is the first warning can you see any errors also before "Attribute hass is None for {self}"?
@hakana Unfortunately not, those appear to be the first Shelly Errors that show up.
2022-08-03 23:42:35.355 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 504, in async_update_ha_state raise RuntimeError(f"Attribute hass is None for {self}") RuntimeError: Attribute hass is None for <Entity Shelly Dimmer - D17771-1 - Switch: off>
2022-08-03 23:42:35.357 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 504, in async_update_ha_state raise RuntimeError(f"Attribute hass is None for {self}") RuntimeError: Attribute hass is None for <Entity Shelly Dimmer - D17771-2 - Switch: off>
2022-08-03 23:42:35.474 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 504, in async_update_ha_state raise RuntimeError(f"Attribute hass is None for {self}") RuntimeError: Attribute hass is None for <Entity Shelly 1 - C45BBE787604 - Switch: off>
@hakana I think this issue is coming when you hide an entry. In my case I don't use the Shelly Cloud, so I do not need to see if the shelly is connected or not. If I am hiding the entry this issue is shown in the logs.
@hakana I think this issue is coming when you hide an entry. In my case I don't use the Shelly Cloud, so I do not need to see if the shelly is connected or not. If I am hiding the entry this issue is shown in the logs.
Yes, but should be fixed in latest release?
I still have errors, as reported by @proxlx, for hidden entities and high memory usage, though to a lesser extent than before.
WARNING (MainThread) [homeassistant.helpers.frame] Detected integration that uses deprecated `async_get_registry` to access entity registry, use async_get instead. Please report issue to the custom integration author for shelly using this method at custom_components/shelly/__init__.py, line 356: await self.hass.helpers.entity_registry.async_get_registry()
ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/srv/homeassistant/lib/python3.9/site-packages/homeassistant/helpers/entity.py", line 504, in async_update_ha_state
raise RuntimeError(f"Attribute hass is None for {self}")
RuntimeError: Attribute hass is None for <Entity Shelly RGBW2 - XXXXXX-1: 0.0>
ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/srv/homeassistant/lib/python3.9/site-packages/homeassistant/helpers/entity.py", line 504, in async_update_ha_state
raise RuntimeError(f"Attribute hass is None for {self}")
RuntimeError: Attribute hass is None for <Entity Shelly RGBW2 - XXXXXX-1: off>
...
A similar error for each hidden entity. Noticed that, for the same entity, something the error report <Entity Shelly RGBW2 - XXXXXX-1: off>, other times <Entity Shelly RGBW2 - XXXXXX-1: 0.0>. Do not know if this has any relevance.
Yes. @d3m3trius @hakana Still have this issue on V1.0.1 Usually need to reboot every 2 days due to memory usage. I hadn’t noticed that it was hidden entity’s that was possibly causing it but I do have a lot of Shelly devices with unused switch inputs etc which are all hidden in HA.
still getting this issues:
RuntimeError: Attribute hass is None for <Entity Shelly Plus 2PM - 4855199B4110-1 - Switch: off>
2022-08-22 20:52:00.977 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 504, in async_update_ha_state
raise RuntimeError(f"Attribute hass is None for {self}")
RuntimeError: Attribute hass is None for <Entity Shelly Plus 2PM - 4855199B4110-2 - Switch: off>
2022-08-22 20:52:00.979 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 504, in async_update_ha_state
raise RuntimeError(f"Attribute hass is None for {self}")
RuntimeError: Attribute hass is None for <Entity Shelly Plus 2PM - 4855199B4110-2 - Switch: off>
2022-08-22 20:52:01.134 ERROR (MainThread) [homeassistant.components.switch] Platform shelly does not generate unique IDs. ID shelly_ShellyPlus2PM_4855199B4110-1 is already used by switch.shelly_shellyplus2pm_4855199b4110_1 - ignoring switch.shelly_shellyplus2pm_4855199b4110_1
2022-08-22 20:52:01.136 ERROR (MainThread) [homeassistant.components.sensor] Platform shelly does not generate unique IDs. ID shelly_ShellyPlus2PM_4855199B4110-1_current_consumption is already used by sensor.shelly_shellyplus2pm_4855199b4110_1_current_consumption - ignoring sensor.shelly_shellyplus2pm_4855199b4110_1_current_consumption
2022-08-22 20:52:01.139 ERROR (MainThread) [homeassistant.components.switch] Platform shelly does not generate unique IDs. ID shelly_ShellyPlus2PM_4855199B4110-2 is already used by switch.shelly_shellyplus2pm_4855199b4110_2 - ignoring switch.shelly_shellyplus2pm_4855199b4110_2
2022-08-22 20:52:01.141 ERROR (MainThread) [homeassistant.components.sensor] Platform shelly does not generate unique IDs. ID shelly_ShellyPlus2PM_4855199B4110-2_current_consumption is already used by sensor.shelly_shellyplus2pm_4855199b4110_2_current_consumption - ignoring sensor.shelly_shellyplus2pm_4855199b4110_2_current_consumption
and this a LOT!
Yesterday, it depleted my monthly Sentry quota for second time. For anyone here who is interested: I switched to combination of MQTT & official Shelly integration. No big luck, unfortunately - official Shelly integration reports various timeout errors quite often (I assume from CoAP/HTTP long polling). Manually configured MQTT works flawlessly.
I have the same issue: `Logger: homeassistant.components.sensor Source: helpers/entity_platform.py:652 Integration: Sensor (documentation, issues) First occurred: 14:14:55 (60 occurrences) Last logged: 14:21:51
Platform shelly does not generate unique IDs. ID shelly_SHUNI-1_C45BBE6AD67A_voltage is already used by sensor.shelly_shuni_1_c45bbe6ad67a_voltage - ignoring sensor.shelly_shuni_1_c45bbe6ad67a_voltage`
I also have the issue with hidden entities. It is flooding my log with a couple of errors each second. It is present in all versions 1.0.x
2022-09-13 16:18:05.731 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 504, in async_update_ha_state
raise RuntimeError(f"Attribute hass is None for {self}")
RuntimeError: Attribute hass is None for <Entity Shelly 3EM - E8DB84D7262B-3 - Total returned: 0.0>
Is it a different error than the original error in this issue as it is marked as fixed? Shall we open a new issue?
@hakana Is it possible to look at this again please? The issue is marked as fixed but it’s still occurring with hidden entity’s causing a huge log and HA restarts. Many thanks!
This is a major issue. I was burning through 8gb of ram in about 6 hours. Couldn't figure out what it was so I started deleting HACS plugins. removing this plugin fixed ALL of my issues in HA.
Definitely replace this plugin with the official Shelly integration. It takes some effort, but mostly you just need to copy all the ids and change the ids accordingly to the new ones. Spent 2 hours last week but finally my ram usage is fine without any leak.
@Stewie3112 what is the official Shelly integration? is it the one which is using Shelly Cloud? 🤔
edit: ah, i think i found it: Shelly smart home?
@Stewie3112 what is the official Shelly integration? is it the one which is using Shelly Cloud? 🤔
edit: ah, i think i found it: Shelly smart home?
No, it uses CoIoT protocol, and it works flawlessly. See more here
Oh, you must uninstall first shellyforhass, remove it from hacs and restart HA
I know this isn’t the best way but if you unhide the hidden Shelly entities it does solve the problem completely also. I spent a few hours naming all the unused switch inputs etc on my devices and enabling them in HA and the RAM usage is back to normal. Hopefully there will be a proper fix for it though.
I also moved to official Shelly integration. No problems since then.
does somebody know if i can install the official shelly integration and shelly4hass parallel? in my case i have one shelly4hass installed and when i am installing the official one it looks like it is installing shelly4hass a second time.
i would like to do a stepp by stepp transision to the official one.
does somebody know if i can install the official shelly integration and shelly4hass parallel? in my case i have one shelly4hass installed and when i am installing the official one it looks like it is installing shelly4hass a second time.
i would like to do a stepp by stepp transision to the official one.
I don't think, having the same problem I had to completely remove shelly4hass before seeing the devices discovered from the official integration. Just make a backup, open a spreadsheet and copy all the relevant ids to keep in the new integration
https://community.home-assistant.io/t/new-ha-shelly-integration/227539/29
for migration discussions and necessary steps.
I had to completely remove shelly4hass before seeing the devices discovered from the official integration.
@Stewie3112 + @Rossez and others who already migrated successfully to the stock Shelly integration:
Did you remove Shelly4HASS by a) removing it only from HACS b) removing it from your integrations page and removing it from HACS (in that order) c) removing it from HACS and from your integrations page (in that order)
?
Cannot uninstall it currently because of https://github.com/StyraHem/ShellyForHASS/issues/712 and https://github.com/hacs/integration/issues/2894.
I had to completely remove shelly4hass before seeing the devices discovered from the official integration.
@Stewie3112 + @Rossez and others who already migrated successfully to the stock Shelly integration:
Did you remove Shelly4HASS by
a) removing it only from HACS
b) removing it from your integrations page and removing it from HACS (in that order)
c) removing it from HACS and from your integrations page (in that order)
?
Cannot uninstall it currently because of https://github.com/StyraHem/ShellyForHASS/issues/712 and https://github.com/hacs/integration/issues/2894.
Definitely option B, with a final full reboot
Environment
Describe the bug
Steps to Reproduce
not very sure where this error message is coming from. It is just flooding the logs
Expected behavior
no error msg.
Screenshots
Traceback/Error logs
Additional context