Performance Issues & Memory Leaks with device / entity drop downs.

niemyjski commented 3 years ago

Checklist

[x] I have updated to the latest available Home Assistant version.
[x] I have cleared the cache of my browser.
[x] I have tried a different browser to see if it is related to my browser.

The problem

All Entity and Device dropdown are insanely slow to load and freeze the browser to the point I can't right click the page or select an entity for a considerable amount of time. This has only gotten worse the more devices I've integrated.

Expected behavior

The user interface and drop downs shouldn't freeze.

Steps to reproduce

Use the log book (or state viewer, love lace entity picker or automation editor).
Type to select an entity (notice slight lag).
Start deleting the entity id with backspace/delete, Mix it up and delete a few characters in from the end and then select all text and delete. I seem to get into this more and more with deletes, and everything becomes slower even selection.
Start typing a new entity id (it starts getting worse every iteration)
Keep repeating #2-4.

I also sometimes see this issue when I paste in the full entity id into a dropdown. It's not consistent, but over time I will have to reload the browser and be frustrated the whole time.

Environment

I notice this mostly on Safari (14.0.1) but have been able to reproduce on chrome (it takes longer to reproduce and isn't as noticeable). I've seen this both on PC running windows 10 (Ryzen 9 5900x) and Mac OS (2018 6core i7 MBP).

Home Assistant release with the issue: All-0.118.3
Last working Home Assistant release (if known):
Browser and browser version: Safari - Version 14.0.1 (16610.2.11.51.8) / Chrome- Version 87.0.4280.67 (Official Build) (x86_64)
Operating system: Mac OS - Big Sur / Windows 10

State of relevant entities

There are way to many entities to list here, but they are in all different kind of states from unavailable (a few). The ones I'm selecting are all available and mostly motion sensors.

Additional information

I have ~1630 entities and ~366 devices that show up in the config/entities lists. There is nothing that I can think of that would be causing this to be slow hardware wise. HA is running on a NUC 10 and I've been able to reproduce the client side issues on my high end desktop (12 Core Zen 3 5900x).

Safari

safari-timeline.zip

Here is a view of longest time taken while the cpu is going crazy:

Chrome

I have a hunch it has to do with massive lists of entities/devices and them all being loaded / rendered and then a bug in the filtering with deletes / adds.

zsarnett commented 3 years ago

@bdraco Did you create an integration to mock having over 1000 entities? Might be useful here to see if we can reproduce this

bdraco commented 3 years ago

@zsarnett http://github.com/bdraco/scaletest

zsarnett commented 3 years ago

Awesome. I will try to reproduce this sometime in the next week

zsarnett commented 3 years ago

Using Bdraco's Scale Test Component I was able to create 1600 extra entities.

I can definitely tell there is some lag in loading all pages but I can also reproduce the lag in the Logbook.

It seems to slow down a bit for me. Definitely not an unusable point but I can see a slow down. Seems to just stutter or lag a bit. I was unable to find it keep growing in memory but the scale test doesn't add any logbook history (not really possible I don't think) so maybe that would make the difference there. I am not sure what to do about the slowdown or what could make it better though. Nothing sticks out to me as obvious. The response from the backend seems pretty quick to pull the data when I do have history for an entity.

@bramkragten may have some more insight on what we can do to help performance overall

niemyjski commented 3 years ago

I see that but I also see it really go crazy when you try deleting the partial. Say you have entity named sensor.back_door_motion, type that in and then delete tion from the end, then delete _door then select all text and hit the delete key. Try it a few times.

niemyjski commented 3 years ago

I just was watching DrZzs stream and this happened twice while he was looking for entities using the new picker: https://youtu.be/tkdAFGf-_jU?t=2761

MoOx commented 3 years ago

Just so you know, I have a brand new Home Assistant OS installation and I am facing huge performance issues whenever a list is opened (integrations). It's unusable in Safari currently. I just have around 10 devices and 40 entities but it was like that since I installed my Hass OS.

This issue is the worst when I try to "+ Add Integration". Sometimes it freeze for a few seconds (which is frustrating) and sometimes the tab just crash (safari reload the tab).

It seems that this lists are not virtualised. I think they should (speaking as a frontend/mobile dev - using virtualised list whenever I have more items that can be show on a screen or 2).

Some infos: I have a raspberry pi 4 with 4g of ram and I am using this from a MacBook Pro 15" late 2016 2,9 GHz Quad-Core Intel Core i7, so perf should not be a problem. Happy to help if I can.

spacegaier commented 3 years ago

Since both you and OP are on Mac/Safari it seems to be browser specific, because I do not have the issue in Firefox or Chrome on Win 10 (also running an RPi 4, but with more than 40 entities) and zsarnett also needed huge amounts of dummy entities to notice any slowdowns.

github-actions[bot] commented 3 years ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

niemyjski commented 3 years ago

This is still an issue

github-actions[bot] commented 3 years ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

niemyjski commented 3 years ago

Not resolved

dtx360 commented 3 years ago

Yes this is still not resolved. Tried on the latest Safari 15 and it's still an issue. It's works fine on the same Mac using Chromium browser. Definitely a Safari issue someone needs to look into.

github-actions[bot] commented 2 years ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.