Slow performance opening state cards

[raccettura]

I think this started around 0.58.x..

Performance on this card is uncharacteristically slow, taking several seconds to open. I've noticed a few things about it:

1. Initially when HA starts it's zippy. It slows down after HA's been up for a little while.
2. in console only thing I've noticed is OMException: Quota exceeded
3. This is reproducible in Chrome and Fully (both using WebKit) on a Kindle Fire. All software at latest release versions.

History request I see doesn't seem to reflect the amount of time. I suspect it's JS.

Attached is a screenshot of several minutes after restart... it's only 2-ish seconds, not even that bad.

[raccettura]

I have a did save a dump from the chrome profiler as well.

[raccettura]

Here's an even worse one.

[raccettura]

Still reproducing this. Anyone have thoughts on some better way to debug this?

[raccettura]

Seems a few other people are having this issue:

https://community.home-assistant.io/t/ui-slow-down/35373/5 https://community.home-assistant.io/t/firefox-entity-state-cards-very-slow/33635/2

[raccettura]

After several hours of debugging, I've been able to reduce that it has something to do with the group component. It's the only thing that when disabled seems to bring back normal performance with no degradation over time.

There's not to much in mine to be honest. Several views, and I guess a fair number of entities at this point, but I don't think that should be terribly impactful.

[raccettura]

Also reproducible on Chrome Desktop. 1hr generated about a 3 second delay on opening a state card. I'm starting to think the UI keeps laying events on top of each other, and as time goes on it just becomes to slow to get through the stack.

[andrey-git]

@raccettura could you explain what to do to reproduce the issue on Chrome Desktop and what do you observe there? 3s delay between tap an "more-info" opening? Which card's more-info is that?

[andrey-git]

Did you leave it for an how with "more-info" open? Or with the "regular" view open and then tapped on an entity?

[arsaboo]

I have also been experiencing it, especially after the HA window is not manually refreshed for some time. After refreshing the tab, state cards load.

@andrey-git It sounds something like you are alluding to.

[raccettura]

@andrey-git : So here's a basic rundown as I can reduce it down at this point:

group.yaml has several views with 1-8 groups each. Nothing about them seems unusual, nothing that you don't see in the majority of the sample configs in github.

Open browser and tap on an entity. It opens in a few milliseconds. Go ahead and close it.

Wait about 60-90 minutes.

Try to view that entity again... now takes several seconds, the UI seems to have hung, suggesting it's a JS issue.

Restart the browser and performance returns to normal.

Server load is nominal the entire time.

[raccettura]

One other note: It seems much more pronounced on tablets as compared to a desktop browser. This further makes me think it's a polymer/js issue, the CPU is more limited making the issue much more obvious.

Reasonably sure sure this started around 0.58.

[andrey-git]

Not sure if related, but in #742 I'm adding some graphs improvements.

[raccettura]

I'm not really familiar with polymer, but I think it's time spent opening the card vs rendering the graph. Posted some breakdowns up above... not sure if that helps, or if there's a way for me to provide more helpful data.

[andrey-git]

If you can run a dev version and provide a breakdown from it it would be helpful, as with minified J's code it is hard to tell what those functions are.

[raccettura]

Gave it a shot, but so far no dice. cloned the dev repo and pointed to my production config... no errors in the log, but no http(s) for some reason. So that's kinda blocking me.

[raccettura]

Hasn't been running terribly long, but this is potentially interesting... looks like it's firing twice. But I'm not really familiar with polymer.

[raccettura]

Here's another.

[andrey-git]

What type of state card are you clicking?

[andrey-git]

Also: do you have any customui? If yes - do you observe the problem when it is turned off?

[raccettura]

No custom UI. I've been testing with the alarm card since that's where I first noticed it. Eyeballing it, seems to be universal. The toggles for things like lights seem to still be quick to respond. It's opening cards that has this problem.

No custom UI.

[andrey-git]

In what setting did you create the last 2 screenshots? The function names are not minimized but everything still comes from frontend_xxxx. Is this with

frontend: 
  development_repo: ...

[Coolie1101]

FYI: As per @raccettura request, my issue seems to be unrelated to "GROUPS"

I had configured a "HISTORY GRAPH" for four sensors which seem to be the cause of the front end slow down/halt. everything seems to be fine as of now after removing the graph, groups are still configured, front end, and card pop-ups respond instantaneously for now.

[andrey-git]

@Coolie1101 how many hours of data where you pulling with history_graph? Did you set the graphs to auto-refresh or not?

[Coolie1101]

Yes. hours_to_show: 240 refresh: 05

[andrey-git]

So it is drawing lots of points every 5 seconds. Hopefully #742 will make it a little better, but in any case it will probably be ok if you just remove the refresh or enlarge it to >60

[Coolie1101]

I'm going to do without it until after the holiday's, as it pretty much, rendered HA useless.

[raccettura]

In what setting did you create the last 2 screenshots? The function names are not minimized but everything still comes from frontend_xxxx.

I'm running dev against my prod configs with the recorder component commented out (was using mysql, can't get it working outside docker).

The history component is simply history:, so I don't think that's having any impact.

[andrey-git]

@raccettura #745 should remove the multiple handleMoreInfo fire. But I would suspect the performance gain here would be minor.

Could you patch it in and test?

[raccettura]

So I'm now about 90m into testing, and performance is still great even on tablet. Previously on desktop about 1hr in it got to roughly 1 sec delay as shown previously. Tablet was several times worse. Both are speedy right now.

Either this patch works, or building the frontend myself made the difference. Maybe merge it in and I'll do another test using docker: dev? That would likely be the most definitive test.

[raccettura]

Oh, I guess it's not really testable as part of a build until the frontend version gets bumped https://github.com/home-assistant/home-assistant/commit/3d5d90241f2bc5778f42f352b871171145a5acbb

[ciotlosm]

with the latest 0.60 I have seen a considerable interface slowdown. I had to remove initially "latest" (some people switched back to "es5" to make front-end work again in 0.60, but now if I leave home-assistant opened more in a browser I see this error on the backend:

Sat Dec 23 2017 12:56:40 GMT+0200 (GTB Standard Time)

Error handling request
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 410, in start
    resp = yield from self._request_handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web.py", line 325, in _handle
    resp = yield from handler(request)
  File "/usr/lib/python3.6/site-packages/aiohttp/web_middlewares.py", line 93, in impl
    return (yield from handler(request))
  File "/usr/lib/python3.6/site-packages/homeassistant/components/http/ban.py", line 58, in ban_middleware
    return (yield from handler(request))
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 213, in coro
    res = yield from res
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 213, in coro
    res = yield from res
  File "/usr/lib/python3.6/site-packages/homeassistant/components/http/__init__.py", line 430, in handle
    result = yield from result
  File "/usr/lib/python3.6/site-packages/homeassistant/components/frontend/__init__.py", line 485, in get
    _is_latest(self.js_option, request)
  File "/usr/lib/python3.6/site-packages/homeassistant/components/frontend/__init__.py", line 583, in _is_latest
    useragent = parse(request.headers.get('User-Agent'))
  File "/usr/lib/python3.6/site-packages/user_agents/parsers.py", line 254, in parse
    return UserAgent(user_agent_string)
  File "/usr/lib/python3.6/site-packages/user_agents/parsers.py", line 135, in __init__
    ua_dict = user_agent_parser.Parse(user_agent_string)
  File "/usr/lib/python3.6/site-packages/ua_parser/user_agent_parser.py", line 222, in Parse
    'user_agent': ParseUserAgent(user_agent_string, **jsParseBits),
  File "/usr/lib/python3.6/site-packages/ua_parser/user_agent_parser.py", line 249, in ParseUserAgent
    family, v1, v2, v3 = uaParser.Parse(user_agent_string)
  File "/usr/lib/python3.6/site-packages/ua_parser/user_agent_parser.py", line 51, in Parse
    match = self.user_agent_re.search(user_agent_string)
TypeError: expected string or bytes-like object

Also on the browser front-end:

Uncaught TypeError: Cannot read property 'forEach' of undefined
    at core-8f7092d0539f93e721211e2ee881a5f7.js:1
    at Array.forEach (<anonymous>)
    at Object.splitByGroups (core-8f7092d0539f93e721211e2ee881a5f7.js:1)
    at HTMLElement.computeCards (frontend-c873dc80aa0057847f22bcf3fff98f57.html:85)
    at t.r._debouncer.Polymer.Debouncer.debounce [as _callback] (frontend-c873dc80aa0057847f22bcf3fff98f57.html:85)
    at _timer._asyncModule.run (frontend-c873dc80aa0057847f22bcf3fff98f57.html:1)

[andrey-git]

it doesn't make sense for the backend error to be related to UI being open. It is caused by a missing useragent and I don't see how a browser can do that.

In any case setting javascript_version to a value other than auto will prevent HA from looking at the useragent.

[andrey-git]

@raccettura backend dev version was updated with a new FE pypi version.

[ciotlosm]

@andrey-git I am using google chrome on windows, Version 63.0.3239.84 (Official Build) (64-bit). Thank you for the suggestion. I will switch to using es5 and report back if I see slowing down of the front-end when clicking cards after it's been up for a while.

[raccettura]

@ciotlosm that sounds like the issue with events that might be fixed now. I'm actively testing now.

[raccettura]

2hrs in and it's still performing well @andrey-git... I think this did the trick! I'm going to keep it running on this build just in case.

I'm guessing if I kept digging down in that tree it was just recursively adding events. And that might explain why HA started to even freeze up and stop recording history... I likely DDoS'd myself by clicking on something after letting it run all night.

[ciotlosm]

@raccettura is this issue solved?

[raccettura]

Yes.

[ciotlosm]

Can you please close the issue?