Refactor statistics to avoid creating tasks

bdraco commented 2 weeks ago

Proposed change

Refactor statistics to avoid creating tasks. Everything here was scheduling timer handles or events and can avoid creating tasks. Having the tasks in play made this integration much harder to debug.

The only task we need to create is the one time task to init from the database

We have been seeing increased cpu usage with statistics since at least 2024.2.x https://github.com/home-assistant/core/issues/110433 and it seems to have gotten worse in 2024.5.x https://github.com/home-assistant/core/issues/116650 https://github.com/home-assistant/core/issues/116731 as there is potential for tasks to race

Type of change

[ ] Dependency upgrade
[x] Bugfix (non-breaking change which fixes an issue)
[ ] New integration (thank you!)
[ ] New feature (which adds functionality to an existing integration)
[ ] Deprecation (breaking change to happen in the future)
[ ] Breaking change (fix/feature causing existing functionality to break)
[ ] Code quality improvements to existing code or addition of tests

Additional information

This PR fixes or closes issue: fixes #116650 fixes #116731
This PR is related to issue:
Link to documentation pull request:

Checklist

[x] The code change is tested and works locally.
[ ] Local tests pass. Your PR cannot be merged unless tests pass
[ ] There is no commented out code in this PR.
[ ] I have followed the development checklist
[ ] I have followed the perfect PR recommendations
[ ] The code has been formatted using Ruff (ruff format homeassistant tests)
[ ] Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

[ ] The manifest file has all fields filled out correctly.
Updated and included derived files by running: python3 -m script.hassfest.
[ ] New or updated dependencies have been added to requirements_all.txt.
Updated by running python3 -m script.gen_requirements_all.
[ ] For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
[ ] Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

[ ] I have reviewed two other open pull requests in this repository.

home-assistant[bot] commented 2 weeks ago

Hey there @thomdietrich, mind taking a look at this pull request as it has been labeled with an integration (statistics) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `statistics` can trigger bot actions by commenting: - `@home-assistant close` Closes the pull request. - `@home-assistant rename Awesome new title` Renames the pull request. - `@home-assistant reopen` Reopen the pull request. - `@home-assistant unassign statistics` Removes the current integration label and assignees on the pull request, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

bdraco commented 1 week ago

Tagged since a handful of users have confirmed this fixes the race between tasks that was cause the system to get slower and slower over time

ThomDietrich commented 1 week ago

@bdraco did you eventually figure out what caused this development?

bdraco commented 1 week ago

@bdraco did you eventually figure out what caused this development?

There were multiple tasks racing against each other, creating more tasks. By making everything callbacks the flow is synchronous so it can't happen anymore.

ThomDietrich commented 1 week ago

Okay, thanks for resolving this!!

home-assistant / core