Abnormal disk usage after usage of the integration for a couple hours

[fdebrus]

Disk is quickly getting full, even with no debug logging turned on.

[fdebrus]

When you report the issue for your setup could you

1- what is the level of default logging in configuration.yaml if any ? eg logger: default: info

2- what is the level of retention for the recorder ? eg recorder: purge_keep_days: 5 # Only keep the last 5 days of data commit_interval: 60 # Increase time between commits to the database

3- what the current size of logfile ?

4- what the current size of recorder (homeassistant DB)

Before erasing any data; check for the keywork aquarite in your logfile, is there plenty of entries ?

reminder, as you delete by restarting HA your home-assistant.log will be moved to home-assistant.log.1 so not cleaning any disk space. connect to your /config and delete the data accordingly.

if you trust you db has grow too much because of too many data point. Uninstall the integration, restart HA, go to developer / statistics, delete any Aquarite related statistics, update your recorder setting to record less data points; re-install the integration

[Hans1205]

I have now updated to 0.0.5.4

Her are my answers regarding your questions: No entry for logger in configuration.yaml No entry for recorder in configuration.yaml

@.*** Actually only this entry in logfile: 2024-04-22 18:23:04.648 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration aquarite which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant

I had deleted home-assistant.log and home-assistant.log.1, so I can’t take a look into the old logfiles. But I am still wondering why the Disk Usage is now above 90 % instead of 59 % yesterday. Do you have any idea how I could find out where the space is used?

@.***

Many thanks for your time and help! Hans

Von: fdebrus @.> Gesendet: Montag, 22. April 2024 17:09 An: fdebrus/hayward-ha @.> Cc: Subscribed @.***> Betreff: Re: [fdebrus/hayward-ha] Abnormal disk usage after usage of the integration for a couple hours (Issue #4)

When you report the issue for your setup could you

1- what is the level of default logging in configuration.yaml if any ? eg logger: default: info

No entry for logger in configuaration.yaml

2- what is the level of retention for the recorder ? eg recorder: purge_keep_days: 5 # Only keep the last 5 days of data commit_interval: 60 # Increase time between commits to the database

3- what the current size of logfile ? image.png (view on web)https://github.com/fdebrus/hayward-ha/assets/33791533/7f944e12-33a9-4aa7-88c5-b40d628af4b7

4- what the current size of recorder (homeassistant DB) image.png (view on web)https://github.com/fdebrus/hayward-ha/assets/33791533/fb0fd651-76e7-4abb-9c65-4fcb1d8814b1

Before erasing any data; check for the keywork aquarite in your logfile, is there plenty of entries ?

reminder, as you delete by restarting HA your home-assistant.log will be moved to home-assistant.log.1 so not cleaning any disk space. connect to your /config and delete the data accordingly.

if you trust you db has grow too much because of too many data point. Uninstall the integration, restart HA, go to developer / statistics, delete any Aquarite related statistics, update your recorder setting to record less data points; re-install the integration

— Reply to this email directly, view it on GitHubhttps://github.com/fdebrus/hayward-ha/issues/4#issuecomment-2069814145, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BFFSQBDGMS3NMZSWYKFVLSTY6URXFAVCNFSM6AAAAABGS6DMLOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRZHAYTIMJUGU. You are receiving this because you are subscribed to this thread.Message ID: @.**@.>>

[fdebrus]

What is the size of your recorder DB ? I suspect the issue is not related to log but data points. I need to trace it to confirm

[Hans1205]

If we are talking about the home-assistant_v2.db It was also in the screenshot: 840 MB …

Von meinem iPhone gesendet

Hans Moggert

Am 22.04.2024 um 19:40 schrieb fdebrus @.***>:



What is the size of your recorder DB ? I suspect the issue is not related to log but data points. I need to trace it to confirm

— Reply to this email directly, view it on GitHubhttps://github.com/fdebrus/hayward-ha/issues/4#issuecomment-2070385052, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BFFSQBCDIWH3TE2EBBRBCN3Y6VDSJAVCNFSM6AAAAABGS6DMLOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZQGM4DKMBVGI. You are receiving this because you commented.Message ID: @.***>

[fdebrus]

The original code do real time refresh of the data of the pool. Anytime the pool data is updated so is the integration and all sensors, etc... it happen every 10secs I'm not interested in realtime refresh, I will change towards a scheduled refresh interval, default to 5min and configurable by the user. This should clear the issue with your disk getting full with data points.

[fdebrus]

I've noticed a weird behavior

Start the integration, do not issue any commands.

The firestore trigger an update every minutes sometimes every 10min? Which is fine.

2024-04-22 22:22:54.949 DEBUG (MainThread) [custom_components.aquarite.coordinator] Manually updated Aquarite data 2024-04-22 22:33:14.722 DEBUG (MainThread) [custom_components.aquarite.coordinator] Manually updated Aquarite data 2024-04-22 22:34:17.085 DEBUG (MainThread) [custom_components.aquarite.coordinator] Manually updated Aquarite data 2024-04-22 22:45:17.125 DEBUG (MainThread) [custom_components.aquarite.coordinator] Manually updated Aquarite data 2024-04-22 22:56:17.186 DEBUG (MainThread) [custom_components.aquarite.coordinator] Manually updated Aquarite data 2024-04-22 23:04:31.235 DEBUG (MainThread) [custom_components.aquarite.coordinator] Manually updated Aquarite data 2024-04-22 23:06:13.574 DEBUG (MainThread) [custom_components.aquarite.coordinator] Manually updated Aquarite data

Now turn on / off the light, or any other commands. From then on, the update change to every 10secs frequency.

2024-04-22 23:18:45.566 DEBUG (MainThread) [custom_components.aquarite.coordinator] Manually updated Aquarite data 2024-04-22 23:18:56.830 DEBUG (MainThread) [custom_components.aquarite.coordinator] Manually updated Aquarite data 2024-04-22 23:19:08.091 DEBUG (MainThread) [custom_components.aquarite.coordinator] Manually updated Aquarite data 2024-04-22 23:19:19.328 DEBUG (MainThread) [custom_components.aquarite.coordinator] Manually updated Aquarite data 2024-04-22 23:19:30.605 DEBUG (MainThread) [custom_components.aquarite.coordinator] Manually updated Aquarite data

I've limited Firestore kwonledge, back to the documentation as I do not understand this behavior. So when new datapoints are received every 10secs for all sensors, etc... it fills up the recorder DB.

[folke]

When you delete a file on linux (log files for example), it won't be actually physically deleted when some process still has an open file descriptor to the file node.

So if you delete the home assistant log file, you also need to restart home assistant so that it will actually free the disk space.

I had the same issue with an older version of this component, where the log file grew to over 20GB in a matter of minutes. It wasn't kogging every 10 seconds, but the error was getting logged constantly. I had hundreds of thousands of entries in the log file.

With the latest version, the initial problem is now resolved for me.

TLDR: try restarting HA to reclaim that disk space (after deleting the log files).

[fdebrus]

closed in 0.0.6