Disk usage increased by 22GB after the 2024.8 and database migration

[piotrator]

The problem

During the update of the data base, after the 2024.8 update, I could see the disk usage increased by 22 GB, it took around 1h for this database update and in this time something filled the disk. I reverted all the changes and try again with 2024.8.1 but situation is same. System Information shows increase Disk used from 13.7 GB to 16.1 GB, Estimated database size (MiB) from 5537.44 MiB to 8052.45 MiB. But actual disk usage increased from 13 GB to 36 GB.

I don't see anything suspicious in the HA files, it looks almost same before and after the upgrade. Before and after the update this ca. 2GB. I use MariaDB add on, but don't know if this has the impact here or how I could check it. Everything seems to work after the update, however the huge size makes it very problematic to backup the HA. Any idea how to troubleshoot this issue and solve?

What version of Home Assistant Core has the issue?

core-2024.8.1

What was the last working version of Home Assistant Core?

core-2024.7.4

What type of installation are you running?

Home Assistant OS

Integration causing the issue

No response

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[home-assistant[bot]]

Hey there @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (recorder) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `recorder` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign recorder` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

---

recorder documentation recorder source _{^{(message by IssueLinks)}}

[boomam]

Same for me too.
2024.8.0 doubled my database size.
2024.8.2 increased it by another 10%.

[igalg99]

I also have similar issue. I am not sure where it is coming from. The available storage just decreases and decreases. I have added 30gb (!) to the storage and overallnhad 38gb free. The db file is 20gb. So I figured I should have at least that amount for the migration. But it keeps getting full. Within 2 hours since reboot after adding storage from 38 lgb available it dropped to 3.8 and it keep shrinking. I have tried to look through all files and do all the troubleshoot i could but I cannot find which file or files are increasing in size so much. Already deleted all local back ups and all unused images of add-ons to free space but still. I ah e total of 120gb storage for my home assistant. 20gb is the db file and about 15gb is the collective doceks and images. . I cannot find what is using so much space . Afrer I reboot , it goes back to 38gb available space (of the 120gb ) and starts again shrinking.
Recorder also dont record abymore. I tried to repair . I have no idea what is going on and how to troubleshoot it anymore :(

[piotrator]

I tried again from 2024.7.4 to 2024.8.2 and effect is same. Increase the disk usage by 25 GB. In my case the reboot of system or VM does not change anything here. I go back to 2024.7.4 and wait for some solution.

[LukasQ]

same here, BUT it did take some days to increase, not just right after the install. The release notes state this:

This release includes a database upgrade/migration that increases the size of the number we can store for the identifier fields. For a more technical explanation, we are upgrading integer fields for ID columns from a 32-bit integer to BIGINT, which is a 64-bit integer.

Which could increase disk usage, but by +100% (in my case 5G -> 10G)?

[erkr]

This doesn’t fail for all, I’m running haos on a pi4 with mariaDB. I noticed a temporary roughly double disk usage for the database during migration. After that was completed, the database was ca 10% bigger, with a normal growth rate after that

[igalg99]

Update, so after several hours after adding 30gb to my storage it again went and decreased until I had 3gb left. However then I noticed that empty space starting to grow again and I got in the log a message that migration has been completed. Next time I restarted home assistant I had all that extra space I added left free and it doesn't shrink anymore so I guess in my case migration has been completed . It seems that I had to increase and add a lot of space to my VM form this to complete. Not sure if everyone has this option.

However I still find it strange. I got now 120gb space for home assistant and 90 gb is occupied. I have the db file which is roughly 20gb + the db-wal file which is also roughly 20gb. About 15 gb is docker and addons and images. Still in total is about 55gb, let's say 60gb. There is still about 30gb which I don't know where they are hidden. Got no back ups stored locally.

Also is it normal to have both db and db-wal to confsum so much space. I understand the db which holds everything In my 2 years history but does the db-wal need to be as big ?

Probably it's a different question to a different topic/problem though.

[boomam]

Since the 2024.9.0 update, my database has increased in size by another few Gb following the upgrade completing.
Tried optmizing the tables in the database, but it does not reduce.
Equally, tried the purge options in the developer tools area of HA, size still does not reduce.

Logs shows only these of note -

1. Lots of "Adding foreign key constraint..."
2. A note on an upgrade from schema v45 to v47
3. A note on a successful upgrade to scheme v46.