WSL 2 should automatically release disk space back to the host OS

[dsmaher]

Is your feature request related to a problem? Please describe. I occasionally have workloads that consume large amounts of disk space for temporary use in /tmp. For example, transcoding large video files or processing large datasets exported from a database. Since WSL2 stores its filesystem on a .vhdx, that file grows when the dataset is processed and never releases that space. Also, it doesn't appear to re-use released disk space in the .vhdx when files are deleted... it seems to prefer to grow the file rather than re-use existing empty space. (I haven't explicitly tested that theory, but my workload deletes temporary files as they are used and I never had enough of them existing simultaneously to reach 250GB, but the size of the .vhdx eventually expanded to that size.

When I'm done with that workload, I have to:

wsl --shutdown
optimize-vhd -Path .\ext4.vhdx -Mode full

This is annoying since it basically means that my system backups include a huge .vhdx file that is mostly empty. Also, until about 2 weeks ago, it was consuming the entire remainder of my system disk. I had only 50GB free, which was enough to handle the workload, but since it continued to grow even after files were removed, that didn't matter. (I figured it was time to upgrade the disk size anyway)

Describe the solution you'd like Automatic compaction of the .vhdx, or a way to do so while WSL2 is still running so that I can schedule it for frequent cleanups.

Describe alternatives you've considered

• Going back to WSL1 since it uses the host filesystem and is faster accessing that than WSL2. However, this alternative makes docker less useful. Docker on WSL2 is actually almost perfect; not so much on WSL1.
• Exporting/Importing WSL 2 to move it to another disk. Not even sure if this is possible, but if it is, it would probably be fine for my desktop, but not really an option for my laptop. Not really worth the time either since that entire process would probably take about 30 minutes, but compacting is usually under 3.

[onomatopellan]

@Melkaz Did you try installing the Powershell module?

[Gigaclank]

leaving this here not a solution. but same idea as the author's solution just a bit more generic. Running this in powershell Admin mode.

wsl --shutdown
optimize-vhd -Path $env:LOCALAPPDATA\Docker\wsl\Data\ext4.vhdx,$env:LOCALAPPDATA\Packages\CanonicalGroupLimited.*\LocalState\ext4.vhdx  -Mode full

[Trass3r]

The problem is you do need to mount the vhd first for optimize to really do its job.

[zaverden]

ncdu helped me to find what I can also remove before slimming. sudo ncdu --exclude /mnt /

For example I found that /tmp directory was never cleaned. So I cleaned it manually.

[szthanatos]

For Windows 10 Home (alternative Optimize-VHD cmdlet):

wsl --shutdown
diskpart
# open window Diskpart
select vdisk file="C:\WSL-Distros\…\ext4.vhdx"
attach vdisk readonly
compact vdisk
detach vdisk
exit

Thanks to @davidwin for the tip #4699 (comment).

mark till this issue closed.

[RichardSilveira]

I tried all the solutions posted here but unfortunately any of them worked for me:

wsl --shutdown Optimize-VHD %path to ext4.vhdx% on both %LocalAppData%\Docker\wsl\data, and %LocalAppData%\CanonicalGroupLimited.Ubuntu18... ~note: you maybe should enable Hyper-V to use the Optimize-VHD command ~ but to use the command from @@davidwin that is not needed.

What worked for me:

• Stop Docker Desktop run the commands: wslconfig /unregister docker-desktop wslconfig /unregister docker-desktop-data

• Restart Docker Desktop

My %LocalAppData%\Docker\wsl\data\ext4.vhdx has decreased to 170gb to 800mb \o/

The solution comes from: https://github.com/docker/for-win/issues/7366#issuecomment-660395350

[yunusemre002]

@RichardSilveira's solution deletes all your images and containers! Before appy it think 2 times.

[nickjj]

If you're trying to compact Docker Desktop's VM you can Optimize-VHD (or equivalent) the ext4.vhdx file in: C:\Users\$USER\AppData\Local\Docker\wsl\data in the way same you would your WSL 2 distro.

Whether or not you decide to docker system prune (delete dangling images) or docker system prune -a (delete all images) before compacting it is up to you. I ran it with -a (knowing I was deleting everything) and I reclaimed ~50gb on my SSD after about 10 months of general usage.

I made a video about this topic here: https://nickjanetakis.com/blog/reclaiming-tons-of-diskspace-by-compacting-your-docker-desktop-wsl-2-vm because at first glance you might think compacting your WSL 2 instance is enough to reclaim your disk space but it's not and it makes sense on why it's not enough once you realize Docker Desktop is running in its own WSL 2 instance.

[sourcedelica]

I also get

The system failed to compact 'C:\Ubuntu_2004.2020.424.0_x64\ext4.vhdx': The process cannot
access the file because it is being used by another process. (0x80070020).

even after wsl --shutdown and sc vmcompute stop.

Using Sysinternals HANDLE to check, it is the System process (pid 4) that has the file open.

[GeorgII-web]

If compact don't helps (runs but not clean it up), move ext4.vhxd file to another folder and then use compact from diskpart. That helps me

!Do not start ubuntu while ext4.vhxd in different place, that can install new instance of os!

[phammanhhiep]

I made a small tool to find the vhdx and compact it on Windows 10 Home, using the method above:

https://github.com/mikemaccana/compact-wsl2-disk

powershell.exe .\compact-wsl2-disk.ps1

This should save you a little time if you have to fire up diskpart regularly.

For someone using Windows Home, use the command to run the script, powershell.exe -noprofile -executionpolicy bypass -file PATH_TO_THE_SCRIPT

I also make a simpler, but more error-prone script. Paste the below code to a file with extension ps1,

wsl -e sudo fstrim /
wsl --shutdown

$disk = "C:\Users\YOUR_NAME\AppData\Local\Packages\DISTRO_FOLDERNAME\LocalState\ext4.vhdx"

@"
select vdisk file=$disk
attach vdisk readonly
compact vdisk
detach vdisk
exit
"@ | diskpart

[RichardSilveira]

@yunusemre002 Thanks, I know that the problem is that compacts do not help. I'll try the solution from @GeorgII-web the next time.

[baranaldemir]

@merkuriy Thanks it worked. Hope they bring a quick solution for this soon.

[Gorthog]

I've used sudo fstrim, and then optimized the vhdx, and indeed vhdx size was reduced significantly.

However, looking at my vhdx size on windows it reports as 53GB:

while from inside linux it says just 41GB:

Where does the 12GB difference comes from?

[theAkito]

Once again, I'm stumbling over this issue for the Xth time. Usually, the workarounds provided here, worked fine so far, though now not even they seem to work anymore.

[nazdridoy]

With this Self-elevate script you can compact multiple vhdx at once.

# Self-elevate the script if required
if (-Not ([Security.Principal.WindowsPrincipal] [Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] 'Administrator')) {
    if ([int](Get-CimInstance -Class Win32_OperatingSystem | Select-Object -ExpandProperty BuildNumber) -ge 6000) {
        $CommandLine = "-File `"" + $MyInvocation.MyCommand.Path + "`" " + $MyInvocation.UnboundArguments
        Start-Process -FilePath PowerShell.exe -Verb Runas -ArgumentList $CommandLine
        Exit
    }
}

#Full Path of vhdx file
$d1 = #Ex: "E:\Docker\wsl\data\ext4.vhdx"
$d2 = ""
$d3 = ""
$d4 = ""
$d5 = ""
$d6 = ""

$paths = $d1, $d2, $d3, $d4, $d5, $d6

foreach ($file in $paths) {
    echo ""
    echo "Befor Shrinking File sizes in MB"
    Get-ChildItem -Path $file | Select-Object FullName, @{Name = "Size"; E = { $_.Length / 1MB } }
    echo ""
    echo "---------"
    echo "Shrinking $file"
    echo "---------"

    wsl --shutdown
    Optimize-VHD -Path $file -Mode Full

    echo ""
    echo "After Shrinking File sizes in MB"
    Get-ChildItem -Path $file | Select-Object FullName, @{Name = "Size"; E = { $_.Length / 1MB } }
    echo ""
    echo ""
    echo ""
    echo ""
}

Read-Host -Prompt "Press Enter to exit"

Btw, with a little tweak you can also run it on startup ("Start Docker when you login" should be disabled)

[onomatopellan]

For scripters, remember you can get the vhdx path of every distro (BasePath) trough registry in HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Lxss

[liyuling]

come on, Microsoft, this should be default

[Arniox]

For Windows 10 Home (alternative Optimize-VHD cmdlet):

wsl --shutdown
diskpart
# open window Diskpart
select vdisk file="C:\WSL-Distros\…\ext4.vhdx"
attach vdisk readonly
compact vdisk
detach vdisk
exit

Thanks to @davidwin for the tip #4699 (comment).

For some reason, when I try attach the vdisk to readonly; it says it's already attached. But then when I run compact vdisk, it fails and says it must be attached read only.

DISKPART> attach vdisk readonly

Virtual Disk Service error:
The virtual disk is already attached.

DISKPART> compact vdisk

Virtual Disk Service error:
The requested operation requires that the virtual disk be
attached read only.

DISKPART>

[ykfq]

You need to enable Hyper-V Module for Windows PowerShell if you got optimize-vhd is not recognized in CMD. follow this https://superuser.com/a/1307442/622176 Then use this command find and optimize the disk:

wsl.exe --shutdown
Set-Variable -Name "vhdxFile" -Value (Get-ChildItem -Recurse -Path $home\AppData\Local\Packages -Include ext4.vhdx).fullname
optimize-vhd -Path $vhdxFile -Mode full

[faberc]

Is there any way that this can get fixed? I don't have administrative access and I can't be doing this each time.

[MartinMuzatko]

npm/yarn are very heavy on the tmp folders, so usually in my biggest project, I can do ~4-6 times a full install before the disk fills up with basically air.

[shuoyang129]

after I run wsl.exe --shutdown optimize-vhd -Path .\ext4.vhdx -Mode full wsl -d Ubuntu I got an error: The file or directory is corrupted and unreadable and I can never get in the wsl system, any idea?

[Querela]

Always make a backup beforehand. It also seems that exporting and later re-importing will also shrink the space usage?

I tried the optimize-vhd command for my docker-* images, too, but afterwards it would recognize it anymore (and even resetting windows docker failed), so I did a clean re-install. I fortunately never touched my WSL system. Maybe you can use some VHDX tools to inspect and extract you important files, and copy them to a new install? (I would still suggest to backup the corrupted one. And maybe someone else has a better suggestion later on. Corrupted might only mean that WSL can't work with it, it might still completely fine as a filesystem for other tools?)

[shuoyang129]

I tried a tool(systools) suggest in blog, it never works. For now I'll backup the corrupted vhd for a while, looking forward that someone can have solutions.

[pryme0]

I tried all the solutions posted here but unfortunately any of them worked for me:

wsl --shutdown Optimize-VHD %path to ext4.vhdx% on both %LocalAppData%\Docker\wsl\data, and %LocalAppData%\CanonicalGroupLimited.Ubuntu18... ~note: you maybe should enable Hyper-V to use the Optimize-VHD command ~ but to use the command from @@davidwin that is not needed.

What worked for me:

• Stop Docker Desktop run the commands: wslconfig /unregister docker-desktop wslconfig /unregister docker-desktop-data
• Restart Docker Desktop

My %LocalAppData%\Docker\wsl\data\ext4.vhdx has decreased to 170gb to 800mb \o/

The solution comes from: docker/for-win#7366 (comment)

This solved the issue for me

[Querela]

I tried all the solutions posted here but unfortunately any of them worked for me: wsl --shutdown Optimize-VHD %path to ext4.vhdx% on both %LocalAppData%\Docker\wsl\data, and %LocalAppData%\CanonicalGroupLimited.Ubuntu18... ~note: you maybe should enable Hyper-V to use the Optimize-VHD command ~ but to use the command from @@davidwin that is not needed. What worked for me:

• Stop Docker Desktop run the commands: wslconfig /unregister docker-desktop wslconfig /unregister docker-desktop-data
• Restart Docker Desktop

My %LocalAppData%\Docker\wsl\data\ext4.vhdx has decreased to 170gb to 800mb \o/ The solution comes from: docker/for-win#7366 (comment)

This solved the issue for me

But this only works for Docker (not WSL2), and correct me if I'm wrong, this seems to purges all the data Docker uses. This in the end cleans everything up but so should a "Factory Reset" from Docker Desktop. If I would do this with WSL2, all my data would be lost which would be a concern for my setup on WSL2 even if not for Docker.

[woss]

after i ran the optimize-vhd -Path .\ext4.vhdx -Mode full i started getting the corrupted vhdx disk.

(base)PS C:\WINDOWS\system32> debian
The file or directory is corrupted and unreadable.
Press any key to continue...

Is there any way to fix this?

[zaverden]

wow, there are 2 VHD corruption reports in a week. I wonder if these are related to one of the latest windows updates..

the irony of the situation is that I want to shrink my disk to SAVE some space, and I don't have enough space for a backup copy of bloated VHD :\

[SunSatION]

Same issue, I had my ext4.vhdx corrupted once I applied the optimize-vhd command on the docker-data image. I had the same issue, resize of disk prior to corruption as it was full.

[gsxryan]

VHD corrupted here as well after running the optimize-vhd. Windows 10 20H2. I also have docker-desktop installed, but was not using it at the time.

I don't save anything important on it, luckily, so i just unregistered it and re-installed. wslconfig /unregister Ubuntu*

[umbersar]

I stopped docker and then ran the Optimize-VHD command: Optimize-VHD -Path .\ext4.vhdx -Mode Full

After compaction, I restarted docker and it ran fine. The vhdx file was compacted from 107 GB to 80 GB but I expected more as the total size of images on my machines is <10GB. So it does work but did not do a thorough job.

[brechtm]

I'm wondering whether this VHD corruption is consistently reproducible. @shuoyang129 @woss @SunSatION @gsxryan

If so, does the same happen when using diskpart compat instead of Optimize-VHD?

[nickjj]

but I expected more as the total size of images on my machines is <10GB

Did you run a docker system prune beforehand and did you shrink your Docker VM or only your WSL2 VM? You need to shrink both VMs. Docker's VM is in C:\Users\YOUR_USERNAME\AppData\Local\Docker\wsl\data and it can be shrunk using the same command.

[Trass3r]

Also it needs to be mounted first for Optimize-VHD to fully work. And fstrim could help as well.

[Makeshift]

For what it's worth, I'm running 20H2 (without Docker Desktop) and was able to optimize-vhd my Ubuntu WSL2 disk twice today with no corruption, so it isn't fully reproducible on 20H2 only.

[GeorgII-web]

No corruption optimize-vhd on: Win10 21H1 19043.1165

[aetheryx]

Corruption on 21H1 19043.1165 using diskpart compact vdisk ☹️

[aetheryx]

My corrupted ext4.vhdx seems to be unmountable even as a regular disk outside of WSL; it's not mountable by things like guestmount or 7zip's vhdx extractor. Maybe Diskpart is touching the vhdx header, or a partition table, causing the compacted vhdx to be unreadable.

The files and directories themselves still exist though. You are able to recover the files with DMDE:

(@woss @shuoyang129)

[redbaron]

For Windows 10 Home (alternative Optimize-VHD cmdlet):

wsl --shutdown
diskpart
# open window Diskpart
select vdisk file="C:\WSL-Distros\…\ext4.vhdx"
attach vdisk readonly
compact vdisk
detach vdisk
exit

Thanks to @davidwin for the tip #4699 (comment).

Also works for me without attach/detach commands

[darkvertex]

Just chiming in that I used the diskpart method via https://github.com/mikemaccana/compact-wsl2-disk/blob/main/compact-wsl2-disk.ps1 (and did sudo fstrim / just before.)

No apparent corruption for me. Windows version 10.0.19042 (aka Win10 20H2)

[rez23]

Hi I have write a little script for auto optimize all wsl distro in system (docker too) using optimize-vhd method, if it may be usefull is on my repository: https://github.com/rez23/PS-utility/blob/main/clean-wsl.ps1

[jerryasher]

As @nickjj did on https://github.com/microsoft/WSL/issues/4699#issuecomment-654517833 when I used diskpart to compact the vhdx I ended up with "The process cannot access the file because it is being used by another process."

This was after

1. shutting wsl down with wsl --shutdown
2. using diskpart in what seemed a successful compaction process (that seemed to recover almost nothing)
3. restart wsl with wsl

And so

1. I exited all my shells and tried wsl again, and the same error occured
2. I signed out of windows and signed back in, tried wsl again and the same error occurred
3. I rebooted my Windows Host, signed in, tried wsl again and this time

Success, I am back in my Ubuntu distro

Not knowing much about Windows, I am a bit surprised that signing out of windows apparently didn't clean up all the file handles

[mbomfim33]

I've tried moving the docker-desktop images from WSL to another disk using this comment's steps, but every time I restart Docker, it creates a new image under my old drive. So in the end, it doesn't cut for me.

Is it really not possible to select which drive I want my images/containers/volumes to be stored at? Sorry if I missed some comment here that explains the whys, just wondering if I can sort this out somehow. I'm tired of having to export the image every time I restart my computer, or Docker so my disk doesn't get to 100%.

[sarim]

I've tried moving the docker-desktop images from WSL to another disk using this comment's steps, but every time I restart Docker, it creates a new image under my old drive. So in the end, it doesn't cut for me.

The steps are correct. I store my wsl distro in other drives and didn't encounter any problem like you described. Sounds like a docker desktop issue. If it were me, I'd uninstall docker desktop altogether, create a new wsl distro from debian / ubuntu in a location you prefer, install docker engine (NOT desktop) (or any alternative like podman) for linux in that distro.

[mayhuifu]

This is a must fix. How long we live in a city without garbage collection!!!

[tanisc]

Still a problem. Cannot there be at least a limit to set when creating the distro image?

[theAkito]

• Stop Docker Desktop run the commands: wslconfig /unregister docker-desktop wslconfig /unregister docker-desktop-data

• Restart Docker Desktop

Apparently, this is so far the safest workaround to at least gain some space back. Thanks to the people warning about the possible corruption. I have a lot of important data saved in my WSL2 instances and even restoring from a backup would be a huge hassle and I would lose a couple of days of work.

I still have no idea, why this issue isn't classified with a higher priority, yet, especially after the reports of completely corrupted VHDs.

This issue is very serious and needs to be tackled ASAP.

[tanisc]

@theAkito It didnt work for me

[anlandu]

My corrupted ext4.vhdx seems to be unmountable even as a regular disk outside of WSL; it's not mountable by things like guestmount or 7zip's vhdx extractor. Maybe Diskpart is touching the vhdx header, or a partition table, causing the compacted vhdx to be unreadable.

The files and directories themselves still exist though. You are able to recover the files with DMDE:

(@woss @shuoyang129)

Thank you so much @aetheryx, had a corrupted disk after compact vdisk as well and this saved days of work!!!