WSL 2 should automatically release disk space back to the host OS

[dsmaher]

Is your feature request related to a problem? Please describe. I occasionally have workloads that consume large amounts of disk space for temporary use in /tmp. For example, transcoding large video files or processing large datasets exported from a database. Since WSL2 stores its filesystem on a .vhdx, that file grows when the dataset is processed and never releases that space. Also, it doesn't appear to re-use released disk space in the .vhdx when files are deleted... it seems to prefer to grow the file rather than re-use existing empty space. (I haven't explicitly tested that theory, but my workload deletes temporary files as they are used and I never had enough of them existing simultaneously to reach 250GB, but the size of the .vhdx eventually expanded to that size.

When I'm done with that workload, I have to:

wsl --shutdown
optimize-vhd -Path .\ext4.vhdx -Mode full

This is annoying since it basically means that my system backups include a huge .vhdx file that is mostly empty. Also, until about 2 weeks ago, it was consuming the entire remainder of my system disk. I had only 50GB free, which was enough to handle the workload, but since it continued to grow even after files were removed, that didn't matter. (I figured it was time to upgrade the disk size anyway)

Describe the solution you'd like Automatic compaction of the .vhdx, or a way to do so while WSL2 is still running so that I can schedule it for frequent cleanups.

Describe alternatives you've considered

• Going back to WSL1 since it uses the host filesystem and is faster accessing that than WSL2. However, this alternative makes docker less useful. Docker on WSL2 is actually almost perfect; not so much on WSL1.
• Exporting/Importing WSL 2 to move it to another disk. Not even sure if this is possible, but if it is, it would probably be fine for my desktop, but not really an option for my laptop. Not really worth the time either since that entire process would probably take about 30 minutes, but compacting is usually under 3.

[adrianciacob]

+1 for being able to move the .vhdx to a different partition, either via config file or the "Move" feature in the Control Panel of Windows (currently getting a message that it's disabled)

[dvescovi1]

+1 here too

[wizhippo]

+1 also add the ability to specify the initial size limit.

[dvescovi1]

I also would like to see a setting for initial partition size limit. It looks like all distros are installed with a 250GB partition size. I see no (easy) way to extend this. My yocto builds require slightly more space than this and they have to be on an ext3/4 partition so I can not use the /mnt/c or other NTFS partitions.

[davidwin]

In Home you don't even have the Optimize-VHD cmdlet, but thankfully (the slightly clumsier) diskpart command which can also be used to compact the .vhdx file. When I did this, it seemed like I first had to run sudo fstrim / in WSL to get a significant saving (14GB reclaimed with 8GB in use). A periodic background task that did this is really needed (and even better if it could be done online, of course).

The default partition size should have some connection to the size of the Windows host partition. What is the use of a limit that exceeds that of the host (unless actual magic is involved)? Also, artificially limiting the maximum size to a fraction of that of the host seems a bit unnecessary.

I think you really want an experience that is as close to that of WSL1 as possible, but with the incredible perf and compat gains, of course. That is, the WSL2 file system shouldn't feel like it was a static volume bound to a file on the host (although it is). Ideally, even the WSL partition/volume size should shrink and grow with available space on the host in a somewhat timely fashion. Now there is a stretch goal for you!

[taylankasap]

FWIW, since these commands looked benign I ran them to reclaim some disk space:

wsl --shutdown
optimize-vhd -Path .\ext4.vhdx -Mode full

This broke my docker. I had to reset to factory defaults & reinstall docker and restart the pc a few times along the way to fix. All docker data is gone (doesn't matter in my case because it's just dev environment). YMMV.

Microsoft Windows 19041.1 with WSL2 Docker Desktop Edge 2.1.7.0 (41536) | Engine: 19.03.5

[StefanScherer]

I've tried the commands as well. I could find an issue that docker engine (docker.sock) is not available after starting. But after restarting Docker Desktop and changing the settings / resources / wsl integration to off + apply, then to on again + apply did the trick. Looks like the sock

I ran the test a second time, but now my whole Ubuntu 18.04 distro is broken, the terminal closes either immediately or after some seconds without any text in it.

[taylankasap]

Maybe disabling wsl integration first, then quitting docker and then using the optimize-vhd command can make it work. But I'm not going to take that risk now.

[StefanScherer]

It's back alive, got the Ubuntu distro working again. Seem like I just have to wait a little bit. Terminal opens and I have the bash prompt. The WSL integration was disabled, after enabling again docker also works again in Ubuntu.

[MatyRi]

If you are looking for a way how to reclaim your disk space from Docker using WSL2, then there is a button for this if you are using Docker for Desktop (Edge 2.3.0.0). In the Dashboard / Troubleshoot - there is a Clean / Purge Data option which will let you select what do you want to remove. It safely shrank the Docker vhdx disk file to a few kilobytes. Just a warning that this obviously removes everything and not just unused space.

It does not solve the original issue but it's a quick way to a fresh start.

[merkuriy]

For Windows 10 Home (alternative Optimize-VHD cmdlet):

wsl --shutdown
diskpart
# open window Diskpart
select vdisk file="C:\WSL-Distros\…\ext4.vhdx"
attach vdisk readonly
compact vdisk
detach vdisk
exit

Thanks to @davidwin for the tip https://github.com/microsoft/WSL/issues/4699#issuecomment-565700099.

[sorenwacker]

I just got no disc space, I have not even installed much. I added a symbolic link that was no problem on my old system.

[nickjj]

I'm on Windows 10 Pro build 2004 here (stable 2020 spring edition) and I just noticed I lost 60GB on my primary SSD from this because I started moving my source code into WSL 2's file system and forgot one of my podcast sites has a bunch of wave files in a private directory.

When I run optimize-vhd -Path .\ext4.vhdx -Mode full as admin from PowerShell it throws an error with:

optimize-vhd : '.\ext4.vhdx' is not an existing virtual hard disk file. 
At line:1 char:1                                                                                                        
+ optimize-vhd -Path .\ext4.vhdx -Mode full
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidArgument: (:) [Optimize-VHD], VirtualizationException
+ FullyQualifiedErrorId : InvalidParameter,Microsoft.Vhd.PowerShell.Cmdlets.OptimizeVhd                                                                    

How can I get around this error? This put my SSD at a dangerously low level of space and I'd rather not have to reinstall the entire WSL 2 instance from scratch and reconfigure Docker every time I accidentally copy an unused file into WSL 2's file system.

[onomatopellan]

@nickjj You need to be in the directory where the distro's vhdx resides. In my case with Ubuntu20.04 I had to

wsl.exe --shutdown
cd C:\Users\onoma\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc\LocalState\
optimize-vhd -Path .\ext4.vhdx -Mode full

[nickjj]

Thanks, I was able to find it there too. That and what @StefanScherer mentioned about restarting Docker, along with disabling / enabling the distro in Docker Desktop again got things back to normal without any system level restart.

[rainabba]

My contribution, bringing together knowledge from above. It still requires a little manual work to get the DistroFolder variable set for the distro you want to target (you could have more than one, find them with wsl --list)

Cheers!

### Optimize (shrink) WSL 2 .vhdx
## Must be run in PowerShell as Administrator user
# DistroFolder found at: $env:LOCALAPPDATA\Packages\
# Examples:
#   CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc
#   CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc

cd $env:LOCALAPPDATA\Packages\REPLACE_ME_WITH_TARGET_DISTRO_FOLDERNAME\LocalState\
wsl --shutdown
optimize-vhd -Path .\ext4.vhdx -Mode full
#Run `wsl` or your favorite terminal to resume use

BELOW UNTESTED!! More info at https://docs.microsoft.com/en-us/powershell/module/hyper-v/resize-vhd?view=win10-ps

### Resize WSL 2 .vhdx
## WARNING!! UNTESTED!! Read and make note of the MaxSizeGB variable and functionality before running
## Must be run in PowerShell as Administrator user
# DistroFolder found at: $env:LOCALAPPDATA\Packages\
# Examples:
#   CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc
#   CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc

Set-Variable -Name "MaxSizeGB" -Value "32GB"
cd $env:LOCALAPPDATA\Packages\REPLACE_ME_WITH_TARGET_DISTRO_FOLDERNAME\LocalState\
wsl --shutdown
resize-vhd -Path .\ext4.vhdx -SizeBytes $MaxSize
#Run `wsl` or your favorite terminal to resume use

[eromoe]

Both cmd and powershell can't find optimize-vhd command even in admin preveliege

optimize-vhd : 无法将“optimize-vhd”项识别为 cmdlet、函数、脚本文件或可运行程序的名称。请检查名称的拼写，如果包括路径
，请确保路径正确，然后再试一次。
所在位置 行:1 字符: 1
+ optimize-vhd -Path .\ext4.vhdx -Mode full
+ ~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (optimize-vhd:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

[onomatopellan]

@eromoe optimize-vhd is only available in Windows 10 Pro with Hyper-v feature installed. Otherwise you will need to use the compact option in Diskpart.

[mieslep]

The WSL2 disk management is definitely a pig. Developing some (legacy app) Docker containers, after a week my "working" WSL2 .vhdx is 110G (with only 14 GB of actual data) and my docker-desktop-data .vhdx is 40G (with only 3 GB of actual data).

[nickjj]

Has anyone ever seen this when trying to run Optimize-VHD to compact things?

Optimize-VHD : Failed to compact the virtual disk.
The system failed to compact 'C:\Users\Nick\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc\LocalState\ext4.vhdx'.   Failed to compact the virtual disk.
The system failed to compact
'C:\Users\Nick\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc\LocalState\ext4.vhdx':   The process cannot access the file because it is being used by another process. (0x80070020).
At line:1 char:1
+ Optimize-VHD -Path .\ext4.vhdx -Mode full
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ResourceBusy: (:) [Optimize-VHD], VirtualizationException
+ FullyQualifiedErrorId : ObjectInUse,Microsoft.Vhd.PowerShell.Cmdlets.OptimizeVhd  

At this point my drive has almost no space and I'm not finding anything on Google on how to compact it, short of destroying the entire WSL 2 instance and remaking it from scratch.

I've stopped Docker Desktop and everything. There's no hint on what's using it.

[onomatopellan]

@nickjj With wsl.exe --shutdown should be enough to free all the mounted vhdx files.

[datocrats-org]

The second alternative mentioned is proving the fastest and most reliable for me, Exporting/Importing WSL 2 to move it to another disk. The other thing I have tried is converting between Fixed and Dynamic volumes in combination with this. My use is for docker's WSL2 integration, and the need to export and restore large data volumes (>250GB). This is similar to what most of the respondents are referring to, yet they do not always say whether they are using WSL1 vs. WSL2. WSL1 sets a fixed disk size as specified in your Docker settings. WSL2 uses windows default setting of 256GB max volume size. I could not bash into or ssh into the docker-data distro shown or access its ext4.vhdx volume through various methods. What would be really nice is if WSL would adopt a docker overlay2 driver or some other more friendly HyperV linux filesystem, faster than vhdx on ntfs, with an API to control allocation. Even better would be some type of cloud filesystem integration that could rival local dev speeds.

Here are my notes in gist

[jcoutch]

I just symlinked ~\AppData\Local\Docker\wsl\data to a folder on a different partition/drive, and moved the VHDX. While the problem with the VHDX growing is still there, at least it won't eat all the space on my primary drive. I wrote up a blog post about it, and here's the script I used (use at your own risk):

$ErrorActionPreference = "Stop"

$newLocation = "E:\VMs\WSL2\"

cd "~\AppData\Local\Docker\wsl\data"
wsl --shutdown
Optimize-VHD .\ext4.vhdx -Mode Full
mkdir $newLocation -Force
mv ext4.vhdx $newLocation
cd ..
rm "data"
New-Item -ItemType SymbolicLink -Path "data" -Target $newLocation

[onomatopellan]

@mikemaccana That's where Docker Desktop with WSL2 backend saves the docker images. Use the Troubleshoot option in the Dashboard to clean them.

[mikemaccana]

OK, ran:

To get rid of the docker images that had been accumulating. I have my hard drive back, and Docker is now pulling down images (good) but it's failing with:

ERROR: Service 'postgres' failed to build: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 192.168.65.1:53: read udp 192.168.65.3:39176->192.168.65.1:53: i/o timeout

Which I'm now troubleshooting.

Edit: since the IP addresses are local, poking around in Docker networking config and restarting docker fixed it.

[onomatopellan]

For further Docker Desktop issues is better to ask them in https://github.com/docker/for-win/issues

[AmineI]

@mikemaccana 150GB taken in Docker wsl/data vdhx

This issue is also logged here indeed https://github.com/docker/for-win/issues/244 , go there for Docker-specific elements; as mentioned by onomatopellan above.

[blackk100]

The .vhdx file can be moved by using the --export and --import options: https://docs.microsoft.com/en-us/windows/wsl/reference ; https://github.com/MicrosoftDocs/WSL/issues/412

wsl --shutdown
wsl -l -v
wsl --export <DistroName> <PathToTarArchive>
wsl --unregister <DistroName>
wsl --import <DistroName> <PathToDistroNewDirectory> <PathToTarArchive>
wsl -l -v

You'll get the <DistroName> from the output of wsl -l -v.

[monotykamary]

I'm fine with wsl2 just having an convenient cli option to compact the disk. Releasing disk space periodically sounds like an ssd killer unless we can schedule it when the disk size balloons over a certain threshold (e.g: 20gb).

[acmilannesta]

For Windows 10 Home (alternative Optimize-VHD cmdlet):

wsl --shutdown
diskpart
# open window Diskpart
select vdisk file="C:\WSL-Distros\…\ext4.vhdx"
attach vdisk readonly
compact vdisk
detach vdisk
exit

Thanks to @davidwin for the tip #4699 (comment).

Thanks for sharing. But after I tried your method, the ubuntu terminal close immediately after it opens. Just like this case: https://github.com/microsoft/WSL/issues/4699#issuecomment-575635657

[caot]

https://stephenreescarter.net/how-to-shrink-a-wsl2-virtual-disk/

The above link mentioned the way as the follows:

PS C:\tmp> diskpart

Microsoft DiskPart version 10.0.19041.1

Copyright (C) Microsoft Corporation.
On computer: *****

DISKPART> select vdisk file="C:\Users\<UID>\AppData\Local\Docker\wsl\data\ext4.vhdx"

DiskPart successfully selected the virtual disk file.

DISKPART> compact vdisk

  100 percent completed

DiskPart successfully compacted the virtual disk file.

Optimize-VHD failed miserably the same as that mentioned by nickjj above.

[triump2020]

My Win10 is not Pro version , so only can solve it through compacting vdisk ? Oh, my god . But Is there a risk of disk-performance degradation?

[triump2020]

How did i know the free space of vhdx in WSL2 ? Thanks !

[onomatopellan]

@triump2020 Correct. In your case you will need to use Diskpart compact vdisk pointing to the ext4.vhdx inside the Ubuntu folder. Compacting a vhdx file won't degrade disk performance.

Your WSL2 disk is /dev/sdb. You are using 67Gb but it's taking 127gb because is not compacted.

[triump2020]

@onomatopellan Thanks!

[arnisjuraga]

For Windows 10 Home (alternative Optimize-VHD cmdlet):

wsl --shutdown
diskpart
# open window Diskpart
select vdisk file="C:\WSL-Distros\…\ext4.vhdx"
attach vdisk readonly
compact vdisk
detach vdisk
exit

Thanks to @davidwin for the tip #4699 (comment).

Important thing to note: Sometimes compacting WSL2 virtual disk with diskpart has no effect. Try running sudo fstrim / from WSL before wsl --shutdown:

sudo fstrim /

[mikemaccana]

I made a small tool to find the vhdx and compact it on Windows 10 Home, using the method above:

https://github.com/mikemaccana/compact-wsl2-disk

powershell.exe .\compact-wsl2-disk.ps1

This should save you a little time if you have to fire up diskpart regularly.

[marceloavf]

Thanks @mikemaccana, able to reduce 5GB in ext4

[Trass3r]

For Windows 10 Home (alternative Optimize-VHD cmdlet):

You can get it via

cd %TEMP%
dir /b %SystemRoot%\servicing\Packages\*Hyper-V*.mum > hyper-v.txt
for /f %i in ('findstr /i . hyper-v.txt 2^>nul') do dism /online /norestart /add-package:"%SystemRoot%\servicing\Packages\%i"
dism /online /enable-feature /featurename:Microsoft-Hyper-V-All /LimitAccess /ALL

[truboxl]

You don't need to enable all Hyper-V features in Optional Features Just Hyper-V Module for Windows Powershell to get Optimize-VHD cmdlet and Hyper-V Services to get it working https://superuser.com/questions/1307441/powershell-resize-vhd-is-not-recognized-as-the-name-of-a-cmdlet

Not enabling Hyper-V Services causes this error:

optimize-vhd : The Hyper-V Management Tools could not access an expected WMI class on computer '<pc-name-here>'. This
may indicate that the Hyper-V Platform is not installed on the computer or that the version of the Hyper-V Platform is
incompatible with these management tools.
At line:1 char:1
+ optimize-vhd -Path .\ext4.vhdx -Mode Full
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Optimize-VHD], VirtualizationException
    + FullyQualifiedErrorId : Unspecified,Microsoft.Vhd.PowerShell.Cmdlets.OptimizeVhd

Otherwise, https://github.com/microsoft/WSL/issues/4699#issuecomment-627133168 is perfectly good alternative and can be written as a script for a one line diskpart command

[mikemaccana]

WSL2 Compacter has gotten a couple of PRs from the community and now does both WSL2 and Docker disks at once 🙂

[Trass3r]

The cmdlet version is basically

$files = Get-ChildItem -Path C:\Users -Recurse -Include "ext4.vhdx" -Force -ErrorAction SilentlyContinue | Select-Object FullName, Length
foreach ($file in $files) {
    echo "Compacting $file"
    wsl --shutdown
    Mount-VHD -Path $file.FullName -ReadOnly
    Optimize-VHD -Path $file.FullName -Mode Full
    Dismount-VHD $file.FullName
}

[stefanloerwald]

For people who see only a tiny improvement on optimize-vhd: It helps to prune docker first:

docker system prune -a --filter "until=240h"

Careful! With this command you prune everything that is older than 240 hours (10 days). It'll remove

• all stopped containers
• all networks not used by at least one container
• all images without at least one container associated to them
• all build cache

The result in my case was:

• Original size of vhdx: 200GB
• After optimization: 195GB
• Pruning reported "reclaimed 155GB"
• Optimizing again brought the disk size down to 94GB

So while it is still quite large, pruning resources that were older than 10 days made quite a big difference in how much the virtual disk could be optimized.

[h45h-1nclud3]

I hope this helps to automate solving this issue

https://github.com/h45h-1nclud3/WSL-Slim

[NikhilPr95]

In Home you don't even have the Optimize-VHD cmdlet, but thankfully (the slightly clumsier) diskpart command which can also be used to compact the .vhdx file. When I did this, it seemed like I first had to run sudo fstrim / in WSL to get a significant saving (14GB reclaimed with 8GB in use). A periodic background task that did this is really needed (and even better if it could be done online, of course).

The default partition size should have some connection to the size of the Windows host partition. What is the use of a limit that exceeds that of the host (unless actual magic is involved)? Also, artificially limiting the maximum size to a fraction of that of the host seems a bit unnecessary.

I think you really want an experience that is as close to that of WSL1 as possible, but with the incredible perf and compat gains, of course. That is, the WSL2 file system shouldn't feel like it was a static volume bound to a file on the host (although it is). Ideally, even the WSL partition/volume size should shrink and grow with available space on the host in a somewhat timely fashion. Now there is a stretch goal for you!

This does not work for me as I get the error -

DiskPart has encountered an error: The process cannot access the file because it is being used by another process.
See the System Event Log for more information.

I cannot use Optimize-VHD either as it is not recognized as a command on my Windows 10 (Home) system with Powershell v5.1. Is there anyway to get around these issues?

[NikhilPr95]

@eromoe optimize-vhd is only available in Windows 10 Pro with Hyper-v feature installed. Otherwise you will need to use the compact option in Diskpart.

This does not work for me either as I get the following response -

DiskPart has encountered an error: The process cannot access the file because it is being used by another process.
See the System Event Log for more information

How do I get around this?

[leifjones]

Regarding the first set of commands from this comment above:

If you have Docker Desktop running and set up to use wsl, it may automatically restart wsl after wsl --shutdown. So in my case (with Ubuntu), the following worked:

# 1) Quit Docker Desktop

# 2) Enter the following in PowerShell
cd $env:LOCALAPPDATA\Packages\
$wslDirectory = ls | sls "Canonical"
cd $wslDirectory/LocalState
wsl --shutdown
optimize-vhd -Path .\ext4.vhdx -Mode full

# 3) Open Docker Desktop again

[Myron-S]

Just a small point. if ext4.vhdx is not where it's supposed to be then it may have been moved to another directory or even another physical storage media.

[alex-way]

I've also found that even after running docker system prune -a my VHDX file wasn't decreasing in size.

I found the following directory which contains some additional files I cleared out which allowed me to reduce the size of my VHDX.

Please note; This isn't recommended as I honestly have no idea what the files are used for. However, in my case it worked a treat:

\\wsl$\docker-desktop-data\isocache

[Melkaz]

optimize-vhd is not recognized on my Window 10 pro machine when invoking from Powershell, yet Hyper-V is installed.

I'm definitely not an expert of Windows systems, but the fact that we have to jump through so many hoops to get this done is quite disconcerting.