WSL 2 consumes massive amounts of RAM and doesn't return it

[LordMonoxide]

• Your Windows build number: 18917

• What's wrong / what should be happening instead: WSL 2 starts using huge amounts of RAM after a while, just using it like normal. At the moment I'm using phpstorm, and did a dump/load of a database. Vmmem is using 7 GB of my 16 GB of RAM and not returning any, even though Ubuntu is actually using much less. I have seen it grow until nearly 100% of my system memory is in use, and it will not release it until I shut down the WSL 2 VM.

This may or may not be related to #4159

corey@Corey-Laptop:/mnt/c/WINDOWS/system32$ vmstat -s
     15235516 K total memory
       920348 K used memory
      1886048 K active memory
      6434312 K inactive memory
      6606548 K free memory
        76280 K buffer memory
      7632340 K swap cache
            0 K total swap
            0 K used swap
            0 K free swap
       163729 non-nice user cpu ticks
          298 nice user cpu ticks
        13177 system cpu ticks
     68988300 idle cpu ticks
         8962 IO-wait cpu ticks
            0 IRQ cpu ticks
        10022 softirq cpu ticks
            0 stolen cpu ticks
      1481417 pages paged in
      6792976 pages paged out
            0 pages swapped in
            0 pages swapped out
      1079177 interrupts
      5131981 CPU context switches
   1560599814 boot time
         8772 forks

[yodaldevoid]

I think the reboot issue is separate as I've been running into that even when I don't use WSL.

[cmeiklejohn]

Ah, good to know.

[mareksuscak]

The reboot issue got sorted in an update yesterday. Available in the fast ring.

[cmeiklejohn]

Sounds great, I'll upgrade accordingly.

[fanvinga]

@cmeiklejohn However if you have AMD GPU,I strongly recommend NOT to upgrade to this update (Windows Version 19002.1002).In this version,AMD GPU performance will be pretty bad.It seems to be influenced by the DirectX Driver issue. Related Discussion: https://amp.reddit.com/r/windowsinsiders/comments/dk3mo1/windows_insider_build_19002_bad_performance_with/

[cmeiklejohn]

I'm on Surface Book 2 13", so it's the NVIDIA GPU that just got fixed. :)

[cmeiklejohn]

I just bumped to 19002 and I've still got the reboot issue -- is there an issue somewhere on GitHub tracking this issue?

[mareksuscak]

@cmeiklejohn please note that it got stuck at reboot for me one last time upon the update was downloaded and was about to be installed, then I force rebooted and attempted the installation again. Then it actually got installed and work from then on.

[jimjenkins5]

It seems like this is diverting from the original issue.

@benhillis Can we get an update on the fix for the VM memory issue? ETA was 2 weeks almost 3 weeks ago.

[benhillis]

@jimjenkins5 - there are a lot of variables of when a given fix will make an Insider build. I would expect this change in the next week or so.

[pcottrell]

I updated to the latest "fast" build, and the issue still exists.

[dariothornhill]

Computer useless converting exiting Ubuntu to wsl 2. Is there any work around I can do to release memory or clear without interrupting the conversion?

[luciorq]

@dariothornhill, the best workaround that is working for me is to create a wslconfig file (%UserProfile%\.wslconfig) limiting the amount of RAM of the wsl2 vm and using additional swap space for RAM demanding operations.

[wsl2]
memory=4GB
swap=16GB
localhostForwarding=true

It is running really smooth in a laptop with SSD storage for the swap.

For the changes to the take effect you have to restart WSL 2 vm through powershell or cmd command wsl --shutdown

[cmeiklejohn]

@luciorq I've tried this, but these settings do not seem to be restricting the size of the VM for me. I have the same file with the same contents, but I still see that process using up all available memory on the machine.

[luciorq]

@cmeiklejohn, I'm not sure if it makes any difference, but I created the file in VSCode and using UTF-8 encoding and CRLF line endings.

Also restarted the machine and the WSL 2 VM.

[WSLUser]

Alternative to nocache : memcached. This should be used anyways for good practice.

[seasona]

Same problem in 19008.1000, the ram will surge if you compile or open a vscode, but it returns really slow if you close the vscode and so on. I tested that the ram charges from 280m to 1280m through the compile，drops to 1032m after the compile but never get back to 280m again. Fortunately the .wslcofig is really useful

[guaychou]

How can i fix this ?

[guaychou]

How can i fix this ?

Read above, they said they corrected this error, they only need to be implemented in the next windows update (possibly).

Okay , thank you

[myheartsgoon]

@dariothornhill, the best workaround that is working for me is to create a wslconfig file (%UserProfile%\.wslconfig) limiting the amount of RAM of the wsl2 vm and using additional swap space for RAM demanding operations.

[wsl2]
memory=4GB
swap=16GB
localhostForwarding=true

It is running really smooth in a laptop with SSD storage for the swap.

For the changes to the take effect you have to restart WSL 2 vm through powershell or cmd command wsl --shutdown

Thanks a lot, it really saved my 8GB ram laptop. I set ram limit to 512MB, and use 8GB swap as below:

[wsl2]
memory=512MB
swap=8GB
localhostForwarding=true

Before this change, ram usage always spikes to 1.5G+ after I run the docker, but for now, the ram usage is limit to 512MB as Task Explorer shows below and you can also find in machine that wsl2 start to use swap when memory 500MB exceeds.

[cmeiklejohn]

Weird, the %userprofile%\.wslconfig still doesn't work for me. I verified the right path and the right line endings. Do you have a blank line at the end of the file, perhaps?

[saikrishnav]

@cmeiklejohn - Can you elaborate? what settings are you using? It just worked for me as is.

[cmeiklejohn]

Looks like 19013 fixes this.

[benhillis]

@cmeiklejohn - Correct, 19013 has the fix for this.

[jingchangshi]

My Windows 10 Build 19013.vb_release.191025-1609 is still bothered by this issue. WSL2 Ubuntu 18.04 LTS.

I'm using VSCode remotely editing codes in WSL2. Vmmem consumes large memory and does not release them.

I close VSCode without executing wsl --shutdown. Vmmem does not release the memory.

Only after wsl --shutdown, the memory is given back to Windows.

[peteromio]

My Windows 10 Build 19013.vb_release.191025-1609 is still bothered by this issue. WSL2 Ubuntu 18.04 LTS.

exec free -h the safest is that the cache memory is occupying ram memory, free the cache and see if the memory returns to windows (before this did not work)

[jingchangshi]

My Windows 10 Build 19013.vb_release.191025-1609 is still bothered by this issue. WSL2 Ubuntu 18.04 LTS.

exec free -h the safest is that the cache memory is occupying ram memory, free the cache and see if the memory returns to windows (before this did not work)

free -h shows that buff/cache is not released.

As a try, I copied a folder of large size to another place. And this copy operation leads the buff/cache to increase a lot. When the copy is finished, the buff/cache is not released. So is Vmmem. Check the following pic.

[fanvinga]

My Windows 10 Build 19013.vb_release.191025-1609 is still bothered by this issue. WSL2 Ubuntu 18.04 LTS.

exec free -h the safest is that the cache memory is occupying ram memory, free the cache and see if the memory returns to windows (before this did not work)

free -h shows that buff/cache is not released.

As a try, I copied a folder of large size to another place. And this copy operation leads the buff/cache to increase a lot. When the copy is finished, the buff/cache is not released. So is Vmmem. Check the following pic.

You may need such command like sync; echo 3 > /proc/sys/vm/drop_caches to purge Linux cache.

[jingchangshi]

My Windows 10 Build 19013.vb_release.191025-1609 is still bothered by this issue. WSL2 Ubuntu 18.04 LTS.

exec free -h the safest is that the cache memory is occupying ram memory, free the cache and see if the memory returns to windows (before this did not work)

free -h shows that buff/cache is not released. As a try, I copied a folder of large size to another place. And this copy operation leads the buff/cache to increase a lot. When the copy is finished, the buff/cache is not released. So is Vmmem. Check the following pic.

You may need such command like sync; echo 3 > /proc/sys/vm/drop_caches to purge Linux cache.

OK. It works.

[bobotu]

Can WSL2 automatically drop some cache when the Windows host has pressure on memory?

[peteromio]

I want to report another behavior related to VSCode. I use VS Code to remotely do code development. I'm not so sure if this issue should be reported here or not.

In Windows 10 Build 19013.vb_release.191025-1609, VS Code version 1.39.2 with Remote-WSL plugin 0.39.9, doing search in VS Code leads Vmmem to consume large memory. And I need to manually release the cache by sudo sh -c "/bin/echo 3 > /proc/sys/vm/drop_caches". However, such behavior is really annoying. Why does searching in VS Code give such results?

Or should I report this to VS Code team?

Linux uses RAM as a cache when writing to disk, this to gain speed. It is a usual kernel behavior and yes, you can control this behavior. Research Linux and ways to control the cache. The comments above talk about how to mitigate that behavior automatically.

[jingchangshi]

Can WSL2 automatically drop some cache when the Windows host has pressure on memory?

I feel like it is. But I'm not sure. In a short period of time like 1 minute, I cannot feel the decrease. But it seems like it drops the RAM after some time.

[craigloewen-msft]

We've released a blog post to help go over the details of this feature: Memory Reclaim in the Windows Subsystem for Linux 2. It should help clear up how memory becomes freed!

[i2]

I created a .wslconfig file in %UserProfile% but it doesn't take affect. I test it by setting the following:

[wsl2] swap=0 processors=3

but when I do 'top' it still shows values from before this change. What am I missing?

[ErvalhouS]

Great, now the issue is just about buff/cache not being released.

[Indribell]

Great, now the issue is just about buff/cache not being released.

Its a issue that is being a bit overlooked. Linux really does not have the habit of freeing memory from it cache/buffer and like to hold everything in memory forever ( only pushing out things that are not accessed in a long time, to make space for new things to cache ). So its a valid point, that is unfortunately somewhat ignored.

I think the expectations of the MS team is: You are done with your task, you close the bash and it auto frees memory. But if you have VSC open and you put your PC to sleep every day ( more productive then reopening everything every day ), it never gets freed. Thus it keep growing as you compile more code or do other file operations. And you are forced to put memory limits on those WSL instances.

By default WSL2 needs to have some kind of a limit or force the Linux kernel to be more aggressive in reclaiming the cache. For example: everything in cache, that is not used in 1 hour time, reclaim it ( a setting you can also override per WSL2 instance ).

/Edit:

And it becomes a issue fast when you work with a lot of microservices / have a lot of WSL2 instances open / like to have clean WSL2 instances by having your data cleanly separated per WSL2 /VM instance.

[benhillis]

@Indribell - we are looking at improving the behavior in the future to improve the behavior regarding Linux's cached memory.

[ErvalhouS]

Wouldn't something like setting up a line at sudo crontab -e with the following do the desired effect?

0 * * * * echo 3 > /proc/sys/vm/drop_caches

I don't know the implication of that in the performance of the machine, but since it's not meant to be used in a production environment I think clearing memory cache every hour seems to be a nice way to workaround that issue.

[benhillis]

@ErvalhouS - The performance impacts are substantial. When you drop the page cache you're going to be reading from disk so it's not something we want to do without being sure you're done with the files that are cached.

[discotroll65]

@benhillis et all, I'm a relative noob to the inner workings of the linux kernel, but might some way of addressing this be making a whitelist of processes whose memory affects can be cleared after they are done? For example, I unzip a .sql.gz file in my WSL instance, and now it permanently has an extra gig of memory tied up to the instance. Would there be an easy way to release from the cache those unzipped files?

[romitzz1]

This might be closer to a Docker issue, but I have been using the edge Docker for Windows with WSL 2. In my case, the `buff/cache' can take 12GB during the building process, and once everything is running it is using 20GB before dropping the cache.
Where as running purely in the old Hyper V version, I had no issues leaving the environment at 12GB max.

[nisarg-ujjainkar]

Can't run the command "sudo sync; echo 3 > /proc/sys/vm/drop_caches". It says permission denied

[peteromio]

Can't run the command "sudo sync; echo 3 > /proc/sys/vm/drop_caches". It says permission denied 1- sudo su 2- sync; echo 3 > /proc/sys/vm/drop_caches

[imaandrew]

So I'm on build 19025 and I still get this problem. Does this still to be fixed or is there something i'm doing wrong?

[jefferai]

Same, came on here because despite having a single docker container running with WSL 2 backend and one VM with two bash prompts vmmem was using 10+ gigs of RAM. Build 19013 (slow ring).

[Congee]

Can't run the command "sudo sync; echo 3 > /proc/sys/vm/drop_caches". It says permission denied

two gotchas here. there's no effect of sudo on echo as echo is a separate statement from sudo sync due to ;. It still does not work if you change ; to && because redirection > happens before sudo gets permission. echo 3 | sudo tee /proc/sys/vm/drop_caches works.

[adam-stamand]

I'm on build 19033 and seeing the same issue with memory consumption. The Vmmem process is currently using 31GB of memory. I'm exclusively running VSCode on WSL2, no containers or the like.

[strayge]

@adam-stamand workaround from https://github.com/microsoft/WSL/issues/4166#issuecomment-526725261 still works like a charm.

[ubaid4j]

Workaround: Create a %UserProfile%\.wslconfig file in Windows and use it to limit memory assigned to WSL2 VM.

Example

[wsl2]
memory=6GB
swap=0
localhostForwarding=true

This will still consume the entire 6GBs regardless of Linux memory usage, but at least it'll stop growing more than that.

Supported settings are documented here.

It worked on Microsoft Windows [Version 10.0.19037.1]

[ajpierce]

EDIT: This is unrelated to memory compaction. The issue was resolved with #4737. Leaving the original below for posterity

Original Post:

Opening a new Ubuntu tab in Windows Terminal (preview) takes an eternity (~40s) for the prompt to appear on [Version 10.0.19037.1]. I ran dmesg and the last thing to occur in the kernel is:

Is it possible that long-running memory compaction is leading to excruciatingly long load times?

Things that I don't think are related to the issue:

• My hardware: NVMe SSD, Intel 8700k, and 32GB of RAM
• My .bashrc: it sources instantly.

The interesting thing is that it used to appear instantly, and then suddenly slowed down. The last thing I remember doing before this started was installing gcc and running make to install VIM 8.1. I have disabled fast boot and rebooted quite a few times to no avail.