search

docker / for-win

Bug reports for Docker Desktop for Windows
https://www.docker.com/products/docker#/windows
1.86k stars 289 forks source link

Windows 10 "C:\Program Files\Docker\Docker\resources\com.docker.backend.exe" handles growth #12916

Closed jrobbins-LiveData closed 1 year ago

jrobbins-LiveData commented 2 years ago

Actual behavior

OS Name Microsoft Windows 10 Pro Version 10.0.19044 Build 19044

According to taskmanager, this process "C:\Program Files\Docker\Docker\resources\com.docker.backend.exe" -addr unix:\\.\pipe\dockerBackendApiServer -native-api is leaking OS handles on my PC at a rate of approximately 50,000 handles a day, despite doing very little with Docker. In fact, the handles count increments even when I am doing nothing with Docker.

Expected behavior

I'd expect the OS handles count to ebb and flow with activity, but reach some sort of stable high water mark.

Information

Output of & "C:\Program Files\Docker\Docker\resources\com.docker.diagnose.exe" check

[2022-08-20T00:17:53.164550600Z][com.docker.diagnose.exe][I] set path configuration to OnHost Starting diagnostics

[PASS] DD0027: is there available disk space on the host? [PASS] DD0028: is there available VM disk space? [PASS] DD0031: does the Docker API work? [PASS] DD0004: is the Docker engine running? [PASS] DD0011: are the LinuxKit services running? [PASS] DD0016: is the LinuxKit VM running? [PASS] DD0001: is the application running? [SKIP] DD0018: does the host support virtualization? [PASS] DD0002: does the bootloader have virtualization enabled? [PASS] DD0017: can a VM be started? [PASS] DD0024: is WSL installed? [PASS] DD0021: is the WSL 2 Windows Feature enabled? [PASS] DD0022: is the Virtual Machine Platform Windows Feature enabled? [PASS] DD0025: are WSL distros installed? [PASS] DD0026: is the WSL LxssManager service running? [PASS] DD0029: is the WSL 2 Linux filesystem corrupt? [PASS] DD0035: is the VM time synchronized? [PASS] DD0015: are the binary symlinks installed? [PASS] DD0003: is the Docker CLI working? [PASS] DD0013: is the $PATH ok? [PASS] DD0005: is the user in the docker-users group? [PASS] DD0007: is the backend responding? [FAIL] DD0014: are the backend processes running? 1 error occurred:

[PASS] DD0008: is the native API responding? [PASS] DD0009: is the vpnkit API responding? [PASS] DD0010: is the Docker API proxy responding? [PASS] DD0006: is the Docker Desktop Service responding? [PASS] DD0012: is the VM networking working? [PASS] DD0032: do Docker networks overlap with host IPs? [SKIP] DD0030: is the image access management authorized? [PASS] DD0033: does the host have Internet access?

Please investigate the following 1 issue:

1 : The test: are the backend processes running? Failed with: 1 error occurred:

Not all of the backend processes are running.

Steps to reproduce the behavior

Simply running "C:\Program Files\Docker\Docker\resources\com.docker.backend.exe" without running any Docker jobs is enough to see the OS handles count grow in taskmanager.

dlmcpaul commented 2 years ago

I have noticed the same issue.

Using https://github.com/randomascii/blogstuff/tree/main/FindZombieHandles you can see lots of zombie processes taking up thousands of handles.

The main System process eventually spends a lot of time trying to process these handles.

mwm-twx commented 2 years ago

I've seen this recently as well. Sysinternals RAMMap shows thousands of wsl.exe processes which each have a 36K page table and ballooned my page table total to more than 5G. wsl.exe --shutdown was unable to cull those wsl.exe processes, but restarting Docker Desktop did clear them out. I restarted last week, so this buildup is happening fairly quickly.

CorrodedCoder commented 2 years ago

Also have just seen com.docker.backend.exe handle leak continuously increasing until Windows 10 operating system performance was impacted, as stated above (count had reached 170,000 handles for that one process). To demonstrate, running no docker containers, running the following powershell command to share handle count: Get-Process com.docker.backend

Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
    245      17    23004      24700       0.86  82804   1 com.docker.backend
   2849      28    32704      41416      40.02  89552   1 com.docker.backend

The second process handle count just keeps on going up and up, it's only that low at the moment because I restarted the process. As mentioned elsewhere, using Process Explorer to view handles, I see thousands of: handles.

hernandp commented 2 years ago

I confirm this. Analyzed using WinDbg memory dump. Over 40K Handles on my system, causes NTOSKRNL!KeBalanceSetManager to be called repeatedly due to memory pressure, system performance impacted.

Twitted over this https://twitter.com/indiocolifa/status/1568215828822196225

image

nicks commented 2 years ago

does the problem persist if you upgrade to Docker Desktop v4.12? This sounds a lot like - https://github.com/docker/for-win/issues/12877

CorrodedCoder commented 2 years ago

Will give it a try - apologies I didn't realise there was an update available. Only been running for 10 minutes, but I can't see any zombies - so I think you were right in my case @nicks - thanks for spotting that!

dlmcpaul commented 2 years ago

Yes it looks like the docker backend is no longer leaking handles with 4.12

docker-robott commented 1 year ago

There hasn't been any activity on this issue for a long time. If the problem is still relevant, mark the issue as fresh with a /remove-lifecycle stale comment. If not, this issue will be closed in 30 days.

Prevent issues from auto-closing with a /lifecycle frozen comment.

/lifecycle stale

docker-robott commented 1 year ago

Closed issues are locked after 30 days of inactivity. This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

/lifecycle locked