docker / for-win

Bug reports for Docker Desktop for Windows
https://www.docker.com/products/docker#/windows
1.86k stars 289 forks source link

Windows 10 Docker processes consuming high CPU with no running containers #1772

Closed mlmeyers closed 5 years ago

mlmeyers commented 6 years ago

Expected behavior

When stopping all containers I would expect CPU usage to drop to normal levels.

Actual behavior

Almost no change to CPU after stopping containers, further after restarting docker it continues to consume a large percentage of CPU. Consumed by both com.docker.service.exe and Docker for Windows.exe

Information

A diagnostic was uploaded with id: 4898CA71-860C-4EE7-85B8-E309B53354D8/2018-03-01_13-17-04

Windows Build 16299.248

This is snapshot of my cpu consumption with no running containers. Looking at process explorer it appears the cpu is taken up by threads in clr.dll!DllCanUnloadNowInternal

Taskmanager

image

Docker for windows.exe procexp

image

com.docker.service.exe procexp

image

Steps to reproduce the behavior

  1. Start docker, start windows containers that do stuff (consume cpu) stop containers.
mlmeyers commented 6 years ago

This seems related to the com.docker.service.exe, so far when running just dockerd via cmdline I am unable to reproduce the high cpu issues. In fact when running just the daemon the container hardly puts a dent in my cpu usage. In numerous runs using the docker for windows client the cpu issue is reproduced on my machine. Must say the windows 10 install feels bloated. Wish it was more aligned with Server 2016 setup. The introduction of the com.docker.service and the behind the scenes work to recreate the daemon service etc based on the client tool seems to muddy the waters a lot in comparison to a simple start of the daemon and a config file. Still not clear on the cause for the high cpu but suspect some threading issue trying to unload dll's still in use based on the active threads in procexp.

mlmeyers commented 6 years ago

Possible to get eyes on this I continue to struggle with Docker for Windows service consuming over half of my CPU when running. Always the cycles are being consumed by clr.dll!DllCanUnloadNowInternal

guba-odudkin commented 6 years ago

Same issue here. Started after I updated to the latest version. Should I rollback? Otherwise it's not usable

SonicGD commented 6 years ago

Same issue with 18.03 and windows skip ahead build.

image

smasherprog commented 6 years ago

Same issue here on windows 10 1709

cocowalla commented 6 years ago

I've got this on Edge build 18.05 too

rapatel0 commented 6 years ago

Same Issue here image

Reproduced after uninstall and reinstall Version 18.03.1-ce-win65 (17513) Channel: stable 93354b3

liquidboy commented 6 years ago

got this on 17711

amorev commented 6 years ago

got this on 19098

rverhe commented 6 years ago

Same here

ddobric commented 6 years ago

Anything new here? Service is taking as much RAM as possible. It is not usable.

guba-odudkin commented 6 years ago

Stopped using docker

ddobric commented 6 years ago

@guba-odudkin this is radical decision. But I can understand,

industrial64 commented 6 years ago

Same on 19098 - No resolution yet. High use with or without running containers. Seen this only on Win10.

piyh commented 6 years ago

Getting this as well. Windows 10, Version: 18.06.1-ce

DealWithMeal-zz commented 6 years ago

Same stuff here. A fix?

industrial64 commented 6 years ago

My fix was to increase memory due to Docker for Win not allocating enough to run kubernetes and a single deb container. Just increase mem (mine from 2.2GB to 2.8GB) reload, and everything should go back to normal CPU-wise.

piyh commented 6 years ago

There aren't any containers running, it's only docker. Like rapatel0's screenshot.

industrial64 commented 6 years ago

Yup, same fix applies. I had nothing running, increasing memory reserves fixed the CPU issue (w/ and w/o containers). Just give it a whirl.

dannyk81 commented 6 years ago

Having a similar issue with high cpu usage, no containers running and Kubernetes feature is disabled.

My stack:

Docker Version 2.0.0.0-beta1-win75 (19925)
Windows 10 Insider Preview 17758.1 (rs5_release)

Kubernetes is disabled and I increased RAM allocation to 3GB.

In my case, the issue seem worse when connecting to VPN, I start seeing a lot of these messages and CPU usage spikes:

[11:16:12.447][DnsUpdater     ][Info   ] Network configuration change detected
[11:16:12.447][PowerShell     ][Info   ] Run script '$(Find-NetRoute -RemoteIPAddress 8.8.8.8).InterfaceIndex[0]'...
[11:16:12.775][DnsUpdater     ][Info   ] Network configuration change detected
[11:16:12.775][PowerShell     ][Info   ] Run script '$(Find-NetRoute -RemoteIPAddress 8.8.8.8).InterfaceIndex[0]'...
[11:16:13.098][DnsUpdater     ][Info   ] Network configuration change detected
[11:16:13.098][PowerShell     ][Info   ] Run script '$(Find-NetRoute -RemoteIPAddress 8.8.8.8).InterfaceIndex[0]'...
[11:16:13.459][DnsUpdater     ][Info   ] Network configuration change detected
[11:16:13.459][PowerShell     ][Info   ] Run script '$(Find-NetRoute -RemoteIPAddress 8.8.8.8).InterfaceIndex[0]'...
[11:16:13.781][DnsUpdater     ][Info   ] Network configuration change detected
[11:16:13.782][PowerShell     ][Info   ] Run script '$(Find-NetRoute -RemoteIPAddress 8.8.8.8).InterfaceIndex[0]'...
[11:16:14.100][DnsUpdater     ][Info   ] Network configuration change detected
[11:16:14.100][PowerShell     ][Info   ] Run script '$(Find-NetRoute -RemoteIPAddress 8.8.8.8).InterfaceIndex[0]'...
[11:16:14.446][DnsUpdater     ][Info   ] Network configuration change detected
[11:16:14.446][PowerShell     ][Info   ] Run script '$(Find-NetRoute -RemoteIPAddress 8.8.8.8).InterfaceIndex[0]'...
[11:16:14.772][DnsUpdater     ][Info   ] Network configuration change detected
[11:16:14.772][PowerShell     ][Info   ] Run script '$(Find-NetRoute -RemoteIPAddress 8.8.8.8).InterfaceIndex[0]'...
[11:16:15.087][DnsUpdater     ][Info   ] Network configuration change detected
[11:16:15.087][PowerShell     ][Info   ] Run script '$(Find-NetRoute -RemoteIPAddress 8.8.8.8).InterfaceIndex[0]'...
wclr commented 6 years ago

Advice: try to restart Docker for Windows Service in services.msc, then restart docker for windows, in Settings/Reset.

dannyk81 commented 6 years ago

thanks @whitecolor, but I tried restart, reseting, reinstalling, everything... nothing helps 😟 d4w became totally unusable for me 😔

wclr commented 6 years ago

I see such pattern: as I rarely tend to restart windows (at least for days) - sometimes (after a certain os restart) docker works fine (without consuming CPU and mem) for a long time, but sometimes after each few hours of work (or idle) it starts consuming CPU/mem - then service restart trick helps (after some hours it will again start consuming CPU/mem).

Also with such (service restart) workflow I need to restart docker (in Settings window) by unchecking and checking again Exspose deamon setting - without it localhost:2375 is not available after the restart. =) Quite cumbersome, but it still tends to work, though I'm still currently on Version 17.09.1-ce-win42 (14687)

dannyk81 commented 6 years ago

thanks again @whitecolor, but the restart tricks are not working for me 😢

Windows just got updated to build 18252, however the issue still persists and docker logs now shows constantly repeating:

[13:11:53.398][DnsUpdater     ][Info   ] Network configuration change detected
[13:11:53.399][DnsUpdater     ][Error  ] Unable to update dns settings: Value cannot be null.
Parameter name: first

This is so frustrating...

Diagnostics ID: 5459E06B-6398-479F-B0E7-08E317758B02/20181004134710

/cc @djs55

antogh commented 6 years ago

Same problem here , docker service eating 15% cpu for doing nothing , all containers are stopped. My fix is to quit docker and start it again when I need it .

mts57017 commented 6 years ago

same issue here. Killing docker and using when i need too.

FrankSzendzielarz commented 6 years ago

Likewise started just this evening. Nothing changed AFAICT

ma1f commented 5 years ago

uninstall/reinstall with linux containers instead of windows containers - then once running switch to windows containers. Fixed the issue for me.

dannyk81 commented 5 years ago

@ma1f good suggestion, for some reason I didn't consider LCOW as an option, this indeed solved the CPU issue and seems to be good for my use cases, not sure what limitations there might be, but for general purpose linux development stuff looks ok so far.

ma1f commented 5 years ago

@dannyk81 it actually seems to set it up better if anything when installed as linux containers - eg. the hyper-v virtual network properly creates the DockerNAT, when installing as windows it doesn't... Once running i've switched to windows containers from the menu and had no issues / no high CPU usage and seems to work great.

DealWithMeal-zz commented 5 years ago

Reinstalling with linux containers and switching to windows containers didn't solve the issue for me :(

dannyk81 commented 5 years ago

@ma1f this looked like a promising solution and in my case the high CPU usage issue is gone after switching to LCOW, but now I'm hitting another issue 😞

Looks like DNS resolution is broken... I have a dns resolver running on my host (192.168.2.13), it is picked up correctly in /etc/resolv.conf of the container:

/ # cat /etc/resolv.conf
nameserver 172.29.144.1
nameserver 192.168.2.13

But for some reason I just can't access it from the containers.

from Windows:

C:\WINDOWS\system32>nslookup google.com
Server:  UnKnown
Address:  192.168.2.13

Non-authoritative answer:
Name:    google.com
Address:  172.217.9.46

From a container:

/ # dig @192.168.2.13 google.com

; <<>> DiG 9.10.2 <<>> @192.168.2.13 google.com
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

I can query external dns servers (like 8.8.8.8 or 1.1.1.1):

/ # dig @8.8.8.8 google.co

; <<>> DiG 9.10.2 <<>> @8.8.8.8 google.co
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 53964
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;google.co.                     IN      A

;; ANSWER SECTION:
google.co.              299     IN      A       172.217.2.174

;; Query time: 38 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Tue Oct 23 16:44:37 UTC 2018
;; MSG SIZE  rcvd: 54

But not the one running on the host itself... this is so frustrating...

cocowalla commented 5 years ago

@ma1f @dannyk81 a good reason LCOW hasn't worked for me at least, is that LCOW has some serious limitations that mean a lot of Linux containers don't work. Specifically, chmod, chown and chattr don't work, and these fairly fundamental tools are essential for a lot of containers. See here for more info.

dannyk81 commented 5 years ago

@cocowalla good points!

well... I'm out of ideas, short of installing Linux 😉

Considering the amount of people reporting on this issue here and on https://github.com/docker/for-win/issues/1900, I would expect someone to take a look at this...

ma1f commented 5 years ago

@cocowalla interesting limitations on LCOW, haven't had issues yet in the way i've been using them but useful to know, i'm using both linux and windows but key thing is that reinstalling without ticking the use Windows Containers resolves the CPU issue. @DealWithMeal for me I only realised this was the issue as none of my work colleagues were having issues but none of them had selected the use Windows Containers checkbox during installation - perhaps make sure you have done a full uninstall - https://success.docker.com/article/how-to-completely-remove-docker-in-windows-10

Dannymx commented 5 years ago

Just came here to report the same thing, high cpu usage, I tried all the suggestions mentioned in this thread but nothing fixed it. Running Version 18.06.1-ce-win73 (19507) Channel: stable.

dannyk81 commented 5 years ago

for anyone interested, I "solved" my dns issue by adding an additional IP to the nat network interface and configured my local resolver to listen on that address.

I then added this IP to the Daemon configuration file:

  "dns": [
    "172.17.160.2"
  ]

Where 172.17.160.2 is the additional IP I added to to the nat Interface (and is where my local resolver was configured to listen).

On a personal note, Windows is becoming a PITA... having to deal with all these issues and obscure configuration stuff...

clarity99 commented 5 years ago

uninstalled/reinstall without checkbox for windows containers, didn't help. still using 15% cpu constantly

mikeparker commented 5 years ago

I'm struggling to reproduce this. If anyone could get specific reproduction instructions that'd be great, including which container(s) you're running and what settings you have for your VM (ram, CPUs etc).

The original post said that it only occurs when running containers then stopping them (it doesn't occur purely on boot), if people could confirm that too that'd be great.

dannyk81 commented 5 years ago

@mikeparker, in my case:

1) Any of the Windows 10 builds from last ~6 months (Stable or Insider) 2) Install d4w (latest stable or edge) with default options 3) Start it and observe cpu usage of the com.docker.service.exe process (should be constant at about 10-15%)

NOTE: As I've mentioned above (https://github.com/docker/for-win/issues/1772#issuecomment-421586273), at least in my case this issue seem to somehow correlate with the fact that I have several VPNs active (connected to various sites as I'm a remote worker).

I use latest OpenVPN with tunXten UI (to be able to connect to multiple VPNs)

mikeparker commented 5 years ago

@dannyk81 Is it possible you can completely disable your VPNs and try? For me the above instructions with no VPN active, all docker processes are at 0% CPU so I can't reproduce.

If it goes to 0 completely for you, it sounds networking related and I can look into reproducing with VPNs, although to reproduce I'm guessing I may need to go through many iterations of trying to set up a VPN to try to get a similar setup so it'd be great to confirm it's definitely related to VPNs before i go down that path!

mts57017 commented 5 years ago

same issue here, tried to uninstall and re-setup as mentioned above. Same result. CPU is at a constant 10-20% just from docker process.

DealWithMeal-zz commented 5 years ago

10-20% ? I mean.. I wouldn't even think that something is wrong in that case.

I have 8th gen I7 CPU and docker starts to consume 70-90% of it once I start it and do whatever! For example, start and stop any kind of container. In both windows and linux containers, using kubernetes and not.

No VPNs are active. The only clue I have is McAfee, which is known to ruin everything. Cannot disable it because of Admin policy or something (process is immortal).

mts57017 commented 5 years ago

it is 10-20% with no containers running at all. Basically a fresh install just hanging around. That should not cause a constant usage spike that high, considering it didn't use too. If it does, i have an 8th Gen I9, but i would like to know why 10-20% of cpu is being allocated 24/7 to something that should be in a sense "twiddling its thumbs"

dannyk81 commented 5 years ago

@mikeparker thanks for looking into this, sorry for the delay, I'm away for few days but can run the test once I get back.

From past experience, if Docker started after a clean boot and no VPN tunnels established, the CPU usage of the docker service was negligible/close to zero.

However, I would like to make a test once I'm back to verify this, will keep you posted.

ma1f commented 5 years ago

@mikeparker one thing I did notice is that if installed with windows containers checkbox selected, no nat network for hyper-v was created, whereas if installed with linux containers (default) then it is created fine.

clarity99 commented 5 years ago

Hi, I don't have any active VPN tunnels and no running docker containers, yet I am experiencing this. It does not matter if I install with linux or windows containers at the beginning.

PS C:\users\roberti> docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

I've managed to get the callstack of the thread using the CPU: ntdll.dll!NtOpenKeyEx+0x14 KERNELBASE.dll!LocalBaseRegOpenKey+0x1ce KERNELBASE.dll!RegOpenKeyExInternalA+0x12e KERNELBASE.dll!RegOpenKeyExA+0x19 IPHLPAPI.DLL!AddDnsServerAddressesToAdapter+0x276 IPHLPAPI.DLL!AddDnsConfiguration+0x4ec IPHLPAPI.DLL!AllocateAndGetAdaptersAddresses+0x82b IPHLPAPI.DLL!GetAdaptersAddresses+0x4f DNSAPI.dll!IpHelp_GetAdaptersAddresses+0xb3 DNSAPI.dll!NetInfo_Build+0xe5 DNSAPI.dll!NetInfo_Get+0x2e0 DNSAPI.dll!Config_GetDnsServerList+0x52 DNSAPI.dll!Config_GetDnsServerListIp4+0x3b DNSAPI.dll!DnsQueryConfig+0x229f8 IPHLPAPI.DLL!AllocateAndGetPerAdapterInfo+0xf8 IPHLPAPI.DLL!GetPerAdapterInfo+0x30 [Native Frame: IL Method without Metadata] [Managed to Unmanaged Transition] System.dll!System.Net.NetworkInformation.SystemIPv4InterfaceProperties.GetPerAdapterInfo+0x66 System.dll!System.Net.NetworkInformation.SystemIPInterfaceProperties..ctor+0x207 System.dll!System.Net.NetworkInformation.SystemNetworkInterface..ctor+0xc7 System.dll!System.Net.NetworkInformation.SystemNetworkInterface.GetNetworkInterfaces+0x191 System.dll!System.Net.NetworkInformation.NetworkInterface.GetAllNetworkInterfaces+0x46 Docker.Backend.dll!Docker.Backend.NetworkInterfaceHelper.GetHostInterfaceByIp+0x3f Docker.Backend.dll!Docker.Backend.NetworkInterfaceHelper.TryEnableSambaShareOnInterface+0x27 [Unmanaged to Managed Transition] clr.dll!CallDescrWorkerInternal+0x83 clr.dll!CallDescrWorkerWithHandler+0x4e clr.dll!CallDescrWorkerReflectionWrapper+0x1a clr.dll!RuntimeMethodHandle::InvokeMethod+0x45f [Managed to Unmanaged Transition] mscorlib.dll!System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal+0x80 mscorlib.dll!System.Delegate.DynamicInvokeImpl+0x99 System.Management.dll!System.Management.WmiDelegateInvoker.FireEventToDelegates+0x97 System.Management.dll!System.Management.ManagementEventWatcher.FireEventArrived+0x28 System.Management.dll!System.Management.SinkForEventQuery.Indicate+0x8d [Unmanaged to Managed Transition] [Native Frame: IL Method without Metadata] clr.dll!COMToCLRDispatchHelper+0x39 clr.dll!COMToCLRWorker+0x1b4 clr.dll!GenericComCallStub+0x57 wminet_utils.dll!CWmiSinkDemultiplexor::InternalWbemObjectSink::Indicate+0x56 fastprox.dll!CSinkStubBuffer::XSinkStublet::Indicate_Stub+0xf7 combase.dll!ObjectMethodExceptionHandlingAction<<lambda_76d9e92c799d246a4afbe64a2bf5673d> >+0x53 combase.dll!DefaultStubInvoke+0x295 combase.dll!ServerCall::ContextInvoke+0x496 combase.dll!AppInvoke+0x328 combase.dll!ComInvokeWithLockAndIPID+0x59c combase.dll!ThreadInvoke+0x1320 RPCRT4.dll!DispatchToStubInCNoAvrf+0x24 RPCRT4.dll!RPC_INTERFACE::DispatchToStubWorker+0x1d8 RPCRT4.dll!RPC_INTERFACE::DispatchToStubWithObject+0x154 RPCRT4.dll!LRPC_SCALL::DispatchRequest+0x18d RPCRT4.dll!LRPC_SCALL::HandleRequest+0x86c RPCRT4.dll!LRPC_ADDRESS::HandleRequest+0x33d RPCRT4.dll!LRPC_ADDRESS::ProcessIO+0x8ad RPCRT4.dll!LrpcIoComplete+0xd8 ntdll.dll!TppAlpcpExecuteCallback+0x22e ntdll.dll!TppWorkerThread+0x258 KERNEL32.dll!BaseThreadInitThunk+0x14 ntdll.dll!RtlUserThreadStart+0x21

so it seems it does have something to do with network interfaces. ipconfig gives the following output on my system: ` PS C:\users\roberti> ipconfig /all

Windows IP Configuration

Host Name . . . . . . . . . . . . : WINDHORSE Primary Dns Suffix . . . . . . . : CREA.SI Node Type . . . . . . . . . . . . : Hybrid IP Routing Enabled. . . . . . . . : No WINS Proxy Enabled. . . . . . . . : No DNS Suffix Search List. . . . . . : CREA.SI

Ethernet adapter Local Area Connection 4 2:

Connection-specific DNS Suffix . : Description . . . . . . . . . . . : SonicWALL Virtual NIC Physical Address. . . . . . . . . : 00-60-73-E0-00-00 DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes Link-local IPv6 Address . . . . . : fe80::58d7:9c92:3167:f3e9%2(Preferred) Default Gateway . . . . . . . . . : DHCPv6 IAID . . . . . . . . . . . : 956326003 DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1C-27-32-D2-D0-50-99-35-4D-23 DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1 fec0:0:0:ffff::2%1 fec0:0:0:ffff::3%1 NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter Ethernet 2:

Media State . . . . . . . . . . . : Media disconnected Connection-specific DNS Suffix . : Description . . . . . . . . . . . : Fortinet SSL VPN Virtual Ethernet Adapter Physical Address. . . . . . . . . : 00-09-0F-AA-00-01 DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes

Ethernet adapter vEthernet (DockerNAT):

Connection-specific DNS Suffix . : Description . . . . . . . . . . . : Hyper-V Virtual Ethernet Adapter #2 Physical Address. . . . . . . . . : 00-15-5D-00-8D-04 DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes IPv4 Address. . . . . . . . . . . : 10.0.75.1(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.255.0 Default Gateway . . . . . . . . . : DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1 fec0:0:0:ffff::2%1 fec0:0:0:ffff::3%1 NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . : CREA.SI Description . . . . . . . . . . . : Intel(R) Ethernet Connection (2) I218-V Physical Address. . . . . . . . . : D0-50-99-35-4D-23 DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes Link-local IPv6 Address . . . . . : fe80::8902:f47f:611e:e96e%27(Preferred) IPv4 Address. . . . . . . . . . . : 192.168.0.141(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.248.0 Lease Obtained. . . . . . . . . . : 2. november 2018 10:21:45 Lease Expires . . . . . . . . . . : 9. november 2018 5:33:56 Default Gateway . . . . . . . . . : 192.168.0.1 DHCP Server . . . . . . . . . . . : 192.168.0.64 DHCPv6 IAID . . . . . . . . . . . : 248533145 DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1C-27-32-D2-D0-50-99-35-4D-23 DNS Servers . . . . . . . . . . . : 192.168.0.61 192.168.0.62 Primary WINS Server . . . . . . . : 192.168.0.62 Secondary WINS Server . . . . . . : 192.168.0.61 NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter VirtualBox Host-Only Network:

Connection-specific DNS Suffix . : Description . . . . . . . . . . . : VirtualBox Host-Only Ethernet Adapter Physical Address. . . . . . . . . : 0A-00-27-00-00-04 DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes Link-local IPv6 Address . . . . . : fe80::e437:95f2:11a1:1f4b%4(Preferred) IPv4 Address. . . . . . . . . . . : 192.168.56.1(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.255.0 Default Gateway . . . . . . . . . : DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1 fec0:0:0:ffff::2%1 fec0:0:0:ffff::3%1 NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter VirtualBox Host-Only Network #2:

Connection-specific DNS Suffix . : Description . . . . . . . . . . . : VirtualBox Host-Only Ethernet Adapter #2 Physical Address. . . . . . . . . : 0A-00-27-00-00-1E DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes Link-local IPv6 Address . . . . . : fe80::b8d0:3a50:c3d:38bc%30(Preferred) IPv4 Address. . . . . . . . . . . : 192.168.99.1(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.255.0 Default Gateway . . . . . . . . . : DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1 fec0:0:0:ffff::2%1 fec0:0:0:ffff::3%1 NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter VMware Network Adapter VMnet1:

Connection-specific DNS Suffix . : Description . . . . . . . . . . . : VMware Virtual Ethernet Adapter for VMnet1 Physical Address. . . . . . . . . : 00-50-56-C0-00-01 DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes Link-local IPv6 Address . . . . . : fe80::99a2:f529:ffeb:d6fe%22(Preferred) IPv4 Address. . . . . . . . . . . : 192.168.241.1(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.255.0 Default Gateway . . . . . . . . . : DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1 fec0:0:0:ffff::2%1 fec0:0:0:ffff::3%1 NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter VMware Network Adapter VMnet8:

Connection-specific DNS Suffix . : Description . . . . . . . . . . . : VMware Virtual Ethernet Adapter for VMnet8 Physical Address. . . . . . . . . : 00-50-56-C0-00-08 DHCP Enabled. . . . . . . . . . . : No Autoconfiguration Enabled . . . . : Yes Link-local IPv6 Address . . . . . : fe80::bc15:19de:a31a:aa1c%12(Preferred) IPv4 Address. . . . . . . . . . . : 192.168.137.1(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.255.0 Default Gateway . . . . . . . . . : DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1 fec0:0:0:ffff::2%1 fec0:0:0:ffff::3%1 NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter Local Area Connection 3:

Media State . . . . . . . . . . . : Media disconnected Connection-specific DNS Suffix . : Description . . . . . . . . . . . : TAP-Windows Adapter V9 Physical Address. . . . . . . . . : 00-FF-C5-CC-F0-1D DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes

Ethernet adapter Ethernet:

Media State . . . . . . . . . . . : Media disconnected Connection-specific DNS Suffix . : Description . . . . . . . . . . . : TeamViewer VPN Adapter Physical Address. . . . . . . . . : 00-FF-66-F3-7B-CB DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes

Ethernet adapter Ethernet 3:

Media State . . . . . . . . . . . : Media disconnected Connection-specific DNS Suffix . : Description . . . . . . . . . . . : Fortinet virtual adapter #2 Physical Address. . . . . . . . . : 00-09-0F-FE-00-01 DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes

Ethernet adapter vEthernet (Default Switch):

Connection-specific DNS Suffix . : Description . . . . . . . . . . . : Hyper-V Virtual Ethernet Adapter Physical Address. . . . . . . . . : F6-15-1E-2C-6D-98 DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes Link-local IPv6 Address . . . . . : fe80::c150:dd41:c19:686c%7(Preferred) IPv4 Address. . . . . . . . . . . : 172.23.152.129(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.255.240 Default Gateway . . . . . . . . . : DHCPv6 IAID . . . . . . . . . . . : 1644172637 DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1C-27-32-D2-D0-50-99-35-4D-23 DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1 fec0:0:0:ffff::2%1 fec0:0:0:ffff::3%1 NetBIOS over Tcpip. . . . . . . . : Disabled

Ethernet adapter vEthernet (nat):

Connection-specific DNS Suffix . : Description . . . . . . . . . . . : Hyper-V Virtual Ethernet Adapter #3 Physical Address. . . . . . . . . : 00-15-5D-84-F0-73 DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes Link-local IPv6 Address . . . . . : fe80::9491:e6fe:775b:55d2%18(Preferred) IPv4 Address. . . . . . . . . . . : 172.26.32.1(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.240.0 Default Gateway . . . . . . . . . : DHCPv6 IAID . . . . . . . . . . . : 1946162525 DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1C-27-32-D2-D0-50-99-35-4D-23 DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1 fec0:0:0:ffff::2%1 fec0:0:0:ffff::3%1 NetBIOS over Tcpip. . . . . . . . : Enabled PS C:\users\roberti>`

hope this helps. let me know if I can give some more information

dannyk81 commented 5 years ago

@mikeparker again, sorry for the delay.

I did a fresh install for D4W and restarted Windows, VPNs are not connected and I still see ~8-10% cpu usage on the Docker service.

I then connected to my VPNs and noticed that CPU usage increased slightly to ~10-15% (also the messages I mentioned here: https://github.com/docker/for-win/issues/1772#issuecomment-421586273 started flooding the log file), I'm not sure if there's any clear correlation here but looks like it does make the situation worse, though the issue is still there when VPNs are not connected.

This seems to correlate with reports from many other users here on this thread and over at https://github.com/docker/for-win/issues/1900

Switched back to Windows Containers and CPU usage is now under control, for now this is an acceptable workaround for me, although I do hit the limitations of LCOW from time to time, which is a bit frustrating.

mikeparker commented 5 years ago

After some investigation I have a few ideas that I'm looking into. I think it's possible this is because we are listening for changes to the network adapter configurations and refreshing the DNS, which causes a fair amount of registry hits which appear to be slow. I found a spike in the performance profile in this area and that would correlate with reports about VPNs and other related reports above. There may be a way we can either listen for more specific network adapter changes or optimise the way we refresh the DNS.

No progress on a fix yet but I'll keep in touch.

jedidja commented 5 years ago

Glad to hear there's some progress on this :) I've experienced the same problem as @mts57017 on two machines.