Closed stephenc closed 4 years ago
FYI Was also happening on stable and I upgraded to edge to see if if made a difference... (Ron Howard voice: "It didn't make a difference")
SAME PROBLEM HERE!!!!
Same problem here. Tried solving it with solutions mentioned in #1759 like decreasing the CPU count and increasing but that didn't help.
Same, also #1759, which this is probably a duplicate of.
@jammi so I had been experiencing #1759 but that seemed to go away around 18.06
This issue only appeared since the new year. And the symptoms sound different. CPU usage is fine until :boom: and then the Docker VM is unresponsive to any control. I need to force kill all the Docker processes.
Another strange datapoint. In an effort to see what might be going wrong, I kept a window open on the Docker VM’s TTY and had that running top
and as long as the window is open, no reproduction (“Heisenbug!”)
@stephenc Ok, this sounds like the issues I've been having since Docker Desktop 2.0; November-December. I can just leave it idle and it'll still go crazy high on CPU and not respond to anything. Killing and restarting it does not help, it goes right back up. "Reset to factory defaults" helps for an hour to a day or so, and then it goes on again. It's a huge PITA and made Docker Desktop essentially useless, so I've resorted to running VirtualBox for any local Docker and Kubernetes operations, which is far from ideal and seriously hurts productivity.
I'm having a similar issue as well. I've had to resort to running all my services natively on a mac mini instead of docker, I can't even run kube because idle without anything running it consumes like 50+% on all my cores making my laptop really sluggish. Now I need to buy a Linux box to run kube on so I can test stuff :( spinning it up frequently just for tests and having to shut it down after is becoming tedious.
SAME PROBLEM HERE!!!!
The same issue with 2.0.0.2 on Mac-mini (macOS Mojave 10.14.3).
Same here Docker desktop Version 2.0.0.3 (31259) Engine: 18.09.2 Compose: 1.23.2 Machine: 0.16.1
What is the workaround for running docker on a mac until this issue gets ack'd or addressed?
Same problem here. Is there any workaround to solve this issue? Thanks.
Upper in this topic somebody suggested to use docker-sync on macos. For me this worked. But it have some bugs, so i'd like to see native solution from docker team
I'm basically back to Virtualbox and docker-machine configurations on it. Since it's not exactly a lightweight configuration either, I'll probably have to invest in some portable Linux cluster to develop locally on.
The same Here Using Docker 2.0.0.3 Mac OS 10.14.3
I think we all know that docker is the real cause of global warming. 😛
Same problem here using Docker 2.0.0.3 Mac OS 10.14.3
Same issue.
Mac OS 10.13.6 Docker Desktop 2.0.0.3.
Eats 100% CPU with no containers running, and after quitting Docker Desktop com.desktop.hyperkit must be force quit. Did reset to factory defaults.
Same issue here
Still happening for me... though it hasn't struck for a while, its back with vengeance today!
(Mostly hasn't struck in a while because I haven't been starting, running and stopping containers)
Same issue...
Happening on 10.14.4 too.
Docker Desktop 2.0.0.3
Same issue: i tried to reset to factory defaults and reset the disk image. It does not help. CPU of com.docker.hyperkit goes to 200% even before i started any containers. Restarting the computer temporary fix the issue!
macOS 10.14.14 Filesystem: APFS (Encrypted) Docker desktop: 2.0.0.3 (31259)
The same issue on Edge version Os: Mojave 10.14.4 Filesystem: APFS (UnEncrypted) Docker: 2.0.3.0 (31778) Edge
Docker Desktop is almost unusable now 8(
Same issue, do you have a solution?
Running into this as well Mojave 10.14.13 APFS (Unencrypted) Docker 2.0.0.3 (31259)
In my case I was doing a useradd
... found adding -l
solved my issue by way of reading through this issue
Same here - any update?
EDIT: Below did not actually resolve the issue. After about 1 hour, CPU started to spike again
I had the same issue which now seems to be resolved:
I first tried the maxfiles fix noted here: https://github.com/docker/for-mac/issues/1759 That had no change on my system.
I then adjusted my memory to 6G and my swap down to 512MB and the issue is gone.
MBP i7 OSX 10.14.3 Docker Desktop 2.0.0.3 Engine 18.09.2
Same issue Version 2.0.0.3 (31259) Engine: 18.09.2
Same issue Version 2.0.0.3 (31259) Engine: 18.09.2
Same problem! Docker cli does not responding at all...
MacOS Version 2.0.0.3 (31259) Engine: 18.09.2
Same here
MacOS 10.14.4 Docker Version 2.0.0.3 (31259) Engine: 18.09.2
I also had this problem (2.0.0.3 (31259), engine 18.09.2). Following the steps described here https://markshust.com/2018/01/30/performance-tuning-docker-mac/ seemed to remove the issue for me. CPU load now looking more like what you would expect (i.e. matching container behaviour). Can't say which step specifically helped.
Thank you @peterataylor . Seem it works!
I haven't seen my CPU at 100%, but I have seen it constantly around 80% with no running containers. I then noticed I had the local Kubernetes cluster enabled. I clicked to disable that, and now hyperkit is maxing around 3% sometimes with nothing running; this is the only thing I've done to date. With the local K8s cluster running high idle CPU, and without, low; only change I made; hope it helps someone else.
Same HEre .. Making my Mac CPU Burning 200% 24/7
I am on 600% !!!!!!
Data point:
I read some other user's comments and I checked to see and my Docker was not using the .raw
file format.
I deleted by docker disk and the recreated one is .raw
, see screenshot:
Since then I have not observed the CPU space-heater behaviour... however this could just be luck... I had it all good before until it wasn't.
I encourage everyone encountering this issue to check the Disk Image Location and let's see how many encountering the issue have .raw
and how many have .qcow2
... I'm going to run a poll in my next comment
POLL If you are having this issue react to this comment with:
😕 if your docker image is .raw
👀 if your docker image is .qcow2
(basically anything that isn't .raw
)
(note you are allowed to react to both, e.g. if you have experienced the CPU space heater with both image types then react with both emoji)
I had same issue and fix it by:
Stop Docker Just delete Docker.raw image file. Restart Docker
I tried some of the fixes mentioned here. Deleting the image file or increasing available resources didn't seem to help, though limiting the number of CPUs to 1 seems to have worked.
A docker system prune --volumes
helped in my case. I suspect exited containers with volumes mapped to (Docker) host directories, and then through to (Mac) VM host.
I'm guessing there are numerous reasons why Docker might start using high CPU. I'll add one more possibility, with the hope it might help someone else.
In my case com.docker.hyperkit was constantly using around 50% cpu (quad-core i7 MBP) even with no running containers. I traced it to "Visual Studio Code — Insiders" which I have recently been trialling, as it has Docker integration (can edit code stored inside a container). Unfortunately, when you select the docker sidebar in VSCode, it starts polling the docker binary every second for info. Worse, it doesn't stop when you navigate away from the docker sidebar. Worse still, if you do this in multiple VSCode windows, you will end up with multiple of these polling requests happening every second. I'm seeing six/sec at the moment.
If this is your issue, you will see something like this in your macOS Console app:
default 14:13:27.711141 +1200 com.docker.driver.amd64-linux proxy >> GET /containers/json?filters=%7B%22status%22%3A%5B%22created%22%2C%22restarting%22%2C%22running%22%2C%22paused%22%2C%22exited%22%2C%22dead%22%5D%7D
default 14:13:27.711230 +1200 com.docker.driver.amd64-linux proxy >> GET /images/json?filters=%7B%22dangling%22%3A%5B%22false%22%5D%7D
Quitting VSCode immediately stops Docker's high CPU usage. You can restart VSCode and use it just fine so long as you don't visit the docker sidebar at all. Which kind of defeats the purpose of their integration, but that's a separate issue.
Current Docker, MacOS Mojave 10.14.5. Same problem.
I can confirm the same problem on MacOS Mojave 10.14.5. CPU utilization sometimes 200+ %. I have to kill the process manually every time, otherwise it slows down my computer considerably.
Same problem for me on MacOS Mojave 10.14.5 and up-to-date stable Docker. CPU utilization 200% and more when simply idling. Quitting Docker does not change it. I have to kill the process manually in Activity Monitor.
I had the same issue. Disabling kubernetes
removed the problem.
Had this issue as well, tried delete the .raw disk image file but that didn't help. Stopping the Kubernetes service is what did the trick for me.
How do you disable Kubernetes?
And is it enabled by default when you install Docker, or do you enable it manually first?
Also, does our help to reduce the set of directories on the host that Docker is allowed to access?
Summary
This is an issue with a lot of comments and rumors of fixes and confusion over what the exact issue underlying this bug is.
False leads
Here is what we know, because it can get lost in the comments.
Current hypothesis
There is an edge case in the filesystem synchronization code and when triggered it causes the receiver process in the Docker VM to go into an infinite loop and suck all the CPU cycles of one CPU core.
As of 2020-05-28, the leading candidate for this hypothesis is around this line of code: https://github.com/moby/hyperkit/blob/79c6a4d95e3f8a59f774eb66e3ea333a277292c6/src/lib/mirage_block_ocaml.ml#L422 and see this comment: https://github.com/docker/for-mac/issues/3499#issuecomment-623960890
Mitigations
Things that can cause the infinite loop seem to involve syncing of file system events between OS-X and the docker VM. The fewer file system shares you have, the less likely this is to occur. Similarly, if you can switch your filesystem mounts to
:cached
then that means there will be fewer notifications being sent back and forth between the Docker VM and OS-X, just less chance of the issue.The native Docker K8s implementation seems to involve a lot of this kind of sync, so you are more likely to trip over the issue.
Stuff I have found that makes life easier:
/Volumes
but once I removed that the frequency of occurrence was greatly reduced:/private
and/tmp
because some of the projects I work on have tests that use TempDirs as docker volume mounts.Hope this helps.
P.S. Remember your actual containers that you are running may be causing their own CPU usage spikes... that is not what this bug is about... those CPU spikes are likely the result of bugs in your containers or perhaps how you have configured volume mounts causing your containers to see excessive file system changes. This bug is for Docker CPU spiking when not running any containers or pods at all
P.P.S. I have heard interesting things about using Multipass to run k3s on OS-X without docker at all: https://medium.com/@zhimin.wen/running-k3s-with-multipass-on-mac-fbd559966f7c but that would, AIUI, force me to use something like Kaniko to actually build docker images within the k3s and that gets really ugly really fast. My current recommendation is to use k3d as my usage patters of Docker with k3d is only spiking the CPU about once a week for me.
Original bug report
Expected behavior
com.docker.hyperkit
should not turn my Mac into a space heater with megawatt output when stopping all running containersActual behavior
com.docker.hyperkit
process jumps to use all available CPU resources multiple times per day, some times when only vaguely interacting with docker.All docker commands lock up, e.g.
docker ps
Information
Diagnostic logs
This seems relevant
A second log after restarting docker from previous one. Only docker commands executed in between restart and this lock-up:
And the logs:
Steps to reproduce the behavior