High CPU Utilization of Hyperkit in Mac

[taufek]

My CPU utilization by hyperkit on my mac seems too high. It caused my mac to overheat.

Below is my docker version info:

╰─ docker version                                                                                                                                                                                                  13:06:29
Client:
 Version:      17.06.0-ce-rc4
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   29fcd5d
 Built:        Thu Jun 15 17:29:01 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      17.06.0-ce-rc4
 API version:  1.30 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   29fcd5dfae
 Built:        Thu Jun 15 18:50:07 2017
 OS/Arch:      linux/amd64
 Experimental: true

Below is my diagnostic id when this issue is happening. 1B8B3901-6393-41B7-B7D3-0FC914F1C668

I'm running rails container with docker-sync.

[vdugnist]

+1

docker version 17.12.0-ce-mac49 (21995) macos high sierra 10.13.3 (17D47)

all docker containers are down

[josja]

@vdugnist I also noticed 40% cpu load (hyperkit) with all containers down on colleagues machine. The only difference with my setup was that he wasn't logged into the tray app with his Docker ID. When he logged in CPU load fell back to 6%. Not sure why this helped (polling to docker server?).

[miguelmota]

hyperkit consuming insane amounts of CPU (266%) rendering my computer unusable when docker is running

[mkrakiewicz]

+1

[commadelimited]

Hyperkit taking 324% of my resources.

Took my battery from 100% to 12% in just over 30 minutes.

El Capitan 15" MBP mid 2015 2.5ghz core i7 16gb DDR3 1600mhz

[chexex]

I managed to reduce hyperkit CPU consumption from 250+% to 70%.

Docker version 18.02.0-ce-rc2, build f968a2c (EDGE)

MacBook Pro (13-inch, 2017)
2,3 GHz Intel Core i5
16 GB 2133 MHz LPDDR3
High Sierra 10.13.3 (17D47)

Steps:

1. Docker icon -> Preferences -> File Sharing -> remove /Users/ add directory with project (/Users/your_user_name/directory_with_project)
2. Docker icon -> Preferences -> Advanced -> Limit CPU to 1 Add memory to 4 GiB

[baldurpan]

+1 Completely unusable like this.

[nvaken]

Off-topic rant: I think there should be a penalty system for "+1" comments.

[baldurpan]

@nvaken I would agree, but only if that user neglects to "thumb up" the original post. ;)

[BrocksiNet]

Hello,

normally i work with magento2 on docker and use docker-sync for my volumes. The last two days i realized the same problem about the CPU consumption that is described here in this issue. I could watch in minutes how my battery get's empty. But i never had such a problem before so i know that something was changed.

After a lot of testing, destroying and rebuilding the containers i realized that the log files get spamed a lot by the normal magento2 cronjob. By the way hyperkit taking ~300 something energy. So docker-sync tried every millisecond to sync the new file between my local system and the containers. I disabled all cronjobs and then hyperkit get back to ~2-5 energy, same for CPU consumption.

So i would suggest everybody to look on the container and search if there is something that is writing in logs or generating files that get synced every millisecond.

greetings björn

Docker version 17.12.0-ce, build c97c6d6

[MikeMitterer]

I experience the same behavior since today maybe yesterday?? Version 18.02.0-ce-mac53 (22617)

### 2018-02-09 18.02.0-ce-mac53 "18.02.0-ce-mac53"

* Upgrades
  - [Docker 18.02.0-ce](https://github.com/docker/docker-ce/releases/tag/v18.02.0-ce)
  - [Docker compose 1.19.0](https://github.com/docker/compose/releases/tag/1.19.0)

* Bug fixes and minor changes
  - Fix update startup failure in some cases. 
  - Fix empty registry added by mistake in some cases in the Preference Daemon Pane. Fixes [docker/for-mac#2537](https://github.com/docker/for-mac/issues/2537)
  - Clearer error message when incompatible hardware is detected. Diagnostics are not proposed in the error popup in this case.

### 2017-02-08 18.02.0-ce-mac52 "18.02.0-ce-mac52"

* Upgrades
  - [Docker 18.02.0-ce](https://github.com/docker/docker-ce/releases/tag/v18.02.0-ce)
  - [Docker compose 1.19.0](https://github.com/docker/compose/releases/tag/1.19.0)

* Bug fixes and minor changes
  - Fix empty registry added by mistake in some cases in the Preference Daemon Pane. Fixes [docker/for-mac#2537](https://github.com/docker/for-mac/issues/2537)
  - Clearer error message when incompatible hardware is detected. Diagnostics are not proposed in the error popup in this case.

[ximet]

I have the same problem with version (Docker version 17.12.0-ce, build c97c6d6): CPU 120-180%.

[dfee]

Eight months later this issue is still where it began, at "status/0-triage". And everyday, I get +1's in my inbox.

So, I'm going to unsubscribe. And, I hope the Docker team gets around to solving this, or figuring it out, or taking the community seriously and communicating back. But that's not today, and I'm guessing it's not going to happen in the next eight months either. So good luck and farewell.

[moisesrodriguez]

After a lot of looking around I fixed my issue with constant CPU at around 200%. I use pm2 and for dev I use the watch feature so my node server restarts when I save something. Don't know why but I had the watch option with polling set to true. After I disabled polling, my CPU it now constantly at around 2%.

Hopefully this helps someone.

[JofArnold]

Quick summary of my findings.

For me the fix was a factory reset as per this comment.

Additionally some people have experienced minor improvements from the following:

• Adding :cached to the end of your volumes defs e.g. - ./api:/var/www/beta/api:cached as per this forum comment: https://forums.docker.com/t/com-docker-hyperkit-up-cpu-to-340/13057/72

• Setting docker to use 1 CPU as per this coderwall post https://coderwall.com/p/pl3rfg/fix-docker-for-mac-high-cpu-usage

• Upping the allocated memory to 4GB

Hope this is useful to someone. It has been an incredibly annoying issue for me!

[omarsafwany]

I kept the allocated CPUs and Memory without changing them. Whenever this issue would occur, I would quit the hyperkit using the activity monitor which mainly restarts docker. I may repeat the process 2-3 times till works normally below 10%. Not the smartest solution but it works for now.

[RobertDeRose]

@moisesrodriguez Can you give a more detailed explanation of this? What was polling? Was it a Docker setting or something external.

For me personally, The HyperKit with no dockers being deployed hovers around 5-10% CPU, but as soon as I deploy even one Container, it jumps to 40%, then as I add containers, it seems to double, even if the containers themselves are doing no actual work, just something like Nginx listening for connections.

[davidsielert]

I've had this happen off of 2 computers now.. This issue was definitely a contributing factor to burning up my old macbook pro due to overheating from the 320% Hyperkit CPU usage.. this still happens on a completely new macbook pro (not copying settings from the old), the only resolution is to stop all containers restart docker and then bring them back up, super annoying. and I have to remember to monitor my cpu usage throughout the day as it will randomly start doing the CPU spikes on hyper kit... This problem needs serious resolution. as a side note I've also noticed that sometimes when I restart the computer with all containers stopped when it restarts and docker is started its starting one of the containers (a ubuntu:latest one) and then my CPU usage is pinned until I stop it and restart docker..

[atombender]

From what I can tell from the comments, there are two separate issues here:

1. High CPU usage in combination with disk I/O (e.g. due to file system watchers); and

2. High CPU usage with nothing running.

I'm running DFM with Kubernetes enabled, and CPU usage stays around 40% even with nothing else deployed. According to top, Kubernetes on the VM is only taking around 2% when idling.

[moisesrodriguez]

@RobertDeRose polling is not docker related. I use pm2 which is a process manager for node.js which I use in my docker container. In the settings for my pm2 I have watch turned on which listens to changes in my node project and restart the server. With polling on, the file system watcher apparently was overwhelming hyperkit. After turning it off my hyperkit stays constantly at 2%

[archangel2080]

[rudietuesdays]

Looks like I'm late to the game, but I'm having the same issue. Tried the factory reset, still having the same problem. Running Docker by way of Lando. Kind of bummed that this has been going on for almost a year at least and Docker hasn't responded :/

@bjoern-flagbit do you mind posting instructions for how you stopped the cronjob? I'd like to give that a try.

[vkvasnytsky]

I used docker stats -a to detect a problem container, then added 2 second time out to npm configuration which was inside the container with the highest CPU load to rid of the issue. My CPU usage of the hyperkit process is back to 15% from 225%.

[BrocksiNet]

@bjoern-flagbit do you mind posting instructions for how you stopped the cronjob? I'd like to give that a try.

For the magento2 setup we had a file below /etc/cron.d/ in the php container, so i just comment out every line and then do docker-compose stop php-container-name and docker-compose start php-container-name to restart the container. I'm pretty sure the issue is some how connected to the system/framework you are using, because of that i suggested to look for files that are changing often you can use command's like this find . -name "*" -mtime -30s

[FrancescoBorzi]

I have the same problem using Version 17.12.0-ce-mac49 (21995)

[jeremychone]

Just updated to Version 18.03.0-ce-rc1-mac54 (23022) (from previous edge version) and to macos 10.13.3 (17D102), and now, consistently above 33% even though no docker images are running. Not sure which update triggered it.

This does make it unpractical to work unplugged (with the constant fan noise a smaller annoyance).

[RobertDeRose]

The newest "default to qcow2" update, and reset my Docker for Mac, I am now running at 20% CPU when I used to use 40 - 60%. I can't prove it, but I wonder if the raw disk has something to do with things. More testing is required of course

[chrinor2002]

We run a number of python django containers and found that adding --noreload to our runserver commands greatly reduced CPU use of hyperkit. As per above comments, it is likely related to filesystem monitoring. Docker: 17.12.0-ce-mac49 (21995) Mac: 10.13.3 (17D47)

[wz185]

I have the same problem using Version 17.12.0-ce-mac55 (23011)

[sietekk]

Same issue with Version 17.12.0-ce-mac55 (23011)

Hyperkit is regularly over 200%. I've tried just about every suggestion here.

[quochuy]

+1 Hyperkit over 300%, all cores are full on and the fan gets noisy

[dariagrudzien]

Same thing, Docker Version 17.12.0-ce-mac55 (23011), Mac 10.13.3 (17D102)

[ptpt]

Same here.

[rdsedmundo]

Same here.

[simongray]

Same here.

[atombender]

Please stop spamming with "same here" comments. Use the reaction buttons on the story if you don't have any new information.

[Tim-The-Woodsman]

A bug that keeps users from using the software, unfixed since june and you thing "same here" is the worst that can happen? I mean, you're correct, it's not helping. But ffs...

Btw, "same here"... Version 17.12.0-ce-mac55 (23011), stable version. Just did a factory reset. At least our servers don't suffer from that.

[dep-deprecated]

MacOS users, running Docker Edge with CPUs set to 4 and Memory @ 3.0GB took pageloads from ~12 seconds down to ~4 seconds. As usual your mileage may vary, but hope this helps someone.

[JofArnold]

How many of you are using nodemon here? Because I'm 95% that's our issue. Add a response to this comment like so (rather than saturating the notifications further)

• 👍 emoji for "yes I use it and have this problem"
• 👎 emoji for "no I don't use it and yet I have this problem"
• 🎉 emoji for "I use it, I've removed it and now everything works"
• 😕 emoji for "I use it, I've removed it but the problem remains"

[Tim-The-Woodsman]

@dep I'm tempted to try it, but I'm afraid the files would not work on our staging systems. What are your experiences?

[hedefalk]

Constant hyperkit 20% CPU even if no containers running.

docker for mac Version 17.12.0-ce-mac47 (21805).

OSX 10.13.3 (17D47)

[wamphlett]

@JofArnold I have confirmed that nodemon is causing this issue for me. I only have this problem when running one of my projects which is using nodemon, all my other containers are fine.

I am running command: node_modules/.bin/nodemon -L src/server.js through my docker-compose config and immediately get 300+ CPU usage from hyperkit. Changing this command to command: node src/server.js resolves the issue.

Putting the command back to how it originally was and running nodemon with verbose set to true showed me that nodemon was wrongly looking at directories that were used for resource management (which include regularly updated database log files).

[nodemon] reading config /project/nodemon.json
[nodemon] starting `node --inspect=0.0.0.0:9229 src/server.js`
[nodemon] files triggering change check: store/mysql/ibdata1
[nodemon] changes after filters (before/after): 1/0
[nodemon] files triggering change check: store/mysql/ibtmp1
[nodemon] changes after filters (before/after): 1/0
[nodemon] watching 383 files

By adding a watch to my nodemon config to explicitly look at the src directory only, the number of watched files reduced.

// /project/nodemon.json
{
    "verbose": true,
    "watch": ["src/"]
}

[nodemon] reading config /project/nodemon.json
[nodemon] starting `node --inspect=0.0.0.0:9229 src/server.js`
[nodemon] watching 30 files

Hyperkit is still running higher than it does without nodemon but instead of 300%+, its running around 20%-28% now which is a considerable improvement.

[JofArnold]

@wamphlett Great sleuthing. I'll try the same later. Thanks.

[JofArnold]

After some experimentation last night and this morning I can now reproduce and then stop heavy CPU consumption by Docker by deliberately introducing problems in our application. For instance having filesystem watchers like nodemon go into infinite loop (e.g. trigging an action from a file change that then causes the file to change again), not excluding files from build correctly, flowtype/eslint bad configs, breaking dependencies and so-forth. There are a lot of ways of triggering - some of which aren't necessarily conspicuous when running natively. What I'm saying is that for me the problems appear to be entirely eliminated by fixing the application itself. That Docker uses so much resource when it gets into this state is the only remaining frustration for me right now.

If you face these CPU problems I recommend the following steps before adding to the thread here:

• Change your docker-compose file to limit the CPU usage in dev to something like 0.5 cores (which frees up some resource so you can have any hope of fixing the issue!)

• Check your application - especially filesystem watchers as fs i/o seems to be a perf issue with docker on mac anyway so the problem is multiplied.

Not saying that will fix it for you, but it may especially within the node ecosystem. Good luck.

[kmyllyvi]

Yup, me too. Started happening or at least got worse after updating to OSX 10.13.3. Always over 100% CPU, usually round 150%

[ptpt]

It seems that my docker apps use more than 2GB (the default volume), then the swap is triggered, which consumes CPU I guess.

I increased memory, and the problem got solved.

[yelvert]

hyperkit spikes my CPU to around 150% every 45 seconds like clockwork. I am running the following 4 containers, all of which are idle:

┗━⫸ docker ps
CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                               NAMES
c6c90e17520b        elasticsearch:2.4     "/docker-entrypoint.…"   2 weeks ago         Up 8 days           9300/tcp, 0.0.0.0:32769->9200/tcp   elasticsearch
c74c7ce47ae2        postgres:9.6-alpine   "docker-entrypoint.s…"   2 weeks ago         Up 8 days           0.0.0.0:32771->5432/tcp             postgres
6f9b96ad991d        mariadb:10            "docker-entrypoint.s…"   2 weeks ago         Up 8 days           0.0.0.0:32770->3306/tcp             mysql
f6833da91b4e        redis:3.2.11-alpine   "docker-entrypoint.s…"   2 weeks ago         Up 8 days           0.0.0.0:32768->6379/tcp             redis

My docker version is Docker version 17.12.0-ce, build c97c6d6

[cloutiertyler]

FWIW, this comment fixed my problem: https://forums.docker.com/t/com-docker-hyperkit-up-cpu-to-340/13057/73

I have many containers with nodemon/directory watchers

[diegogub]

+1

[mmacvicar-splunk]

I can somewhat work around this issue by restarting docker.

If I let docker run long enough, a few hours, in this state with hyperkit at 300% CPU, my containers start reporting "No space left on device" and hang. Killing Docker makes it so restarting docker report "Failed to start Docker daemon" and I have to do a factory reset to recover.