Docker CLI Hangs for all commands

[dvndrsn]

Description

Docker CLI hangs when running any command. Fresh MacOS and Docker install on M1 Mac.

MacOS: 13.5 (22G74) Chip: Apple M1 Pro Docker Desktop: 4.22.0 (117440)

Tried re-installing / factory resetting docker and clearing docker files as suggested in this issue: https://github.com/docker/for-mac/issues/6527

Self diagnosis shows that VM cannot be started. Any tips for next steps?

dave.anderson@updater.com@Dave's-Mac ~ % /Applications/Docker.app/Contents/MacOS/com.docker.diagnose check
Starting diagnostics

[PASS] DD0027: is there available disk space on the host?
[PASS] DD0028: is there available VM disk space?
[PASS] DD0018: does the host support virtualization?
[PASS] DD0001: is the application running?
[FAIL] DD0017: can a VM be started? vm has not started: failed to open kmsg.log: open log/vm/kmsg.log: no such file or directory
[FAIL] DD0016: is the LinuxKit VM running? prereq failed: can a VM be started?
[FAIL] DD0011: are the LinuxKit services running? prereq failed: is the LinuxKit VM running?
[FAIL] DD0004: is the Docker engine running? prereq failed: are the LinuxKit services running?
[PASS] DD0015: are the binary symlinks installed?
[FAIL] DD0031: does the Docker API work? prereq failed: is the Docker engine running?
[PASS] DD0013: is the $PATH ok?
[FAIL] DD0003: is the Docker CLI working? prereq failed: is the Docker engine running?
[FAIL] DD0038: is the connection to Docker working? prereq failed: is the Docker engine running?
[FAIL] DD0014: are the backend processes running? prereq failed: is the LinuxKit VM running?
[FAIL] DD0007: is the backend responding? prereq failed: are the backend processes running?
[PASS] DD0008: is the native API responding?
[FAIL] DD0009: is the vpnkit API responding? prereq failed: are the backend processes running?
[FAIL] DD0010: is the Docker API proxy responding? prereq failed: are the backend processes running?
[FAIL] DD0030: is the image access management authorized? prereq failed: is the Docker engine running?
[PASS] DD0033: does the host have Internet access?
[PASS] DD0018: does the host support virtualization?
[PASS] DD0001: is the application running?
[WARN] DD0017: can a VM be started? vm has not started: failed to open kmsg.log: open log/vm/kmsg.log: no such file or directory
[WARN] DD0016: is the LinuxKit VM running? prereq failed: can a VM be started?
[WARN] DD0011: are the LinuxKit services running? prereq failed: is the LinuxKit VM running?
[WARN] DD0004: is the Docker engine running? prereq failed: are the LinuxKit services running?
[PASS] DD0015: are the binary symlinks installed?
[WARN] DD0031: does the Docker API work? prereq failed: is the Docker engine running?
[WARN] DD0032: do Docker networks overlap with host IPs? prereq failed: does the Docker API work?

Please note the following 6 warnings:

1 : The check: can a VM be started?
    Produced the following warning: vm has not started: failed to open kmsg.log: open log/vm/kmsg.log: no such file or directory

The Docker engine runs inside a Linux VM. Therefore we must be able to start Virtual Machines.

2 : The check: is the LinuxKit VM running?
    Produced the following warning: prereq failed: can a VM be started?

The Docker engine runs inside a Linux VM. Therefore the VM must be running.

3 : The check: are the LinuxKit services running?
    Produced the following warning: prereq failed: is the LinuxKit VM running?

The Docker engine runs inside a Linux VM as a service. Therefore the services must have started.

4 : The check: is the Docker engine running?
    Produced the following warning: prereq failed: are the LinuxKit services running?

The Docker engine manages all containers and images on the host. Check the dockerd.log to see why it failed to start.

5 : The check: does the Docker API work?
    Produced the following warning: prereq failed: is the Docker engine running?

If the Docker API is not available from the host then Docker Desktop will not work correctly.

6 : The check: do Docker networks overlap with host IPs?
    Produced the following warning: prereq failed: does the Docker API work?

If the subnet used by a Docker network overlaps with an IP used by the host, then containers
won't be able to contact the overlapping IP addresses.

Try configuring the IP address range used by networks: in your docker-compose.yml.
See https://docs.docker.com/compose/compose-file/compose-file-v2/#ipv4_address-ipv6_address

Please investigate the following 1 issue:

1 : The test: can a VM be started?
    Failed with: vm has not started: failed to open kmsg.log: open log/vm/kmsg.log: no such file or directory

The Docker engine runs inside a Linux VM. Therefore we must be able to start Virtual Machines.

Reproduce

1. Run any command - docker ps, docker info, etc.

Expected behavior

Command should output as expected (ex. list of processes or info)

docker version

Running `docker version` hangs on CLI.

% docker --version
Docker version 24.0.5, build ced0996

docker info

Hangs, no output.

Diagnostics ID

CDA29737-2B38-4A57-9384-068DB58FFD94/20230809183121

Additional Info

First diagnostic from docker desktop: CDA29737-2B38-4A57-9384-068DB58FFD94/20230809183121 Ran another diagnostic from the command line: CDA29737-2B38-4A57-9384-068DB58FFD94/20230809183808

[ctalledo]

Hi @dvndrsn, sorry to hear you hit a problem but thanks for uploading the diagnostics bundle, that really helps us.

I looked at the logs in the diagnostics bundle, and I can see that the Docker Desktop VM fails to come up, though it's not exactly clear why (we have some hints in there though).

Question: was this a one-time failure, or does it reoccur frequently with Docker Desktop v4.22?

Thanks and apologies again for the issue.

[MuzzamielBarrows]

Having the same issue on mac m1

[ctalledo]

Having the same issue on mac m1

Hi @MuzzamielBarrows: if possible, could you upload a diagnostics bundle please?

Thanks!

[dvndrsn]

Question: was this a one-time failure, or does it reoccur frequently with Docker Desktop v4.22?

I've not yet been able to run docker successfully on this machine.

I can see that the Docker Desktop VM fails to come up, though it's not exactly clear why (we have some hints in there though)

Hints? Please share! I can see from the self-diangostic (great tool btw) that the Linux VM isn't coming up, but I'm not clear what intervention I can do on my end to try to restart that VM and help it along..

[kvasylchenko]

I have exactly the same on my Mac with M2 chip.

MacOS: 13.5 Chip: Apple M2 Pro Docker Desktop: 4.22.0 (117440)

[VerstraeteBert]

Same issue here on M1, MacOs 13.5, Docker Desktop 4.22. Identical diagnosis report.

[ctalledo]

Thanks all for filing the reports.

We have a new (not yet released) build with a potential fix.

If you could give it a try and let us know if it fixes the problem (or not), we would much appreciate it.

Thanks!

[dvndrsn]

Update: I installed Docker Desktop 4.21.1 from the release notes page and simple commands like docker info and docker version no longer hang.

There must be a problem specifically with 4.22.0 and Apple Silicon (at least for fresh installs?).

[ctalledo]

Thanks @dvndrsn.

To provide more context, in DD v4.22 we have a new resource saver feature that turns off the Docker Desktop Linux VM whenever Docker goes unused for a period of time (e.g., 30 secs or more). The goal is to reduce host memory utilization (by 2GB+), whenever you are not using Docker.

This feature works well for most users, but a few have reported issues with Docker Desktop startup as well as exit from resource saver mode.

We have an unreleased build with a potential fix, if you could try it and let us know if it works, we would much appreciate it.

Thanks again!

[dvndrsn]

Thanks @ctalledo! I will try the unreleased build once I've gotten my environment up and running with the prior release of docker for mac.

[guillaumebl]

Hello, I've got the same issue and I tried upgrading to v4.23. Sadly it did not solve the issue, the only way to fix it has been to downgrade to 4.21

[ctalledo]

Hello, I've got the same issue and I tried upgrading to v4.23. Sadly it did not solve the issue, the only way to fix it has been to downgrade to 4.21

Thanks @guillaumebl for letting us know; would you have a diagnostics bundle that we can look at (ideally with v4.23)?

[adrianwix]

I have the same issue even on 4.21.1 (114176) sometimes docker just stops working

Apple M2 macOS 13.5

Then I have restart the computer, delete all my imagines and volumes and pray the gods it works. You can imagine the drop in productivity

I can't even update it

[guillaumebl]

@ctalledo Here's the Diagnostics ID : BA6F13F0-E794-489B-80A5-26CF0E0A7E9C/20230817155545 Thanks

[ickeundso]

I'm not sure this my problem is the same but seems to be in that direction. I can run DD 4.22 in Mac M1 / MacOS 13.5 . Everthing works fine. But after a wile , several minutes / sometime hours . All containers are hanging (except my nginx container is delivering a http 502) errors) and all docker commands like docker ps hanging too. I can open the DD UI and can see 400% for one CPU but ( normally if it works I have configured 6CPU's and is show as such after restart) But no running container is shown. I attach the /Applications/Docker.app/Contents/MacOS/com.docker.diagnose check docker-stuck-completely.log A restart from the DD-UI brings the containers and the docker commands to work again. So I'm going to disable the [resource saver feature] , I will see then.

[ickeundso]

Ok, also when the Experimental feature for resource saving is deselected in the DD-UI , the engine will stop after a while. But on the other hand on my MacBook Pro with M1 , with no running container . Docker is running and also a e.g docker ps will respond with the right values.

[ickeundso]

It seems as if version 4.22 doesn't cause issues on every M1 device. On my MacBook: M1 Max 64GB, Docker runs without problems, both with "Resource Saver" enabled and with or without running containers. For example, "docker ps" can still be executed even after hours. However, on my Mac mini M1 16GB, it's different. Whether "Resource Saver" is enabled or disabled, Docker stops working shortly after some time. Unfortunately, I can't pinpoint exactly when this happens, but it's not too long. My Mac isn't connected to a monitor; I only access it via SSH or, if necessary, through Remote Desktop.So, after downgrading to version 4.21.1 on my Mac mini and keeping the running containers unchanged, same behaviour as with 4.22 .

[ickeundso]

Happens also with 4.20.0 on my mac min, after around 1h: Support Id: B0B019C8-0DE1-452F-AC20-31E518C7CDA6/20230820160627

Not only the "docker SOMETHING" does not work also a "Quit Docker Desktop" doesn't. But the "Restart" on Docker Desktop was working on 4.22. And after that I can also use "Quit Docker Desktop" or everything else. But with 4.20 I have to restart the machine or to kill this docker ' docker serve --address unix:///Users/maex/.docker/run/docker-cli-api.sock' from the shell.

[ickeundso]

one more difference between my MacBook M1 and the Mac mini. The diagnostic check on the MacBook has no issues but on then Mac mini I get every time: `[FAIL] DD0014: are the backend processes running? 1 error occurred:

• com.docker.vpnkit is not running

[PASS] DD0007: is the backend responding? [PASS] DD0008: is the native API responding? [FAIL] DD0009: is the vpnkit API responding? dial unix vpnkit.diag.sock: connect: connection refused`

can this be related to this issue?

[ickeundso]

So after downgrading to 4.18 and after disable Use Virtualization framework and using gRPC for file sharing. Docker works for 2 days without any problem. So then I updated (per DD-UI) to 4.22 again an checked that Use Virtualization framework is still disabled. The problem occurs again after a few hours. B0B019C8-0DE1-452F-AC20-31E518C7CDA6/20230823053837

[ickeundso]

So now I tried to go forward to find out what version breaks the docker stability. But I need no much time. 4.19 worked perfect but with 4.20 the docker hangs again after a short time <2h . Here the diagnose Id for 4.20 : B0B019C8-0DE1-452F-AC20-31E518C7CDA6/20230825065317

after the fresh installation on my mac mini: ` /Applications/Docker.app/Contents/MacOS/com.docker.diagnose check [2023-08-25T07:35:40.903328000Z][com.docker.diagnose][I] set path configuration to OnHost Starting diagnostics

[PASS] DD0027: is there available disk space on the host? [PASS] DD0028: is there available VM disk space? [PASS] DD0018: does the host support virtualization? [PASS] DD0001: is the application running? [PASS] DD0017: can a VM be started? [PASS] DD0016: is the LinuxKit VM running? [PASS] DD0011: are the LinuxKit services running? [PASS] DD0004: is the Docker engine running? [PASS] DD0015: are the binary symlinks installed? [PASS] DD0031: does the Docker API work? [PASS] DD0013: is the $PATH ok? [PASS] DD0003: is the Docker CLI working? [PASS] DD0038: is the connection to Docker working? [PASS] DD0014: are the backend processes running? [PASS] DD0007: is the backend responding? [PASS] DD0008: is the native API responding? [SKIP] DD0009: is the vpnkit API responding? [PASS] DD0010: is the Docker API proxy responding? [SKIP] DD0030: is the image access management authorized? [PASS] DD0033: does the host have Internet access? [PASS] DD0018: does the host support virtualization? [PASS] DD0001: is the application running? [PASS] DD0017: can a VM be started? [PASS] DD0016: is the LinuxKit VM running? [PASS] DD0011: are the LinuxKit services running? [PASS] DD0004: is the Docker engine running? [PASS] DD0015: are the binary symlinks installed? [PASS] DD0031: does the Docker API work? [PASS] DD0032: do Docker networks overlap with host IPs? No fatal errors detected.`

and after <2 h running until I figure out that the containers not longer reachable and the command a hanging again: produces then :

`[PASS] DD0027: is there available disk space on the host? [PASS] DD0028: is there available VM disk space? [PASS] DD0018: does the host support virtualization? [PASS] DD0001: is the application running? [PASS] DD0017: can a VM be started? [PASS] DD0016: is the LinuxKit VM running? [FAIL] DD0011: are the LinuxKit services running? failed to ping VM diagnosticsd with error: Get "http://ipc/ping": context deadline exceeded (Client.Timeout exceeded while awaiting headers) [2023-08-25T07:12:11.085836000Z][com.docker.diagnose][I] ipc.NewClient: 9359e6b3-diagnose -> diagnosticd.sock diagnosticsd [2023-08-25T07:12:11.085981000Z][com.docker.diagnose][I] (9d3f2e99) 9359e6b3-diagnose C->S diagnosticsd GET /ping [2023-08-25T07:12:12.088495000Z][com.docker.diagnose][W] (9d3f2e99) 9359e6b3-diagnose C<-S NoResponse GET /ping (1.002422792s): Get "http://ipc/ping": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

[FAIL] DD0004: is the Docker engine running? Get "http://ipc/docker": context deadline exceeded (Client.Timeout exceeded while awaiting headers) [2023-08-25T07:12:12.089228000Z][com.docker.diagnose][I] ipc.NewClient: ad7c7947-com.docker.diagnose -> lifecycle-server.sock VMDockerdAPI [2023-08-25T07:12:12.089315000Z][com.docker.diagnose][I] (7054c471) ad7c7947-com.docker.diagnose C->S VMDockerdAPI GET /docker [2023-08-25T07:17:12.098781000Z][com.docker.diagnose][W] (7054c471) ad7c7947-com.docker.diagnose C<-S NoResponse GET /docker (5m0.002550833s): Get "http://ipc/docker": context deadline exceeded (Client.Timeout exceeded while awaiting headers) [2023-08-25T07:17:12.100408000Z][com.docker.diagnose][I] (7054c471-1) ad7c7947-com.docker.diagnose C->S VMDockerdAPI GET /ping [2023-08-25T07:17:13.102354000Z][com.docker.diagnose][W] (7054c471-1) ad7c7947-com.docker.diagnose C<-S NoResponse GET /ping (1.001904958s): Get "http://ipc/ping": context deadline exceeded (Client.Timeout exceeded while awaiting headers) [2023-08-25T07:17:14.103689000Z][com.docker.diagnose][I] (7054c471-2) ad7c7947-com.docker.diagnose C->S VMDockerdAPI GET /ping [2023-08-25T07:17:15.105425000Z][com.docker.diagnose][W] (7054c471-2) ad7c7947-com.docker.diagnose C<-S NoResponse GET /ping (1.001625416s): Get "http://ipc/ping": context deadline exceeded (Client.Timeout exceeded while awaiting headers) [2023-08-25T07:17:16.106867000Z][com.docker.diagnose][I] (7054c471-3) ad7c7947-com.docker.diagnose C->S VMDockerdAPI GET /ping [2023-08-25T07:17:17.108166000Z][com.docker.diagnose][W] (7054c471-3) ad7c7947-com.docker.diagnose C<-S NoResponse GET /ping (1.001270208s): Get "http://ipc/ping": context deadline exceeded (Client.Timeout exceeded while awaiting headers) [2023-08-25T07:17:18.109422000Z][com.docker.diagnose][I] (7054c471-4) ad7c7947-com.docker.diagnose C->S VMDockerdAPI GET /ping [2023-08-25T07:17:19.111342000Z][com.docker.diagnose][W] (7054c471-4) ad7c7947-com.docker.diagnose C<-S NoResponse GET /ping (1.001871417s): Get "http://ipc/ping": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

[PASS] DD0015: are the binary symlinks installed? [FAIL] DD0031: does the Docker API work? Get "http://docker.raw.sock/v1.24/containers/json": context deadline exceeded [PASS] DD0013: is the $PATH ok? [FAIL] DD0003: is the Docker CLI working? signal: killed [PASS] DD0038: is the connection to Docker working? [PASS] DD0014: are the backend processes running? [PASS] DD0007: is the backend responding? [PASS] DD0008: is the native API responding? [SKIP] DD0009: is the vpnkit API responding? [PASS] DD0010: is the Docker API proxy responding? [SKIP] DD0030: is the image access management authorized? [PASS] DD0033: does the host have Internet access? [PASS] DD0018: does the host support virtualization? [PASS] DD0001: is the application running? [PASS] DD0017: can a VM be started? [PASS] DD0016: is the LinuxKit VM running? [WARN] DD0011: are the LinuxKit services running? failed to ping VM diagnosticsd with error: Get "http://ipc/ping": context deadline exceeded (Client.Timeout exceeded while awaiting headers) [WARN] DD0004: is the Docker engine running? Get "http://ipc/docker": context deadline exceeded (Client.Timeout exceeded while awaiting headers) [PASS] DD0015: are the binary symlinks installed? [WARN] DD0031: does the Docker API work? Get "http://docker.raw.sock/v1.24/containers/json": context deadline exceeded [WARN] DD0032: do Docker networks overlap with host IPs? Get "http://docker.raw.sock/v1.24/networks": context deadline exceeded

Please note the following 4 warnings:

1 : The check: are the LinuxKit services running? Produced the following warning: failed to ping VM diagnosticsd with error: Get "http://ipc/ping": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

The Docker engine runs inside a Linux VM as a service. Therefore the services must have started.

2 : The check: is the Docker engine running? Produced the following warning: Get "http://ipc/docker": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

The Docker engine manages all containers and images on the host. Check the dockerd.log to see why it failed to start.

3 : The check: does the Docker API work? Produced the following warning: Get "http://docker.raw.sock/v1.24/containers/json": context deadline exceeded

If the Docker API is not available from the host then Docker Desktop will not work correctly.

4 : The check: do Docker networks overlap with host IPs? Produced the following warning: Get "http://docker.raw.sock/v1.24/networks": context deadline exceeded

If the subnet used by a Docker network overlaps with an IP used by the host, then containers won't be able to contact the overlapping IP addresses.

Try configuring the IP address range used by networks: in your docker-compose.yml. See https://docs.docker.com/compose/compose-file/compose-file-v2/#ipv4_address-ipv6_address

Please investigate the following 1 issue:

1 : The test: are the LinuxKit services running? Failed with: failed to ping VM diagnosticsd with error: Get "http://ipc/ping": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

The Docker engine runs inside a Linux VM as a service. Therefore the services must have started.`

[ctalledo]

Hi @ickeundso, could you please try with Docker Desktop v4.22.1? If you hit the issue again, please upload the diagnostics bundle for this version. Thanks!

[ickeundso]

Hi @ctalledo, this first try didn't work. I did an update in DD from 4.18 -> 4.22.1 (unfortunately I didn't get a diagnostic). So now I'm trying another approche. I went back to 4.18 , run this version without issue for 24h. So now stopped the DD . Installed the new version 4.22.1 from a download file and restarted my host machine. For now, it works for at least 12h. This is longer than in most cases described above. So I will wait for another couple of hours. Hopefully it will work again in the same reliable way as I know docker for years.

[ickeundso]

Hi, @ctalledo , so it seems to work for me. For 2 day docker 4.22.1 run without an problem. All my containers and the docker commands are functional and working as expected. I also enabled "Use virtialisation Framwork" and VirtioFs. I do not konw if someone need this, but also for this successfully 2 days run I uploaded the diagnostics. B0B019C8-0DE1-452F-AC20-31E518C7CDA6/20230903071459. Thank you for investigating and fixing the Issue!

[0x2d3c]

v4.22.1 desktop have the same question, in cmd docker ps -a with nothing out put

[serdar]

I have the same issue as well on an M2 MacBook Pro running 4.22.1. (Mac OS version 13.4.1 (c) )

Diagnostics ID: FCA5E48B-4E08-4800-B63F-71AC0585560A/20230907060156

[ctalledo]

Hi @serdar,

I have the same issue as well on an M2 MacBook Pro running 4.22.1. (Mac OS version 13.4.1 (c) ) Diagnostics attached.

Thanks for reporting the issue. When you run the diagnostics, Docker Desktop should have uploaded them and you should have received a diagnostics ID; could you share that ID number so I can pull it up? The serdar-docker.log file you attached is not sufficient for me to debug.

Thanks!

[adamczykpiotr]

Same thing for Docker (24.0.2-rd, build e63f5fa) + Rancher desktop on M2 Pro. For me it hangs only for specific operations - i.e. with composer and then running docker compose run --rm composer install. After this docker becomes unoperable, docker ps doesn't respond.

[0x2d3c]

v4.22.1 desktop have the same question, in cmd docker ps -a with nothing out put

i fix this question by downgrade to v4.20, mac mini m1

[0x2d3c]

I have the same issue as well on an M2 MacBook Pro running 4.22.1. (Mac OS version 13.4.1 (c) )

Diagnostics ID: FCA5E48B-4E08-4800-B63F-71AC0585560A/20230907060156

maybe downgrade to 4.20 can fix your question

[ctalledo]

Hi @serdar,

Thanks for posting the diagnostics bundle ID.

Looking at it, I can see that the Docker Desktop Linux VM fails to start because Docker Engine inside the VM runs out of disk space:

ESC[91m[2023-09-07T00:35:16.677338000Z][com.docker.backend.main][E] dockerd terminated unexpectedly: [2023-09-07T00:35:16.673252708Z][dockerd][I] time="2023-09-07T00:35:16.673180041Z" level=error msg="mkdir /var/lib/docker/overlay2/check-overlayfs-support2886511961/lower1: no space left on device" storage-driver=overlay2ESC[39m

That means the disk space inside the Linux VM is almost full. Looking at the disk space inside the VM, I see that's the case:

Filesystem                Size      Used Available Use% Mounted on
/dev/vda1                58.4G     58.4G         0 100% /var/lib/docker

I am guessing the reason the disk space is full is because you have possibly lots of container images (or maybe some heavy ones)?

If so, try increasing the VM disk space size by editing the settings.json file ($HOME/Library/Group Containers/group.com.docker/settings.json):

 "diskSizeMiB": 131072,       

Then restart Docker Desktop and hopefully you won't see the error in the log file (`$HOME/Library/Containers/com.docker.docker/Data/log/host/com.docker.backend.log).

Hope that helps.

[Meowzz95]

Hitting the same issue using 4.22.1 version.

Most of the time docker is stuck at "docker engine starting" and sometimes even when the engine starts, a simple command like docker ps stuck forever.

[bsousaa]

@Meowzz95 please generate a Diagnostics bundle ID for additional investigation.

[Meowzz95]

@bsousaa I tried but even the diagnostics is not functioning properly, it shows "collecting diagnostics"(I forgot the exact phrase used) and is stuck there

[serdar]

Hi @ctalledo Much appreciated, increasing the allocated Linux VM storage value got things working again.

Thank you.

[ctalledo]

Hi folks,

We just released Docker Desktop v4.23 which had a few fixes in this area.

Could you please install and let us know if the problem persists?

Thanks!

[jamespichardo]

I got Docker Desktop 4.23.0 installed and same issue: docker commands (ps, version) just hang.

[jamespichardo]

It is running fine now... never mind!

[dvndrsn]

The latest release has addressed my problems as well.

[ctalledo]

Closing this issue as it should be solved in 4.22.1 and after.

If that's not the case, please re-open the issue and post the diagnostics bundle so we can debug further.

Thanks all for the feedback and your patience as we improve Docker Desktop.

[nukopy]

I have same issue.

Environment

• OS: macOS 14.1
• CPU: Apple M1 Max

Software versions

• Docker 24.0.6

  $ docker --version
  Docker version 24.0.6, build ed223bc

• Docker Desktop v4.24.2 (124339)

Diagnostics Result

Diagnose from Docker Desktop

• Diagnostics ID: 8C41E227-14F1-43C5-88ED-0437A96119F7/20231022150300

Diagnose from the terminal

$ /Applications/Docker.app/Contents/MacOS/com.docker.diagnose check

Starting diagnostics

[PASS] DD0027: is there available disk space on the host?
[PASS] DD0028: is there available VM disk space?
[PASS] DD0018: does the host support virtualization?
[PASS] DD0001: is the application running?
[PASS] DD0017: can a VM be started?
[PASS] DD0016: is the LinuxKit VM running?
[FAIL] DD0011: are the LinuxKit services running? failed to ping VM diagnosticsd with error: Get "http://ipc/ping": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
[2023-10-22T15:04:34.675741000Z][com.docker.diagnose.ipc][I] ipc.NewClient: 68f5c9a2-diagnose -> <HOME>/Library/Containers/com.docker.docker/Data/diagnosticd.sock diagnosticsd
[2023-10-22T15:04:34.675830000Z][com.docker.diagnose.ipc][I] (ff1981d1) 68f5c9a2-diagnose C->S diagnosticsd GET /ping
[2023-10-22T15:04:35.677176000Z][com.docker.diagnose.ipc][W] (ff1981d1) 68f5c9a2-diagnose C<-S NoResponse GET /ping (1.00129s): Get "http://ipc/ping": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

[FAIL] DD0004: is the Docker engine running? prereq failed: are the LinuxKit services running?
[PASS] DD0015: are the binary symlinks installed?
[FAIL] DD0031: does the Docker API work? prereq failed: is the Docker engine running?
[PASS] DD0013: is the $PATH ok?
[FAIL] DD0003: is the Docker CLI working? prereq failed: is the Docker engine running?
[FAIL] DD0038: is the connection to Docker working? prereq failed: is the Docker engine running?
[PASS] DD0014: are the backend processes running?
[PASS] DD0007: is the backend responding?
[PASS] DD0008: is the native API responding?
[SKIP] DD0009: is the vpnkit API responding?
[PASS] DD0010: is the Docker API proxy responding?
[FAIL] DD0030: is the image access management authorized? prereq failed: is the Docker engine running?
[PASS] DD0033: does the host have Internet access?
[PASS] DD0018: does the host support virtualization?
[PASS] DD0001: is the application running?
[PASS] DD0017: can a VM be started?
[PASS] DD0016: is the LinuxKit VM running?
[WARN] DD0011: are the LinuxKit services running? failed to ping VM diagnosticsd with error: Get "http://ipc/ping": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
[WARN] DD0004: is the Docker engine running? prereq failed: are the LinuxKit services running?
[PASS] DD0015: are the binary symlinks installed?
[WARN] DD0031: does the Docker API work? prereq failed: is the Docker engine running?
[WARN] DD0032: do Docker networks overlap with host IPs? prereq failed: does the Docker API work?

Please note the following 4 warnings:

1 : The check: are the LinuxKit services running?
    Produced the following warning: failed to ping VM diagnosticsd with error: Get "http://ipc/ping": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

The Docker engine runs inside a Linux VM as a service. Therefore the services must have started.

2 : The check: is the Docker engine running?
    Produced the following warning: prereq failed: are the LinuxKit services running?

The Docker engine manages all containers and images on the host. Check the dockerd.log to see why it failed to start.

3 : The check: does the Docker API work?
    Produced the following warning: prereq failed: is the Docker engine running?

If the Docker API is not available from the host then Docker Desktop will not work correctly.

4 : The check: do Docker networks overlap with host IPs?
    Produced the following warning: prereq failed: does the Docker API work?

If the subnet used by a Docker network overlaps with an IP used by the host, then containers
won't be able to contact the overlapping IP addresses.

Try configuring the IP address range used by networks: in your docker-compose.yml.
See https://docs.docker.com/compose/compose-file/compose-file-v2/#ipv4_address-ipv6_address

Please investigate the following 1 issue:

1 : The test: are the LinuxKit services running?
    Failed with: failed to ping VM diagnosticsd with error: Get "http://ipc/ping": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

The Docker engine runs inside a Linux VM as a service. Therefore the services must have started.

[nukopy]

I solved issue. Never mind!

https://github.com/docker/for-mac/issues/6940#issuecomment-1774119004

How to solve

I couldn't stop Docker Desktop on Troubleshoot Restart Docker Desktop button.

I killed all processes related to Docker and Docker Desktop with Force Quit on /System/Applications/Utilities/Activity Monitor.app. After this, Docker Desktop runs correctly!

$ docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

[yelvert]

This is still happening on v4.25.0. All the containers seems to be working fine, but all docker commands hang until restarting the whole docker service, after restarting commands will work for between 5 minutes and a few hours, then go back to hanging every time until restarting again.

[2023-11-08T14:41:58.322891000Z][com.docker.diagnose.ipc][I] ipc.NewClient: ad08253a-diagnose -> <HOME>/Library/Containers/com.docker.docker/Data/backend.sock BackendAPI
[2023-11-08T14:41:58.324523000Z][com.docker.diagnose.ipc][I] (9e82557e) ad08253a-diagnose C->S BackendAPI POST /idle/make-busy
[2023-11-08T14:41:58.327599000Z][com.docker.diagnose.ipc][I] (9e82557e) ad08253a-diagnose C<-S 2c496337-BackendAPI POST /idle/make-busy (3.07425ms): 0x14000010a98
[2023-11-08T14:41:59.325164000Z][com.docker.diagnose.ipc][I] (4b145977) ad08253a-diagnose C->S BackendAPI GET /idle
[2023-11-08T14:41:59.327355000Z][com.docker.diagnose.ipc][I] (4b145977) ad08253a-diagnose C<-S 2c496337-BackendAPI GET /idle (1.757584ms): {"apisInFlight":{"/_ping":128,"/exec/f102e65628b777afa0230bdc95be21344f935d352571f9f27d616d2b51d99a31/start":1,"/system/df":1,"/v1.43/containers/72e89622601477e1d4aa37117810816151a702e27cbcc2a80f3f1e49e97a4620/logs":1},"booted":true,"busyReason":["4 container(s)","in-flight APIs: map[/_ping:128 /exec/f102e65628b777afa0230bdc95be21344f935d352571f9f27d616d2b51d99a31/start:1 /system/df:1 /v1.43/containers/72e89622601477e1d4aa37117810816151a702e27cbcc2a80f3f1e49e97a4620/logs:1]","timed activities: map[/idle/make-busy:29.001394709s]"],"containers":4,"idle":"bool","kubernetesEnabled":false,"reduced":false,"services":0,"timedActivities":{"/idle/make-busy":"float64"},"vmPaused":false,"vmRunning":true,"vmStopped":false,"windowsContainers":false}
[PASS] DD0027: is there available disk space on the host?
[PASS] DD0028: is there available VM disk space?
[PASS] DD0018: does the host support virtualization?
[PASS] DD0001: is the application running?
[PASS] DD0017: can a VM be started?
[PASS] DD0016: is the LinuxKit VM running?
[FAIL] DD0011: are the LinuxKit services running? failed to ping VM diagnosticsd with error: Get "http://ipc/ping": EOF
[2023-11-08T14:41:59.355344000Z][com.docker.diagnose.ipc][I] ipc.NewClient: 51152e83-diagnose -> <HOME>/Library/Containers/com.docker.docker/Data/diagnosticd.sock diagnosticsd
[2023-11-08T14:41:59.355401000Z][com.docker.diagnose.ipc][I] (150a2bbf) 51152e83-diagnose C->S diagnosticsd GET /ping
[2023-11-08T14:41:59.356210000Z][com.docker.diagnose.ipc][W] (150a2bbf) 51152e83-diagnose C<-S NoResponse GET /ping (801.584µs): Get "http://ipc/ping": EOF

[FAIL] DD0004: is the Docker engine running? prereq failed: are the LinuxKit services running?
[PASS] DD0015: are the binary symlinks installed?
[FAIL] DD0031: does the Docker API work? prereq failed: is the Docker engine running?
[PASS] DD0013: is the $PATH ok?
[FAIL] DD0003: is the Docker CLI working? prereq failed: is the Docker engine running?
[FAIL] DD0038: is the connection to Docker working? prereq failed: is the Docker engine running?
[PASS] DD0014: are the backend processes running?
[PASS] DD0007: is the backend responding?
[SKIP] DD0009: is the vpnkit API responding?
[PASS] DD0010: is the Docker API proxy responding?
[FAIL] DD0030: is the image access management authorized? prereq failed: is the Docker engine running?
[PASS] DD0033: does the host have Internet access?
[PASS] DD0018: does the host support virtualization?
[PASS] DD0001: is the application running?
[PASS] DD0017: can a VM be started?
[PASS] DD0016: is the LinuxKit VM running?
[WARN] DD0011: are the LinuxKit services running? failed to ping VM diagnosticsd with error: Get "http://ipc/ping": EOF
[WARN] DD0004: is the Docker engine running? prereq failed: are the LinuxKit services running?
[PASS] DD0015: are the binary symlinks installed?
[WARN] DD0031: does the Docker API work? prereq failed: is the Docker engine running?
[WARN] DD0032: do Docker networks overlap with host IPs? prereq failed: does the Docker API work?
segment 2023/11/08 08:41:59 ERROR: sending request - Post "https://api.segment.io/v1/batch": dial tcp [::]:443: connect: connection refused
segment 2023/11/08 08:41:59 ERROR: 1 messages dropped because they failed to be sent and the client was closed

Please note the following 4 warnings:

1 : The check: are the LinuxKit services running?
    Produced the following warning: failed to ping VM diagnosticsd with error: Get "http://ipc/ping": EOF

The Docker engine runs inside a Linux VM as a service. Therefore the services must have started.

2 : The check: is the Docker engine running?
    Produced the following warning: prereq failed: are the LinuxKit services running?

The Docker engine manages all containers and images on the host. Check the dockerd.log to see why it failed to start.

3 : The check: does the Docker API work?
    Produced the following warning: prereq failed: is the Docker engine running?

If the Docker API is not available from the host then Docker Desktop will not work correctly.

4 : The check: do Docker networks overlap with host IPs?
    Produced the following warning: prereq failed: does the Docker API work?

If the subnet used by a Docker network overlaps with an IP used by the host, then containers
won't be able to contact the overlapping IP addresses.

Try configuring the IP address range used by networks: in your docker-compose.yml.
See https://docs.docker.com/compose/compose-file/compose-file-v2/#ipv4_address-ipv6_address

Please investigate the following 1 issue:

1 : The test: are the LinuxKit services running?
    Failed with: failed to ping VM diagnosticsd with error: Get "http://ipc/ping": EOF

The Docker engine runs inside a Linux VM as a service. Therefore the services must have started.

[ctalledo]

Hi @yelvert, sorry to hear and thanks for reporting; could you please upload the diagnostics bundle so we can take a look. Thanks!

[yelvert]

@ctalledo 99BCA997-319D-42F5-A8E9-313C55500E06/20231108181124

[yelvert]

Downgrading to v4.24.2 seems to fix the issue. At least its gone a few hours without locking.