gRPC FUSE and large projects cause docker compose stop to hang

jaspertandy commented 3 years ago

[x] I have tried with the latest version of Docker Desktop
[x] I have tried disabling enabled experimental features
[x] I have uploaded Diagnostics
Diagnostics ID: B27C3B61-CDC6-473A-AB9F-B25875986ADB/20210506135729

Expected behavior

On projects with a large number of services, I would expect docker compose stop to successfully stop all running containers for that project.

Actual behavior

docker compose stop appears to hang, containers can't be stopped at all by issuing specific docker compose stop SERVICENAME or docker stop CONTAINERNAME. The only way to stop the containers is to restart Docker for Mac.

Two things fix this issue:

Disabling gRPC FUSE (not sustainable in my opinion as it seems a lot faster and more stable than osxfuse)
Stopping containers one-by-one. Fish command: for srv in (docker compose ps | awk 'NR>1{print $2}'); docker compose stop $srv; end

Information

macOS Version: 11.3
Intel chip or Apple chip: Intel
Docker Desktop Version: 3.3.2

Steps to reproduce the behavior

To reproduce this issue, you can:

Clone this repository: https://github.com/jaspertandy/docker-compose-hang
Ensure that you have gRPC FUSE enabled.
Run docker compose up
Wait for all containers to start (sorry, there are lots of them!)
When all containers are running, issue docker compose stop and some containers will stop, but then they just hang on "stopping"

jaspertandy commented 3 years ago

Can confirm this issue persists on macOS 11.4 and Docker for Mac 3.3.3 (64133)

jaspertandy commented 3 years ago

Just to update that this now seems fixed in my example repo above on 3.4.0 (65384) but I am still able to reproduce on other real-world docker compose projects so I will update the repo above with another failing example.

I am still able to reliably stop containers in groups of 3 using the following fish shell script, if that helps at all:

function cs
    docker compose ps | awk 'NR>1{print $2}' | ruby -e '$stdin.read.split("\n").map(&:strip).each_slice(3).map { |b| b.join(" ") }.each { |b| `docker compose stop #{b}` }'
end

docker-robott commented 3 years ago

Issues go stale after 90 days of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

docker-robott commented 3 years ago

Closed issues are locked after 30 days of inactivity. This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle locked

docker / for-mac