Rebuild docker-compose devcontainer fails with removal of container is already in progress

[janhurst]

When trying to rebuild a dev container that is defined in a docker-compose file I consistently see an error about the removal of the previous build of the container failing as it is already in progress:

• VSCode Version: 1.77.1

• Local OS Version: Windows_NT x64 10.0.22624

• Remote OS Version: Debian

• Remote Extension/Connection Type: Containers

• Logs:

[2023-04-11T05:05:13.041Z] Setting up container for folder or workspace: /home/jan/anglo/mogn_eda
[2023-04-11T05:05:13.073Z] Start: Check Docker is running
[2023-04-11T05:05:13.073Z] Start: Run in Host: docker version --format {{.Server.APIVersion}}
[2023-04-11T05:05:13.150Z] Stop (77 ms): Run in Host: docker version --format {{.Server.APIVersion}}
[2023-04-11T05:05:13.150Z] Server API version: 1.41
[2023-04-11T05:05:13.150Z] Stop (77 ms): Check Docker is running
[2023-04-11T05:05:13.151Z] Start: Run in Host: docker inspect --type container 4021104bf03f85fa2d9f2cc08edf9d821b9601361fc8d2eee94211e02a58380e
[2023-04-11T05:05:13.211Z] Stop (60 ms): Run in Host: docker inspect --type container 4021104bf03f85fa2d9f2cc08edf9d821b9601361fc8d2eee94211e02a58380e
[2023-04-11T05:05:13.211Z] Start: Run in Host: docker rm -f 4021104bf03f85fa2d9f2cc08edf9d821b9601361fc8d2eee94211e02a58380e
[2023-04-11T05:05:13.274Z] Stop (63 ms): Run in Host: docker rm -f 4021104bf03f85fa2d9f2cc08edf9d821b9601361fc8d2eee94211e02a58380e
[2023-04-11T05:05:13.277Z] Command failed: docker rm -f 4021104bf03f85fa2d9f2cc08edf9d821b9601361fc8d2eee94211e02a58380e
[2023-04-11T05:05:13.277Z] Error response from daemon: removal of container 4021104bf03f85fa2d9f2cc08edf9d821b9601361fc8d2eee94211e02a58380e is already in progress
[2023-04-11T05:05:13.277Z] Exit code 1
[2023-04-11T05:08:03.730Z] Start: Run: wsl -d Ubuntu-22.04 -e wslpath -u \\wsl.localhost\Ubuntu-22.04\home\jan\anglo\mogn_eda
[2023-04-11T05:08:03.857Z] Stop (127 ms): Run: wsl -d Ubuntu-22.04 -e wslpath -u \\wsl.localhost\Ubuntu-22.04\home\jan\anglo\mogn_eda

Steps to Reproduce:

1. Run "Rebuild without cache and reopen in container" from the command pallette

[bamurtaugh]

Thanks for filing @janhurst. I think I experienced this briefly a few weeks ago, but I no longer see it in the latest VS Code Insiders and Dev Containers pre-release extension. Are you still experiencing this?

[janhurst]

Yes still seeing this every time I run a rebuild :( Here is an updated log dump if that helps... (from VS Code not Insiders)

[2023-04-26T02:38:08.942Z] Dev Containers 0.291.0 in VS Code 1.77.3 (704ed70d4fd1c6bd6342c436f1ede30d1cff4710).
[2023-04-26T02:38:08.942Z] Start: Run: wsl -d Ubuntu-22.04 -e wslpath -u \\wsl.localhost\Ubuntu-22.04\home\jan\anglo\mogn_eda
[2023-04-26T02:38:09.242Z] Stop (300 ms): Run: wsl -d Ubuntu-22.04 -e wslpath -u \\wsl.localhost\Ubuntu-22.04\home\jan\anglo\mogn_eda
[2023-04-26T02:38:09.252Z] Start: Resolving Remote
[2023-04-26T02:38:09.256Z] Start: Run: wsl -d Ubuntu-22.04 -e wslpath -u \\wsl.localhost\Ubuntu-22.04\home\jan\anglo\mogn_eda
[2023-04-26T02:38:09.470Z] Stop (214 ms): Run: wsl -d Ubuntu-22.04 -e wslpath -u \\wsl.localhost\Ubuntu-22.04\home\jan\anglo\mogn_eda
[2023-04-26T02:38:09.471Z] Start: Run: wsl -d Ubuntu-22.04 -e /bin/sh -c cd '/home/jan/anglo/mogn_eda' && /bin/sh
[2023-04-26T02:38:09.485Z] Start: Run in host: id -un
[2023-04-26T02:38:09.596Z] jan
[2023-04-26T02:38:09.596Z] 
[2023-04-26T02:38:09.596Z] Stop (111 ms): Run in host: id -un
[2023-04-26T02:38:09.597Z] Start: Run in host: cat /etc/passwd
[2023-04-26T02:38:09.598Z] Stop (1 ms): Run in host: cat /etc/passwd
[2023-04-26T02:38:09.598Z] Start: Run in host: echo ~
[2023-04-26T02:38:09.599Z] /home/jan
[2023-04-26T02:38:09.599Z] 
[2023-04-26T02:38:09.599Z] Stop (1 ms): Run in host: echo ~
[2023-04-26T02:38:09.600Z] Start: Run in host: test -x '/home/jan/.vscode-remote-containers/bin/704ed70d4fd1c6bd6342c436f1ede30d1cff4710/node'
[2023-04-26T02:38:09.600Z] 
[2023-04-26T02:38:09.601Z] 
[2023-04-26T02:38:09.601Z] Stop (1 ms): Run in host: test -x '/home/jan/.vscode-remote-containers/bin/704ed70d4fd1c6bd6342c436f1ede30d1cff4710/node'
[2023-04-26T02:38:09.601Z] Start: Run in host: test -f '/home/jan/.vscode-remote-containers/dist/vscode-remote-containers-server-0.291.0.js'
[2023-04-26T02:38:09.602Z] 
[2023-04-26T02:38:09.602Z] 
[2023-04-26T02:38:09.602Z] Stop (1 ms): Run in host: test -f '/home/jan/.vscode-remote-containers/dist/vscode-remote-containers-server-0.291.0.js'
[2023-04-26T02:38:09.604Z] userEnvProbe: loginInteractiveShell (default)
[2023-04-26T02:38:09.604Z] userEnvProbe: not found in cache
[2023-04-26T02:38:09.604Z] userEnvProbe shell: /usr/bin/zsh
[2023-04-26T02:38:10.554Z] userEnvProbe PATHs:
Probe:     '/opt/miniconda3/bin:/opt/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/Program Files/BellSoft/LibericaJDK-17/bin:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.4/bin:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.4/libnvvp:/mnt/c/Program Files (x86)/Common Files/Oracle/Java/javapath:/mnt/c/Program Files (x86)/Microsoft SDKs/Azure/CLI2/wbin:/mnt/c/Windows/system32:/mnt/c/Windows:/mnt/c/Windows/System32/Wbem:/mnt/c/Windows/System32/WindowsPowerShell/v1.0:/mnt/c/Windows/System32/OpenSSH:/mnt/c/ProgramData/chocolatey/bin:/mnt/c/Program Files/Microsoft VS Code/bin:/mnt/c/Program Files/dotnet:/mnt/c/Program Files/Microsoft SQL Server/Client SDK/ODBC/170/Tools/Binn:/mnt/c/Program Files (x86)/Microsoft SQL Server/150/Tools/Binn:/mnt/c/Program Files/Microsoft SQL Server/150/Tools/Binn:/mnt/c/Program Files/Microsoft SQL Server/150/DTS/Binn:/mnt/c/Program Files (x86)/Microsoft SQL Server/150/DTS/Binn:/mnt/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common:/mnt/c/Program Files (x86)/dotnet:/mnt/c/Program Files/NVIDIA Corporation/Nsight Compute 2021.2.2:/mnt/c/Program Files/NVIDIA Corporation/NVIDIA NvDLISR:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0:/mnt/c/WINDOWS/System32/OpenSSH:/mnt/c/Program Files (x86)/Pulse Secure/VC142.CRT/X64:/mnt/c/Program Files (x86)/Pulse Secure/VC142.CRT/X86:/mnt/c/Program Files (x86)/Common Files/Pulse Secure/TNC Client Plugin:/mnt/c/Program Files/nodejs:/mnt/c/Program Files/PuTTY:/mnt/c/Program Files/Git/cmd:/Docker/host/bin:/mnt/c/ProgramData/mambaforge/condabin:/mnt/c/Users/jan/.poetry/bin:/mnt/c/Users/jan/AppData/Local/Microsoft/WindowsApps:/mnt/c/tools/Cmder:/mnt/c/Users/jan/.dotnet/tools:/mnt/c/Program Files/Azure Data Studio/bin:/mnt/c/Program Files/Oracle/VirtualBox:/mnt/c/Users/jan/AppData/Local/Microsoft/WindowsApps:/mnt/c/Users/jan/.dotnet/tools:/mnt/c/Users/jan/AppData/Roaming/npm:/mnt/c/Users/jan/AppData/Local/Programs/oh-my-posh/bin:/mnt/c/Users/jan/AppData/Local/Programs/Microsoft VS Code Insiders/bin:~/.dotnet/tools'
Container: None
[2023-04-26T02:38:10.555Z] Setting up container for folder or workspace: /home/jan/anglo/mogn_eda
[2023-04-26T02:38:10.591Z] Start: Check Docker is running
[2023-04-26T02:38:10.591Z] Start: Run in Host: docker version --format {{.Server.APIVersion}}
[2023-04-26T02:38:10.706Z] Stop (115 ms): Run in Host: docker version --format {{.Server.APIVersion}}
[2023-04-26T02:38:10.706Z] Server API version: 1.41
[2023-04-26T02:38:10.706Z] Stop (115 ms): Check Docker is running
[2023-04-26T02:38:10.707Z] Start: Run in Host: docker inspect --type container 53d579b43fb652297252d519a4f69982954fd9c1613abc4b7fce42fb0ec38696
[2023-04-26T02:38:10.779Z] Stop (72 ms): Run in Host: docker inspect --type container 53d579b43fb652297252d519a4f69982954fd9c1613abc4b7fce42fb0ec38696
[2023-04-26T02:38:10.780Z] Start: Run in Host: docker rm -f 53d579b43fb652297252d519a4f69982954fd9c1613abc4b7fce42fb0ec38696
[2023-04-26T02:38:10.855Z] Stop (75 ms): Run in Host: docker rm -f 53d579b43fb652297252d519a4f69982954fd9c1613abc4b7fce42fb0ec38696
[2023-04-26T02:38:10.859Z] Command failed: docker rm -f 53d579b43fb652297252d519a4f69982954fd9c1613abc4b7fce42fb0ec38696
[2023-04-26T02:38:10.859Z] Error response from daemon: removal of container 53d579b43fb652297252d519a4f69982954fd9c1613abc4b7fce42fb0ec38696 is already in progress
[2023-04-26T02:38:10.859Z] Exit code 1
[2023-04-26T02:39:28.996Z] Start: Run: wsl -d Ubuntu-22.04 -e wslpath -u \\wsl.localhost\Ubuntu-22.04\home\jan\anglo\mogn_eda
[2023-04-26T02:39:29.175Z] Stop (179 ms): Run: wsl -d Ubuntu-22.04 -e wslpath -u \\wsl.localhost\Ubuntu-22.04\home\jan\anglo\mogn_eda

[zifik]

I get this same error when rebuilding my container locally on Windows with WSL. However, when rebuilding a container running on a remote Linux machine, I never seen the issue.

[mmaisonette]

I am facing the same issue on both Windows WSL and MacOS. It looks like some timeout happening on the Docker daemon.

[2023-08-24T12:25:44.224Z] Start: Check Docker is running [2023-08-24T12:25:44.224Z] Start: Run: docker version --format {{.Server.APIVersion}} [2023-08-24T12:25:44.348Z] Stop (124 ms): Run: docker version --format {{.Server.APIVersion}} [2023-08-24T12:25:44.348Z] Server API version: 1.41 [2023-08-24T12:25:44.348Z] Stop (124 ms): Check Docker is running [2023-08-24T12:25:44.349Z] Start: Run: docker volume ls -q [2023-08-24T12:25:44.418Z] Stop (69 ms): Run: docker volume ls -q [2023-08-24T12:25:44.419Z] Start: Run: docker inspect --type container 9d97eb36d2fe6dbf983e9b73cf5b70dd5a2cb363d4a0081fb179ad947e1ceeff [2023-08-24T12:25:44.497Z] Stop (78 ms): Run: docker inspect --type container 9d97eb36d2fe6dbf983e9b73cf5b70dd5a2cb363d4a0081fb179ad947e1ceeff [2023-08-24T12:25:44.498Z] Start: Run: docker rm -f 9d97eb36d2fe6dbf983e9b73cf5b70dd5a2cb363d4a0081fb179ad947e1ceeff [2023-08-24T12:25:44.572Z] Stop (74 ms): Run: docker rm -f 9d97eb36d2fe6dbf983e9b73cf5b70dd5a2cb363d4a0081fb179ad947e1ceeff [2023-08-24T12:25:44.574Z] Command failed: docker rm -f 9d97eb36d2fe6dbf983e9b73cf5b70dd5a2cb363d4a0081fb179ad947e1ceeff [2023-08-24T12:25:44.574Z] Error response from daemon: removal of container 9d97eb36d2fe6dbf983e9b73cf5b70dd5a2cb363d4a0081fb179ad947e1ceeff is already in progress [2023-08-24T12:25:44.574Z] Exit code 1`

Any suggestions?

PS.: The retry command always works over here as well.

[janhurst]

@chrmarti is there anything I can try to provide to help diagnose this?

[chrmarti]

Thanks @janhurst, I previously looked into this and it seems we might have to detect this case and wait for the container to go away.

[DavidS-ovm]

I hit this every time, running VSCode natively on Debian.

[jasonmacdonald]

Any update on this? We are hitting this all the time. Whenever anyone on our team does a rebuild.

[ryanhiebert]

Sorry for spamming two different tickets, but I concur that this seems to be a race condition, and that waiting for the container to go away seems like a likely solve to me.

[GaTechThomas]

I'm seeing this as well. Seems to be on some, but not all, types of devcontainers. I'll report back if I can pinpoint it further.

[stewartadam]

Another tidbit - I see this on devcontainers using image, so it's not necessarily related to docker-compose.

[3f6a]

I am also hitting this. So I have to reopen vscode and rebuild a second time to fix it. Seems like a bug?

Also seems related: https://github.com/microsoft/vscode-remote-release/issues/6509

[newbenji]

Still an issue for me also. retry nearly always works

[g-kartik]

I am facing also this issue. OS - Linux VSCode: 1.87.2 863d2581ecda6849923a2118d93a088b0745d9d6 x64

[chrmarti]

Continuing in https://github.com/microsoft/vscode-remote-release/issues/6509.