`docker login` and `docker pull` are abysmally slow (to start) in WSL 2

kallja commented 2 years ago

[x] I have tried with the latest version of Docker Desktop
[x] I have tried disabling enabled experimental features
[x] I have uploaded Diagnostics
Diagnostics ID: A324418D-5D19-41CF-B2BC-C7F37B376055/20221003093353

Actual behavior

In WSL2 (Ubuntu), certain Docker commands such as docker login and docker pull take a very long time before actually starting. The commands seem to hang for a long while before getting to their main business.

docker login hangs between 5 and 7 minutes to provide the username prompt,
docker pull hangs between 5 and 7 minutes before logging download of the first layer

Please note the above times are times it takes before the commands actually start doing what they're going to do. Once docker pull starts downloading layers, it performs it's remaining actions quickly as one would expect.

In the native Windows CLI the same commands perform in seconds, as expected.

The issue always manifests. What I've tried to rule out:

DNS. dig docker.io completes instantaniusly
Network conditions. Issue manifests in different networks, with and without VPN applications running in Windows.

Expected behavior

Docker CLI commands run in a timely manner.

Information

Windows Version:

Edition Windows 11 Pro
Version 22H2
Installed on    20/09/2022
OS build    22621.521
Experience  Windows Feature Experience Pack 1000.22634.1000.0

Docker Desktop Version: v.4.12.0
WSL2 or Hyper-V backend?: WSL2
Are you running inside a virtualized Windows e.g. on a cloud server or a VM: No.
I'm using a Windows account from Azure AD (probably root cause, I also tested using a local Windows user and the same issue manifests with that user, too)
- the user account is not an administrator on the local machine
- "C:\Program Files\Docker\Docker\resources\com.docker.diagnose.exe" check incorrectly reports that the account is not a member of docker-users. It is.
- docker commands work as expected under Windows.

As others have pointed out in https://github.com/docker/for-win/issues/10247#issuecomment-992477200, removing the "credsStore": "desktop.exe" fixes the performance but that hardly seems the ideal solution.

I've tried the steps described in https://github.com/docker/for-win/issues/10247#issuecomment-1152865542 but those didn't solve the issue for me.

Output of `& "C:\Program Files\Docker\Docker\resources\com.docker.diagnose.exe" check`

Steps to reproduce the behavior

Exact setup steps unknown.

kallja commented 2 years ago

Please, let me know if you have any specific data requests regarding this. I'll be happy to help to the best of my ability. Looking at #10247 it seems a fair amount of others are also encountering this.

kallja commented 2 years ago

I did some more pinpointing:

uninstalled my current Docker version
installed 3.0.4
- docker login -u and docker pull docker/getting-started performed quickly, 1 s (including pasting token) and 5 s respectively
Incrementally updated until a version was found that manifested the issue. This version is v3.2.0. Conveniently it's change log indicates a change to credStore.

thaJeztah commented 2 years ago

I think the fix for this is https://github.com/docker/docker-credential-helpers/pull/208, which is in the v0.7.0 release of the credential helpers, but may be used both by the docker-credentials-desktop.exe and by `docker-credentials-wincred.exe. I see there's a pull request "in flight" for Docker Desktop to update the version, but it's not merged yet (there were some review comments, but I'm checking with the Desktop team to see if there's things remaining before it can be merged).

Tails86 commented 2 years ago

I am also experiencing this slowness - docker login hangs, but sudo docker login executes immediately. More importantly for me, the "load metadata for" steps in build take between 5 and 20 minutes to complete, even when those images are already available locally. I don't have those issues when using 3.0.4.

EDIT: Removing credsStore value from settings also fixes these issues in 4.12.0 for me

thaJeztah commented 2 years ago

but sudo docker login executes immediately.

When running sudo, it's likely that docker looks for it's configuration in the home-directory of root, which may not be configured with a credshelper.

In either case, I see that the fix was merged in the Docker Desktop repository (private, so can't link to the pull request), so should be in the next release of Docker Desktop.

Tails86 commented 1 year ago

I still seem to be having this issue with 4.13

joshuadavidthomas commented 1 year ago

Same. I mentioned this in #10247:

I'm running in to this issue and it has ground my docker usage to a halt. The load metadata step stalls for multiple minutes, and it just started happening within the past week or so. This happens across the two container registries I use, the official DockerHub and the GitHub Container Registry. GHCR takes a while (~10 mins), but DockerHub takes AGES (~45 mins).

Environment:

OS: Windows 11 Enterprise, Version 22H2, Build 22621.674 Docker: Docker Desktop 4.13.0 (89412) with WSL2 backend WSL2: Ubuntu 22.04

I have tried many of the suggested workarounds in this issue, but none of them seem to resolve it for me.

Chuxel commented 1 year ago

Yeah, I think this is related, but what I am seeing is => resolve image config for docker.io/docker/dockerfile:1.4 is taking a massive amount of time even when everything is cached locally. Last run was 92.1s for this one step.

Removing "credsStore": "desktop.exe" from ~/.docker/config.json makes the problem go away.

This is with Docker Desktop 4.13.0 (89412), Windows 11 Enterprise Version 10.0.22621 Build 22621, in a Ubuntu 22.04 WSL instance.

ph342 commented 1 year ago

I had the same issue with Docker 4.14 and WSL2 Ubuntu 20.04.5. It appeared right after installing KB5019959 & KB5020030 & updating WSL. The only thing that worked for me was removing "credsStore": "desktop.exe". When will the fix be merged to Docker Desktop?

ph342 commented 1 year ago

Update: The workaround from https://github.com/docker/for-win/issues/10247#issuecomment-1152865542 does in fact work for me.

If you face issues with it, you're probably not doing it correctly: The symlink needs to be created in the $HOME/bin (not /bin) directory, which then needs to be added to path. This way, docker picks it up instead of /bin/docker-credential-desktop.exe.

Also, beware that the file names docker-credential-desktop.exe and docker-credential-wincred.exe differ.

docker-robott commented 1 year ago

There hasn't been any activity on this issue for a long time. If the problem is still relevant, mark the issue as fresh with a /remove-lifecycle stale comment. If not, this issue will be closed in 30 days.

Prevent issues from auto-closing with a /lifecycle frozen comment.

/lifecycle stale

ph342 commented 1 year ago

Shall we close this in favour of the other issue https://github.com/docker/for-win/issues/10247 ?

Tails86 commented 1 year ago

docker login now executes within 3 seconds for me on WSL2 without any workarounds. I am on Docker version 4.16.3.

EDIT: A coworker of mine reported the same thing. We haven't had this issue in a while, and we forgot all about this until I got an email from docker-robot.

docker-robott commented 1 year ago

Closed issues are locked after 30 days of inactivity. This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

/lifecycle locked

docker / for-win