docker / for-win

Bug reports for Docker Desktop for Windows
https://www.docker.com/products/docker#/windows
1.87k stars 289 forks source link

Cannot connect from WSL2 to Docker Desktop Kubernetes via kubectl #13397

Open RemekGdansk opened 1 year ago

RemekGdansk commented 1 year ago

Actual behavior

I cannot connect from WSL 2 (Ubuntu) to Docker Desktop Kubernnetes via kubectl. I can connect via kubectl from PowerShell.

Expected behavior

I should be able to connect via kubectl from WSL2 as well as from PowerShell.

Information

Output of & "C:\Program Files\Docker\Docker\resources\com.docker.diagnose.exe" check

I have provided the Diagnostics ID, the information should be in that report.

Steps to reproduce the behavior

Docker & Kubernetes on Docker Desktop are working correctly. The problem is to connect via kubectl from WSL2. Example:

$ kubectl cluster-info

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
The connection to the server kubernetes.docker.internal:6443 was refused - did you specify the right host or port?

The same command issued from PowerShell works:

PS kubectl cluster-info
Kubernetes control plane is running at https://kubernetes.docker.internal:6443
CoreDNS is running at https://kubernetes.docker.internal:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

When I uninstall Docker for Windows correctly, remove all .cache folders & kubeconfigs, after fresh install the connection is working. However, after the computer restart, the problem reappears.

Edit: I am providing full information on the versions of WSL, Docker and Kubernetes (mine are the same as those provided by @BarclayHill in the comments below):

PS C:\> wsl --version
WSL version: 1.2.0.0
Kernel version: 5.15.90.1
WSLg version: 1.0.51
MSRDC version: 1.2.3770
Direct3D version: 1.608.2-61064218
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22621.1555

PS C:\> docker --version
Docker version 20.10.24, build 297e128

PS C:\> kubectl version --short
Client Version: v1.25.4
Kustomize Version: v4.5.7
Server Version: v1.25.4
brettcurtis commented 1 year ago

I have the same problem. Initially, I found this: https://github.com/docker/for-win/issues/13271. The .wlsconfig "fix" didn't work for me, and it looks like the OP upgraded. I'm fully up to date on the latest stable windows and WSL2.

edit I ran upstairs to test on my desktop that is running Insider beta, and it works fine.

BarclayHill commented 1 year ago

Ran into this same issue 2 days ago and could not figure out but found this thread and followed it.

Today I tried connecting to to local K8s running on docker from WSL2 Ubuntu via kubectl and found it be working which was surprising.

Nothing changed other than a few reboots and caching my MSFT Acct Creds locally to resolve RDP issue from macOS.

fedenusy commented 1 year ago

Same issue here.

Rebooting my machine did not help.

Uninstalling, deleting ~/.docker + ~/.kube, then reinstalling also did not help.

@RemekGdansk do you happen to remember the list of files/folders that you needed to delete before reinstalling?

lordpayder commented 1 year ago

Got the same issue on a fresh install on Windows 11 with wsl2. It's working fine from the host (VS Code Kubernetes Addon, Powershell) but fails to connect from the debian distribution running on wsl2. Docker itself works as expected.

SoerenHenning commented 1 year ago

I experienced this issue twice within this week, both times after rebooting Windows. Only re-installing Docker for Desktop and re-installing WSL solved the issue for me.

BarclayHill commented 1 year ago

Haven't hit the issue recently since last week. Applicable versions of what I'm running to help others.

docker --version
Docker version 20.10.24, build 297e128

wsl --version
WSL version: 1.2.0.0
Kernel version: 5.15.90.1
WSLg version: 1.0.51
MSRDC version: 1.2.3770
Direct3D version: 1.608.2-61064218
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22621.1555

kubectl version --short
Flag --short has been deprecated, and will be removed in the future. The --short output will become the default.
Client Version: v1.25.4
Kustomize Version: v4.5.7
Server Version: v1.25.4
SoerenHenning commented 1 year ago

I'm getting exactly the same responses as @BarclayHill, both on Win11 and WSL Ubuntu, except for kubectl in WSL:

$ kubectl version --short
Flag --short has been deprecated, and will be removed in the future. The --short output will become the default.
Client Version: v1.25.4
Kustomize Version: v4.5.7
The connection to the server kubernetes.docker.internal:6443 was refused - did you specify the right host or port?

To summarize:

Docker Kubernetes
Windows 11
WSL Ubuntu

I once again tried uninstalling Docker Desktop, deleting .docker, AppData\Local\Docker and AppData\Roaming\Docker from the user directory and re-installing Docker Desktop + its Kubernetes, but the issue remains.

8o8inCodes commented 1 year ago

Same issue, this started happening after I've updated the wsl using wsl.exe --update

kubectl get all
The connection to the server kubernetes.docker.internal:6443 was refused - did you specify the right host or port?
RemekGdansk commented 1 year ago

Same issue here.

Rebooting my machine did not help.

Uninstalling, deleting ~/.docker + ~/.kube, then reinstalling also did not help.

@RemekGdansk do you happen to remember the list of files/folders that you needed to delete before reinstalling?

Hi @fedenusy

Not that I remember exactly, but most probably removed the following (my usual list when I need to do a hard-reset on Docker):

Windows

WSL

BarclayHill commented 1 year ago

The last post reminded me of something I did originally when I first encountered the issue. Not sure this will help others or explain why my setup is working.

Stop Docker Desktop

$ cd /home/<user>/.kube
~/.kube$ cp config config.back
~/.kube$ cp /mnt/c/users/<user>/.kube/config config

Start Docker Desktop

~/.kube$ kubectl get all
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   5d2h
RemekGdansk commented 1 year ago

Thank you @BarclayHill for your comment. The solution did not work for me, however.

The only difference I had between the config in Windows and the config in WSL were in properties:

Do you still have your old config from WSL (the one renamed to config.back)? Diff between the not working and working config could provide some insight.

BarclayHill commented 1 year ago

@RemekGdansk - No, sorry, I didn't keep it around or compare them. Like I said in the original post. I could not resolve the issue even copying the config from the windows side. It was few days later when I noticed it was working as expected. What changed between was only the reboots and running the following command to resolve an unrelated RDP issue.

PS> runas /u:MicrosoftAccount\<msftaccount> winver
RemekGdansk commented 1 year ago

I uninstalled & reinstalled Docker Desktop in order to compare changes between kube configs and this time I could not connect from WSL using kubectl after reinstallation. So it seems that the previous success with uninstall/reinstall was rather random and not a reproducible way to prevent the problem even for a short time.

fedenusy commented 1 year ago

I just tried a factory reset of Windows 11 (that's how badly I needed to get WSL k8s working...)

...and after reinstalling everything from scratch, WSL k8s worked!

Now going to buy a UPS-- not taking any chances of a reboot.

8o8inCodes commented 1 year ago

I just tried a factory reset of Windows 11 (that's how badly I needed to get WSL k8s working...)

...and after reinstalling everything from scratch, WSL k8s worked!

Now going to buy a UPS-- not taking any chances of a reboot.

lol, reconfiguring everything is gonna take ages for me.. we need to find a solution without a full reset

brettcurtis commented 1 year ago

I'm curious, are all the folks who are struggling on stable windows release vs. an insider build? It works fine on my insider build but not stable.

fedenusy commented 1 year ago

I'm curious, are all the folks who are struggling on stable windows release vs. an insider build? It works fine on my insider build but not stable.

stable

tuncaytunc-zf commented 1 year ago

I have the same issue, it suddenly stopped working. Could solve it by adding following lines in .wslconfig

[wsl2]
localhostForwarding=false

But if I do so and do a port forwarding within wsl against my local Kubernetes cluster in Docker Desktop, I can't reach the port anymore from my host outside from wsl.

RemekGdansk commented 1 year ago

Thanks @tuncaytunc-zf , it worked on my side as well!

To all readers of this thread - .wslconfig file is located in your Windows user directory. If you don't have the file already, you need to create it.

I am leaving the thread open, it is only a workaround (unless it is going to be the official Docker Desktop instruction to do so).

zevisert commented 1 year ago

I had this issue last week. IIRC I went only far as uninstalling docker desktop and deleting C:\Users\<me>\.docker\ and restarting before reinstalling docker desktop.

The issue for me really seemed related to localhostForwarding (except I think mine works when true, lol). I'd recommend people check if they have a port 6443 open in the docker-desktop WSL backend PS> wsl -d docker-desktop -- netstat -tulpn. If that's there, but not in your default WSL distrobution (PS> wsl -- netstat -tulpn) then something is up with port forwarding.

8o8inCodes commented 1 year ago

I have the same issue, it suddenly stopped working. Could solve it by adding following lines in .wslconfig

[wsl2]
localhostForwarding=false

But if I do so and do a port forwarding within wsl against my local Kubernetes cluster in Docker Desktop, I can't reach the port anymore from my host outside from wsl.

Ok that works, however that blocks the access to my servers that are running under wsl..

ivanvmoreno commented 1 year ago

I can also replicate this issue on version 22H2, build 22621.1555.

BarclayHill commented 1 year ago

Windows 11 Docker/K8s and WSL Transient Issue Notes

My machine bug checked last night and this morning this issue returned.

Here is how I worked around the issue and got it going again.

It's nonsense like this thats defeating my interest of working in a Windows enviroment again.


 $ kubectl get all
E0421 07:52:45.150246     237 memcache.go:265] couldnt get current server API group list: Get "https://kubernetes.docker.internal:6443/api?timeout=32s": dial tcp 127.0.0.1:6443: connect: connection refused

WTF!!!!

$ exit
PS > wsl --list -v
  NAME                   STATE           VERSION
* Ubuntu                 Running         2
  docker-desktop         Running         2
  docker-desktop-data    Running         2
PS > wsl --shutdown
PS > wsl --list -v
  NAME                   STATE           VERSION
* Ubuntu                 Running         2
  docker-desktop         Stopped         2
  docker-desktop-data    Stopped         2
PS > wsl --shutdown ubuntu
PS > wsl --list -v
  NAME                   STATE           VERSION
* Ubuntu                 Stopped         2
  docker-desktop         Stopped         2
  docker-desktop-data    Stopped         2

Think for a second and regret messing with Windows11. Wait, I bet this has something to do with something running post bug check that I shutdown previously trying to get this resolved.

Sort through event log looking at events occurriung after bug check. Aha! Hyper-V has started after bug check doing something with a default network switch called "WSL". Make note to uninstall Hyper-V nonsense for later.

Look in running processes and find orphaned docker desktop client and kill it. Shutdown Hyper-V Service...Start Docker Desktop. Jump back to shell.


PS > wsl --list -v
  NAME                   STATE           VERSION
* Ubuntu                 Running         2
  docker-desktop         Running         2
  docker-desktop-data    Running         2
PS > kubectl get all
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   6d20h
PS > wsl
$ kubectl get all
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   6d20h

Shut down RDP and go back to macOS to get some actual work done.

robertwenquan commented 1 year ago

Ran into this issue with all matched version from the issue.

The .wlsconfig "fix" didn't work for me either and currently haven't found a workaround for this.

wixregiga commented 1 year ago

Microsoft should make their Docker Desktop. Issues like this and issues like losing all of my settings whenever Code or DD updates is too much.

nagaem commented 1 year ago

I have the same issue, it suddenly stopped working. Could solve it by adding following lines in .wslconfig

[wsl2]
localhostForwarding=false

But if I do so and do a port forwarding within wsl against my local Kubernetes cluster in Docker Desktop, I can't reach the port anymore from my host outside from wsl.

This is the only solution that has consistently worked for me - however, I need the localhost forwarding! Are there any workarounds that will allow me access to servers running on wsl with this turned off?

tuncaytunc-zf commented 1 year ago

I have the same issue, it suddenly stopped working. Could solve it by adding following lines in .wslconfig

[wsl2]
localhostForwarding=false

But if I do so and do a port forwarding within wsl against my local Kubernetes cluster in Docker Desktop, I can't reach the port anymore from my host outside from wsl.

This is the only solution that has consistently worked for me - however, I need the localhost forwarding! Are there any workarounds that will allow me access to servers running on wsl with this turned off?

Yes, there is a workaround for that. You can do the port forwarding from your host (not in wsl), then you should be able to access the ports from your host. Of course you need the kubectl installed on your host, too.

brettcurtis commented 1 year ago

Today's windows update seemed to fix this for me: image

sevos commented 4 months ago

Adding kubernetes.docker.internal host in /etc/hosts seems to temporarily fix it for me

127.0.0.1     localhost kubernetes.docker.internal
alahane-techtel commented 4 months ago

this issue is haunting me too. Any resolution please? above /etc/hosts fix doesnt work for me :( No point in trying other year old comments.

Have you all moved to linux already ;)

zazulam commented 1 month ago

What worked for me was adding this to my .wslconfig.

networkingMode=mirrored

I was able to successfully connect to my portforwards after that. Also here is my /etc/hosts as well jic.

127.0.0.1       localhost
127.0.1.1       DESKTOP

192.168.1.34    host.docker.internal
192.168.1.34    gateway.docker.internal
127.0.0.1       kubernetes.docker.internal

Note: I'm running this on Windows 10 with Docker version 26.1.1