search

docker / for-win

Bug reports for Docker Desktop for Windows
https://www.docker.com/products/docker#/windows
1.84k stars 281 forks source link

Can not access local network, with kubernetes installed #1667

Open ghost opened 6 years ago

ghost commented 6 years ago

The kubernetes will create cni0 network, and insert a route table

10.1.0.0

but our local network was defined with 10.1.0.0.

so we can not access our local network.

such as docker login 10.1.41.29:5555

will failed.

would you please allow us to set the kubernetes network range?

guillaumerose commented 6 years ago

For the moment, there is no parameter for that but here is a work-around.

  1. Disable Kubernetes
  2. docker run -it --privileged --pid=host justincormack/nsenter1 and edit /var/lib/cni/conf/10-default.conflist. Change 10.1.0.0/16 mention by the network you want. Don't forget the gateway and the dns.
  3. Enable Kubernetes

Let me know if it works!

ghost commented 6 years ago

Thanks a lot

cdenneen commented 6 years ago

@guillaumerose is there anyway this could be added as UI option or maybe some other network that might not conflict as much as 10.1.0.0 default? Also what causes this to reset? I ran into this issue again today and totally forgot what I did to fix it until I managed to find this issue via Google ;-)

guillaumerose commented 6 years ago

I think the reset is caused by a reset of the images disk or an upgrade.

We need to add this as a parameter, we are just running out of time.

docker-robott commented 5 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

cdenneen commented 5 years ago

Is there anyway to get this added. I keep k8s disabled as manually fixing after each Update has been frustrating but would really like to use k8s locally.

/lifecycle frozen

cdenneen commented 5 years ago

@guillaumerose this is still an issue with latest build when Kubernetes is started... if I disable Kubernetes and restart docker I can access local network... if I make the modifications you suggested above it works as well but after Docker gets updated, you must fix this again manually.

refractalize commented 5 years ago

👍 for a fix to this issue. We also have a local network with IPs in the 10.1.x.x range.

mblaschke commented 5 years ago

I can confirm the issue also, please add an setting to allow a different range.

cdenneen commented 5 years ago

@netroby why is this closed? Is there another issue or corresponding PR that closes this? This is a huge issue.

ghost commented 5 years ago

@cdenneen I gave up . I do not use docker for windows to deploy my kubernetes. I am now using kubeadm setup my kubernetes.

cdenneen commented 5 years ago

This is a larger issue. Could you please reopen?

ghost commented 5 years ago

@cdenneen reopen it as you request.

refractalize commented 5 years ago

Hi all, I just want to point out that I'm seeing this issue in Docker for Mac, so this issue is not at all specific to Docker for Windows.

pselden commented 5 years ago

Hit this same issue today.

cdenneen commented 5 years ago

@refractalize yes I have linked the issue back to one in docker-for-mac as well to hopefully gain traction for either side but so far no momentum from either direction. I just wanted to keep the issue open so it's visible from both projects as outstanding (thanks for that @netroby)

refractalize commented 5 years ago

hi @cdenneen yes i noticed that just after my post ;)

cheers

cdenneen commented 5 years ago

@guillaumerose Is there any traction on this issue? Maybe a docker upstream issue number you can link to that we can subscribe to?

cdenneen commented 5 years ago

/remove-lifecycle stale

stevenpall commented 5 years ago

@guillaumerose To echo the above comment, this is still an issue, and as far as I can tell there isn't a reasonable and repeatable solution to fix it. Has there been any progress (internal or otherwise) to allow users to set the CNI bridge subnet to something other than 10.1.0.0/16? Thanks!

cdenneen commented 5 years ago

@guillaumerose do you have tracking for this feature? I know you mentioned you were going to bring it up but now with 2.x release still curious where this stands on the backlog?

gurkanindibay commented 4 years ago

I changed the address to 192.168.1.0 and gateway and dns to 192.168.1.1. However, kubernetes didn't started. So it doesn't worked for me. I changed back my configuration to original values to make kubernetes work.

cdenneen commented 4 years ago

The network settings aren't the fix for Kubernetes.

Steps to fix this are described here:

https://github.com/docker/for-win/issues/1667#issuecomment-367357067

zolij commented 4 years ago

The network settings aren't the fix for Kubernetes.

Steps to fix this are described here:

#1667 (comment)

That file is reverted to the default if you restart docker daemon, so it's not a solution.

I can't understand how can this issue still exists, it's a major problem on larger corporate networks.

cdenneen commented 4 years ago

@zolij I was told this should be fixed in the "2.x beta" a year ago but still don't see it in the UI to account for this yet.

cdenneen commented 4 years ago

Any updates?

cdenneen commented 4 years ago

/remove-lifecycle frozen

chriswiggins commented 4 years ago

I just ran into this issue now too. The fact that it's 10.1.0.0/16 too means it covers a very wide set of addresses

docker-robott commented 4 years ago

Issues go stale after 90 days of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

zolij commented 4 years ago

/remove-lifecycle stale

zolij commented 4 years ago

I think it's not only a feature request but a bug too. This issue on corporate networks can be very problematic.

ozbillwang commented 3 years ago

@guillaumerose

Your solution is the only one work in my corp's VPN currently.

But seems the setting is not sticky. After restart my laptop or restart docker service, the setting in /var/lib/cni/conf/10-default.conflist is gone. I have to manually update that file again.

Do you know if there are any ways to make the change permentantly?

docker-robott commented 3 years ago

Issues go stale after 90 days of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

zolij commented 3 years ago

/remove-lifecycle stale

zolij commented 3 years ago

/lifecycle frozen

ChBerrich commented 3 years ago

Any updates, still blocking on version 3.1.0.

ChBerrich commented 3 years ago

Always same issue on 3.2.1, any plan to fix it ?

pmorch commented 2 years ago

@guillaumerose - thank you for you workaround it mostly works great. A couple of points:

  1. The change to /var/lib/cni/conf/10-default.conflist reverts back to factory defaults of 10.1.0.0/16 every time docker is restarted, e.g. when Windows is restarted. Which is quite annoying!
  2. After a docker restart, when kubernetes is running with 10.1.0.0/16 again, I had to inject a step between steps 1 and 2. I have to restart docker once more after disabling kubernetes, or kubernetes won't start after modifying 10-default.conflist.

Thank you for showing me/us justincormack/nsenter1: simple nsenter to namespaces of pid 1. I've finally found where the complicated networking setup for docker and kubernetes is going on. I've been looking everywhere for it. But where is that VM visible and where are the details of that VM configured? The VM doesn't show up in Hyper-V Manager, in powershell.exes Get-VM and also not in wsl.exe -l (because surely it ain't the docker-desktop WSL2 distribution). It is a "hidden" VM that is only accessible with nsenter like the justincormack/nsenter1 docker container does it? Which image is it running?

Looking at the VM accessed by nsenter1, I see that /var/lib/cni/conf/10-default.conflist is stored in /dev/sdf:

# mount | grep /var/lib | egrep -v '/var/lib/(docker|kubelet)'
/dev/sdf on /var/lib type ext4 (rw,relatime,discard,errors=remount-ro,data=ordered)

(confusingly, that was /dev/sdd a couple of hours ago)

But that mount line could suggest that it should be persistent and survive reboots (docker restarts). Where is /dev/sdf recreated on every boot/docker restart - or why isn't /dev/sdf actually persistent?

Also, just for info, there is a related networking - Change Kubernetes docker-for-desktop cluster network ip - Stack Overflow post where I've been a little more elaborate and have modified your workaround to include restarting docker (again).

EduardMiller commented 2 years ago

The following solution worked for me (Docker Desktop v4.1.1). I have made the following adjustments in the file %APPDATA%\Docker\cni\10-default.conflist ... "ipam": { "type": "host-local", "subnet": "10.250.0.0/16", "gateway": "10.250.0.1" }, "dns": { "nameservers": ["10.250.0.1"] } ...

After restarting Docker, I can access the local VPN network inside a container without problems. This also works after restarting the PC.

ngraf commented 2 years ago

@edu

The following solution worked for me (Docker Desktop v4.1.1). I have made the following adjustments in the file %APPDATA%\Docker\cni\10-default.conflist

This works for me as well in "Docker for Desktop" for Mac. Mac path is ~/Library/Group Containers/group.com.docker/cni/10-default.conflist

It would be convenient to be able to change CNI settings in the UI of "Docker for Desktop" as part of the "Preferences".

chrisbecke commented 1 year ago

There is still no GUI setting for this in 2022-11?

OpenPj commented 2 months ago

I upgraded Docker Desktop on Mac some days ago and I spent many hours to solve this problem. Thank you so much for documenting this but It would be great to have something that can be configured in the GUI.

drichards-clgx commented 2 weeks ago

Using 4.31.1 on Windows w/WSL I am able to update the config to change the Pod network, but the Serivces are still on the default network, is it possible to change the Services network?

Docker Engine:

{
  "bip": "172.25.1.1/24",
  "builder": {
    "gc": {
      "defaultKeepStorage": "20GB",
      "enabled": true
    }
  },
  "default-address-pools": [
    {
      "base": "172.25.2.1/24",
      "size": 28
    }
  ],
  "experimental": false,
  "fixed-cidr": "172.25.1.1/25"
}

10-default.conflist

{
    "cniVersion": "0.3.1",
    "name": "default",
    "plugins": [
        {
        "type": "bridge",
        "bridge": "cni0",
        "isDefaultGateway": true,
        "ipMasq": true,
        "hairpinMode": true,
        "ipam": {
            "type": "host-local",
            "subnet": "172.20.0.0/16",
            "gateway": "172.20.0.1"
        },
        "dns": {
            "nameservers": ["172.20.0.1"]
        }
        },
        {
        "type": "portmap",
        "capabilities": {
            "portMappings": true
        },
        "snat": true
        }
    ]
}

Pods:

NAME                            READY   STATUS    RESTARTS   AGE     IP            NODE             NOMINATED NODE   READINESS GATES
seas-ibm-seas-0                 1/1     Running   0          3d23h   172.20.0.43   docker-desktop   <none>           <none>
ssp-cm-ibm-ssp-cm-0             1/1     Running   0          4d23h   172.20.0.24   docker-desktop   <none>           <none>
ssp-engine-1-ibm-ssp-engine-0   1/1     Running   0          4d23h   172.20.0.15   docker-desktop   <none>           <none>

Services:

NAME                            TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                        AGE
seas-ibm-seas                   ClusterIP      10.100.219.74   <none>        61365/TCP,61366/TCP,9080/TCP   3d23h
ssp-cm-ibm-ssp-cm               ClusterIP      10.107.15.155   <none>        8443/TCP,62366/TCP             4d23h
ssp-engine-1-ibm-ssp-engine     ClusterIP      10.96.66.72     <none>        63366/TCP,63388/TCP            4d23h

How do I change the addresses of the services as they conflict with the corporate network.

riverfell commented 1 week ago

Using 4.31.1 on Windows w/WSL I am able to update the config to change the Pod network, but the Serivces are still on the default network, is it possible to change the Services network?

Docker Engine:

{
  "bip": "172.25.1.1/24",
  "builder": {
    "gc": {
      "defaultKeepStorage": "20GB",
      "enabled": true
    }
  },
  "default-address-pools": [
    {
      "base": "172.25.2.1/24",
      "size": 28
    }
  ],
  "experimental": false,
  "fixed-cidr": "172.25.1.1/25"
}

10-default.conflist

{
  "cniVersion": "0.3.1",
  "name": "default",
  "plugins": [
      {
      "type": "bridge",
      "bridge": "cni0",
      "isDefaultGateway": true,
      "ipMasq": true,
      "hairpinMode": true,
      "ipam": {
          "type": "host-local",
          "subnet": "172.20.0.0/16",
          "gateway": "172.20.0.1"
      },
      "dns": {
          "nameservers": ["172.20.0.1"]
      }
      },
      {
      "type": "portmap",
      "capabilities": {
          "portMappings": true
      },
      "snat": true
      }
  ]
}

Pods:

NAME                            READY   STATUS    RESTARTS   AGE     IP            NODE             NOMINATED NODE   READINESS GATES
seas-ibm-seas-0                 1/1     Running   0          3d23h   172.20.0.43   docker-desktop   <none>           <none>
ssp-cm-ibm-ssp-cm-0             1/1     Running   0          4d23h   172.20.0.24   docker-desktop   <none>           <none>
ssp-engine-1-ibm-ssp-engine-0   1/1     Running   0          4d23h   172.20.0.15   docker-desktop   <none>           <none>

Services:

NAME                            TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                        AGE
seas-ibm-seas                   ClusterIP      10.100.219.74   <none>        61365/TCP,61366/TCP,9080/TCP   3d23h
ssp-cm-ibm-ssp-cm               ClusterIP      10.107.15.155   <none>        8443/TCP,62366/TCP             4d23h
ssp-engine-1-ibm-ssp-engine     ClusterIP      10.96.66.72     <none>        63366/TCP,63388/TCP            4d23h

How do I change the addresses of the services as they conflict with the corporate network.

4.31.0 on mac. The conflist keeps revert every time when k8s restarts. So i tried 

sudo chmod 555 ~/Library/Group\ Containers/group.com.docker/cni/10-default.conflist

After restart docker service. It works.

drichards-clgx commented 12 hours ago

How do I change the addresses of the services as they conflict with the corporate network.

4.31.0 on mac. The conflist keeps revert every time when k8s restarts. So i tried

sudo chmod 555 ~/Library/Group\ Containers/group.com.docker/cni/10-default.conflist

After restart docker service. It works.

My conflist does not revert, but I tried making it read-only anyway but still the services are in the wrong CIDR