search

coreos / coreos-kubernetes

CoreOS Container Linux+Kubernetes documentation & Vagrant installers
https://coreos.com/kubernetes/docs/latest/
Apache License 2.0
1.1k stars 465 forks source link

Docker not using the flannel IPs #884

Open jerrykan opened 7 years ago

jerrykan commented 7 years ago

In the documentation it says to create a drop-in containing the following:

[Service]
EnvironmentFile=/etc/kubernetes/cni/docker_opts_cni.env

And a /etc/kubernetes/cni/docker_opts_cni.env file containing the following:

DOCKER_OPT_BIP=""
DOCKER_OPT_IPMASQ=""

But it seems that these environment variables only seems nullify the environment variables in /var/run/flannel/flannel_docker_opts.env and cause docker to use its default IPs (ie. pods come up with a 172.17.0.x address. Commenting out the EnvironmentFile line in the drop-in and restarting docker causes the pods to then pick up a 10.2.x.x IP.

Is the documentation incorrect? or am I missing something?

page-fault-in-nonpaged-area commented 7 years ago

I believe the documentation is outdated. I'm having a lot of trouble with DNS following the official guide. Nothing seems to work.

jerrykan commented 7 years ago

Oops, I've just noticed this is a duplicate of #872, although with a different solution.

nesc58 commented 7 years ago

I've run into the same issue. Very disgusting.

My solution for CoreOS Cloud-Config. I deleted the docker_opts_cni.env and added this to the 40-flannel.conf. Maybe the service file could written in a better way but it works for me.

  - path: /etc/systemd/system/docker.service.d/40-flannel.conf
    permissions: 0644
    owner: root
    content: |
      [Unit]
      Requires=flanneld.service
      After=flanneld.service
      [Service]
      EnvironmentFile=/run/flannel/subnet.env
      ExecStartPre=export DOCKER_OPT_BIP="--bip=${FLANNEL_SUBNET}"

Maybe something like this is also necessary ExecStartPre=export DOCKER_OPT_IPMASQ="--ip-masq=${FLANNEL_IPMASQ}"

372046933 commented 5 years ago

After appending $DOCKER_NETWORK_OPTIONS which is defined in /run/flannel/docker to docker's systemd ExecStart, such as ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_NETWORK_OPTIONS . Now IP of docker0is correctly set.