Unable to create a pod on kubernetes cluster using weave

[leodotcloud]

Setup details: Three vagrant machines. 1) kubernetes master --> running etcd, apiserver, scheduler, controller-manager 2) kubernetes host h1 --> weave launch 3) kubernetes host h2 --> weave launch

OS/Environment:

• Docker 1.10.3.
• Ubuntu trusty 14.04
• Weave 1.6

Command run: ./kubectl create -f simple_nginx_pod.yml

Expected result: The pods get created successfully.

Error:

E0627 21:34:43.691382   24255 cni.go:167] Error adding network: Bridge "weave" has no IP addresses
E0627 21:34:43.691601   24255 cni.go:118] Error while adding to cni network: Bridge "weave" has no IP addresses
E0627 21:34:43.691932   24255 manager.go:1874] Failed to setup network for pod "nginx_default(f6366379-3cae-11e6-818d-08002707be12)" using network plugins "cni": Bridge "weave" has no IP addresses; Skipping pod
E0627 21:34:43.703957   24255 pod_workers.go:138] Error syncing pod f6366379-3cae-11e6-818d-08002707be12, skipping: failed to "SetupNetwork" for "nginx_default" with SetupNetworkError: "Failed to setup network for pod \"nginx_default(f6366379-3cae-11e6-818d-08002707be12)\" using network plugins \"cni\": Bridge \"weave\" has no IP addresses; Skipping pod"

I followed the exact steps mentioned in the docs to start weave.

vagrant@ubuntuh1:~$ sudo docker ps -a
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS              PORTS               NAMES
44e0739cbdeb        weaveworks/plugin:1.6.0      "/home/weave/plugin"     7 minutes ago       Up 1 seconds                            weaveplugin
2e2f1749e906        weaveworks/weaveexec:1.6.0   "/home/weave/weavepro"   7 minutes ago       Up 1 seconds                            weave proxy
02c8b683d9e6        weaveworks/weaveexec:1.6.0   "/bin/false"             7 minutes ago       Created                                 weavevolumes-1.6.0
84604b7f9839        weaveworks/weave:1.6.0       "/home/weave/weaver -"   7 minutes ago       Up 1 seconds                            weave
16cf231f83bb        weaveworks/weavedb           "data-only"              7 minutes ago       Created                                 waved
vagrant@ubuntuh1:~$
vagrant@ubuntuh1:~$ sudo weave status

        Version: 1.6.0 (up to date; next check at 2016/06/28 02:44:16)

        Service: router
       Protocol: weave 1..2
           Name: ae:10:77:d2:aa:77(ubuntuh1)
     Encryption: disabled
  PeerDiscovery: enabled
        Targets: 1
    Connections: 1 (1 established)
          Peers: 2 (with 2 established connections)
 TrustedSubnets: none

        Service: ipam
         Status: ready
          Range: 10.32.0.0/12
  DefaultSubnet: 10.32.0.0/12

        Service: dns
         Domain: weave.local.
       Upstream: 10.0.2.3
            TTL: 1
        Entries: 0

        Service: proxy
        Address: unix:///var/run/weave/weave.sock

        Service: plugin
     DriverName: weave

vagrant@ubuntuh1:~$

Launched the kubelet specifying the CNI information.

vagrant@ubuntuh1:~$ sudo ./kubelet     --api-servers=172.25.147.100:8080     --allow-privileged=true     --register-node=true     --network-plugin=cni     --network-plugin-dir=/etc/cni/net.d

[rade]

I followed the exact steps mentioned in the docs to start weave.

You are missing a weave expose.

Did you look at our docs for CNI and k8s? If you didn't find them, I'd love to hear suggestions for how to make them more discoverable.

[leodotcloud]

@rade 👍 . Thanks that helped.

I did go through that document. weave expose : I assumed it was something to do with exposing ports on the host, similar to -P/-p flag of docker, so didn't bother(also missed/forgot) to run. Also from watching the videos and reading the other documents this command was not very obvious.

It would be great if it's explained why this command is needed for kubernetes. Also adding this error in the troubleshooting page would be useful for others.

[rade]

The 'expose' is needed to give the weave bridge an IP. Which in turn is required by k8s for obscure reasons that @bboreham or @errordeveloper might remember.

[leodotcloud]

@rade: Is there any specific reason the bridge is not assigned the IP by default, irrespective of whether it's k8s or something else? If this is part of the weave launch, that really simplifies and reduces one additional step during configuration.

[rade]

Is there any specific reason the bridge is not assigned the IP by default, irrespective of whether it's k8s or something else?

Yes. Assigning it an IP requires allocating an IP, which in turn requires consensus between weave peers, which in turn requires a majority to be reachable. So this places a bunch of constraints on how nodes get deployed / started.

[bboreham]

We expose the network so Weave containers have a route out of the Weave network to contact other services, and Kubernetes healthchecks have a route into the Weave network.

@rade is correct that waiting for consensus is sufficiently problematic that I didn't want to do it by default.

Probably we should have another issue to find a better way to allocate a gateway, e.g. using the same address on all nodes. Needs analysis.