search

shettyg / ovn-docker

Provide network virtualization to Docker via OVN.
Apache License 2.0
20 stars 9 forks source link

OVN/OVS driver not recognized by docker #16

Open sandylss opened 7 years ago

sandylss commented 7 years ago

I'm using etcd, docker v17.03.0-ce, openvSwitch v2.5.2, and ubuntu-4.2.0-42-generic.

I followed the instructions on https://github.com/shettyg/ovn-docker/blob/master/docs/docker.rst

I installed and started openvSwitch as per instructions given here http://openvswitch.org/support/dist-docs-2.5/INSTALL.md.html

I have 2 node, one has $CENTRAL_IP and the other has $HOST_IP

I started etcd and typed etcdctl cluster-health to confirm that the cluster is healthy. Then I started docker daemon on host_machine by docker daemon --cluster-store=consul://127.0.0.1:2379 \ --cluster-advertise=$HOST_IP:0 (2379 is the port when I started etcd)

On the central node (with IP $CENTRAL_IP), I executed the following two commands: ovs-appctl -t ovsdb-server ovsdb-server/add-remote ptcp:6640 /usr/local/share/openvswitch/scripts/ovn-ctl start_northd

On the other machine where I use docker, I exectued ovs-vsctl set Open_vSwitch . external_ids:ovn-remote="tcp:$CENTRAL_IP:6640" external_ids:ovn-encap-ip=$LOCAL_IP external_ids:ovn-encap-type="geneve" Fiinally, I started the ovn-controller by executing the command /usr/local/share/openvswitch/scripts/ovn-ctl start_controller (Also, I inserted vport-geneve.ko module from ovs kernel and type lsmod to confirm geneve is supported)

Then I installed Flask module using pip and start the Open vSwitch driver ovn-docker-overlay-driver --detach

So far, everything works fine.

When I type docker network create -d openvswitch --subnet=172.19.1.0/24 ovs-br1 It shows Error response from daemon: legacy plugin: plugin not found

Seems ovn-docker-overlay-driver is not correctly started?

I've working on it for several days and it does depress me...sad...

thank you for your time

shettyg commented 7 years ago

When I type docker network create -d openvswitch --subnet=172.19.1.0/24 ovs-br1 It shows Error response from daemon: legacy plugin: plugin not found

I have tested till Docker 1.13. Based on the error, it looks like something has changed with v17.03.0-ce in terms of place where it looks for the plugin. I will spend some time.

sandylss commented 7 years ago

I have tested till Docker 1.13. Based on the error, it looks like something has changed with v17.03.0-ce in terms of place where it looks for the plugin. I will spend some time.

I've rolled back my docker to v1.12.1 and the bug is still there, so it has nothing to do with Docker version. Since there's no error message when I type ovn-docker-overlay-driver --detach, I guess it just quits halfway. I'm trying to figure it out. Any idea of why? for example, iptables, port, prequistion, whatever...

shettyg commented 7 years ago

I've rolled back my docker to v1.12.1 and the bug is still there, so it has nothing to do with Docker version.

When you run ovn-docker-overlay-driver (without --detach), does it print any errors?

shettyg commented 7 years ago

FYI, I ran the vagrant that comes with this repo. And I did not see any problems. It used docker 17.03.1-ce. You can quickly look at the scripts the vagrant uses to run consul, docker and ovn.

i.e look at https://github.com/shettyg/ovn-docker/blob/master/vagrant_overlay/Vagrantfile to all the scripts that it invokes and then see the individual simple scripts.

You can always come to IRC (freenode). I am there in the #openvswitch channel as "shettyg". You can ping me if you need live debugging.

sandylss commented 7 years ago

i.e look at https://github.com/shettyg/ovn-docker/blob/master/vagrant_overlay/Vagrantfile to all the scripts that it invokes and then see the individual simple scripts.

Thanks for the scripts. I've made a mistake when setting Consul cluster. Now everything works fine ^_^

I notice that after running creating a ovn-docker network and starting a container attached to the network, docker automatically creates docker_gwbridge network. It seems this bridge functions similarly to docker0.

Does it mean that if a container wish to communication with the outside ( e.g other servers in a LAN ), docker_gwbridge is the only gateway? Since we've created a switch when creating ovn-docker network, is it possible to make the switch function as a gateway?

Thank you for your time.

shettyg commented 7 years ago

Does it mean that if a container wish to communication with the outside ( e.g other servers in a LAN ), docker_gwbridge is the only gateway? Since we've created a switch when creating ovn-docker network, is it possible to make the switch function as a gateway?

OVN has multiple options to connect to outside world. If you are running on baremetal, you can connect your switch to vlans via OVN switch. You can also use OVN routers to connect. But the docker plugin does not do any of that. The kubernetes integration uses OVN router to connect to outside world - https://github.com/openvswitch/ovn-kubernetes/