Closed llastra closed 10 years ago
Hi Lucio,
I can help you out with your issue because I was struggling with similar one some time ago and now it works fine.
What do you get from VM's console output when booting this cirros image during network addressing step?
Do you have any warnings/errors in quantum openvswitch plugin, nova-compute or syslog on your compute node?
Also I would like to see the output of:
1)ifconfig
of cirros's VM tap device from compute node
2)ovs-vsctl show
from compute node
3)grep ovs /var/log/syslog
and grep dns /var/log/syslog
from compute node
4)dnsmasq --version
from compute node
Check this post for some useful troubleshooting: http://lorinhochstein.wordpress.com/2013/05/29/the-challenge-of-the-network/
Cheers, Peter
Peter,
thank you for your help. Here is the info:
root@compute:/var/log/nova# dnsmasq --version Dnsmasq version 2.59 Copyright (c) 2000-2011 Simon Kelley Compile time options IPv6 GNU-getopt DBus i18n DHCP TFTP conntrack IDN
This software comes with ABSOLUTELY NO WARRANTY. Dnsmasq is free software, and you are welcome to redistribute it under the terms of the GNU General Public License, version 2 or 3.
root@compute:/var/log/nova# grep dns /var/log/syslog Sep 9 10:13:26 compute kernel: [ 0.686169] Key type dns_resolver registered
root@compute:/var/log/nova# grep ovs /var/log/syslog Sep 9 10:13:27 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --no-wait --timeout=5 -- init -- set Open_vSwitch . db-version=6.4.0 Sep 9 10:13:27 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --no-wait --timeout=5 set Open_vSwitch . ovs-version=1.4.0+build0 "external-ids:system-id=\"a67439a6-9aac-4e9e-ac89-a8e1a049c363\"" "system-type=\"Ubuntu\"" "system-version=\"12.04-precise\"" Sep 9 10:13:27 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 -- --if-exists del-port br-int patch-tun Sep 9 10:13:27 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 -- --if-exists del-br br-tun Sep 9 10:13:27 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 add-br br-tun Sep 9 10:13:28 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 add-port br-int patch-tun Sep 9 10:13:28 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Interface patch-tun type=patch Sep 9 10:13:28 compute ovs-vswitchd: 00081|netdev_vport|ERR|patch-tun: patch type requires valid 'peer' argument Sep 9 10:13:28 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Interface patch-tun options:peer=patch-int Sep 9 10:13:28 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 add-port br-tun patch-int Sep 9 10:13:28 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Interface patch-int type=patch Sep 9 10:13:28 compute ovs-vswitchd: 00115|netdev_vport|ERR|patch-int: patch type requires valid 'peer' argument Sep 9 10:13:28 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Interface patch-int options:peer=patch-tun Sep 9 10:14:12 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qvof74fa3e1-5a -- set Interface qvof74fa3e1-5a external-ids:iface-id=f74fa3e1-5aee-4373-a366-bd4b8f4091e1 external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:aa:06:2e external-ids:vm-uuid=c7cbebf7-6c4d-40de-900e-a7ca1e869e10 Sep 9 10:14:15 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Port qvof74fa3e1-5a tag=1 Sep 9 11:11:57 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl del-port br-int qvof74fa3e1-5a Sep 9 11:11:57 compute ovs-vsctl: 00001|vsctl|ERR|no row "qvof74fa3e1-5a" in table Interface Sep 9 11:13:41 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qvo7b54ec8b-b6 -- set Interface qvo7b54ec8b-b6 external-ids:iface-id=7b54ec8b-b606-466e-ad29-ad083d585ebb external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:c3:b1:9b external-ids:vm-uuid=21d7df8b-799c-4194-ad0a-cc3e61a08097 Sep 9 11:13:45 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Port qvo7b54ec8b-b6 tag=2 Sep 9 12:03:17 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qvo9ee8840e-3e -- set Interface qvo9ee8840e-3e external-ids:iface-id=9ee8840e-3e8f-4295-b959-b819bcd25a53 external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:a2:af:88 external-ids:vm-uuid=0c2c6c4c-f2ab-4e67-8e24-601768292fd5 Sep 9 12:03:20 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Port qvo9ee8840e-3e tag=3 Sep 9 12:04:20 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl del-port br-int qvo9ee8840e-3e Sep 9 12:25:55 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qvoa0786f2d-24 -- set Interface qvoa0786f2d-24 external-ids:iface-id=a0786f2d-24d9-4c91-a5fe-936b22b2585f external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:6b:21:33 external-ids:vm-uuid=c4bfddfb-0a3a-424a-a590-d8a7669d0983 Sep 9 12:25:59 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Port qvoa0786f2d-24 tag=4 Sep 9 13:38:59 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl del-port br-int qvoa0786f2d-24 Sep 9 13:39:00 compute ovs-vsctl: 00001|vsctl|ERR|no row "qvoa0786f2d-24" in table Interface Sep 9 13:39:21 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qvoc48af542-cb -- set Interface qvoc48af542-cb external-ids:iface-id=c48af542-cbba-4aaf-bd48-384a5fc53234 external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:ce:eb:6d external-ids:vm-uuid=4f93ca28-f455-4333-addf-f7071169cd8f Sep 9 13:39:25 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Port qvoc48af542-cb tag=5 Sep 9 13:54:47 compute ovs-vsctl: 00001|stream_unix|ERR|/tmp/stream-unix.30536.0: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 9 13:54:47 compute ovs-vsctl: 00002|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied) Sep 9 13:54:48 compute ovs-vsctl: 00003|stream_unix|ERR|/tmp/stream-unix.30536.1: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 9 13:54:48 compute ovs-vsctl: 00004|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied) Sep 9 13:54:50 compute ovs-vsctl: 00005|stream_unix|ERR|/tmp/stream-unix.30536.2: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 9 13:54:50 compute ovs-vsctl: 00006|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied) Sep 9 13:54:54 compute ovs-vsctl: 00007|stream_unix|ERR|/tmp/stream-unix.30536.3: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 9 13:54:54 compute ovs-vsctl: 00008|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied) root@compute:/var/log/nova# clear root@compute:/var/log/nova# grep ovs /var/log/syslog Sep 9 10:13:27 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --no-wait --timeout=5 -- init -- set Open_vSwitch . db-version=6.4.0 Sep 9 10:13:27 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --no-wait --timeout=5 set Open_vSwitch . ovs-version=1.4.0+build0 "external-ids:system-id=\"a67439a6-9aac-4e9e-ac89-a8e1a049c363\"" "system-type=\"Ubuntu\"" "system-version=\"12.04-precise\"" Sep 9 10:13:27 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 -- --if-exists del-port br-int patch-tun Sep 9 10:13:27 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 -- --if-exists del-br br-tun Sep 9 10:13:27 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 add-br br-tun Sep 9 10:13:28 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 add-port br-int patch-tun Sep 9 10:13:28 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Interface patch-tun type=patch Sep 9 10:13:28 compute ovs-vswitchd: 00081|netdev_vport|ERR|patch-tun: patch type requires valid 'peer' argument Sep 9 10:13:28 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Interface patch-tun options:peer=patch-int Sep 9 10:13:28 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 add-port br-tun patch-int Sep 9 10:13:28 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Interface patch-int type=patch Sep 9 10:13:28 compute ovs-vswitchd: 00115|netdev_vport|ERR|patch-int: patch type requires valid 'peer' argument Sep 9 10:13:28 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Interface patch-int options:peer=patch-tun Sep 9 10:14:12 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qvof74fa3e1-5a -- set Interface qvof74fa3e1-5a external-ids:iface-id=f74fa3e1-5aee-4373-a366-bd4b8f4091e1 external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:aa:06:2e external-ids:vm-uuid=c7cbebf7-6c4d-40de-900e-a7ca1e869e10 Sep 9 10:14:15 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Port qvof74fa3e1-5a tag=1 Sep 9 11:11:57 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl del-port br-int qvof74fa3e1-5a Sep 9 11:11:57 compute ovs-vsctl: 00001|vsctl|ERR|no row "qvof74fa3e1-5a" in table Interface Sep 9 11:13:41 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qvo7b54ec8b-b6 -- set Interface qvo7b54ec8b-b6 external-ids:iface-id=7b54ec8b-b606-466e-ad29-ad083d585ebb external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:c3:b1:9b external-ids:vm-uuid=21d7df8b-799c-4194-ad0a-cc3e61a08097 Sep 9 11:13:45 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Port qvo7b54ec8b-b6 tag=2 Sep 9 12:03:17 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qvo9ee8840e-3e -- set Interface qvo9ee8840e-3e external-ids:iface-id=9ee8840e-3e8f-4295-b959-b819bcd25a53 external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:a2:af:88 external-ids:vm-uuid=0c2c6c4c-f2ab-4e67-8e24-601768292fd5 Sep 9 12:03:20 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Port qvo9ee8840e-3e tag=3 Sep 9 12:04:20 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl del-port br-int qvo9ee8840e-3e Sep 9 12:25:55 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qvoa0786f2d-24 -- set Interface qvoa0786f2d-24 external-ids:iface-id=a0786f2d-24d9-4c91-a5fe-936b22b2585f external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:6b:21:33 external-ids:vm-uuid=c4bfddfb-0a3a-424a-a590-d8a7669d0983 Sep 9 12:25:59 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Port qvoa0786f2d-24 tag=4 Sep 9 13:38:59 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl del-port br-int qvoa0786f2d-24 Sep 9 13:39:00 compute ovs-vsctl: 00001|vsctl|ERR|no row "qvoa0786f2d-24" in table Interface Sep 9 13:39:21 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qvoc48af542-cb -- set Interface qvoc48af542-cb external-ids:iface-id=c48af542-cbba-4aaf-bd48-384a5fc53234 external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:ce:eb:6d external-ids:vm-uuid=4f93ca28-f455-4333-addf-f7071169cd8f Sep 9 13:39:25 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=2 set Port qvoc48af542-cb tag=5 Sep 9 13:54:47 compute ovs-vsctl: 00001|stream_unix|ERR|/tmp/stream-unix.30536.0: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 9 13:54:47 compute ovs-vsctl: 00002|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied) Sep 9 13:54:48 compute ovs-vsctl: 00003|stream_unix|ERR|/tmp/stream-unix.30536.1: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 9 13:54:48 compute ovs-vsctl: 00004|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied) Sep 9 13:54:50 compute ovs-vsctl: 00005|stream_unix|ERR|/tmp/stream-unix.30536.2: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 9 13:54:50 compute ovs-vsctl: 00006|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied) Sep 9 13:54:54 compute ovs-vsctl: 00007|stream_unix|ERR|/tmp/stream-unix.30536.3: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 9 13:54:54 compute ovs-vsctl: 00008|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied)
latechco@compute:/var/log/nova$ ifconfig eth0 Link encap:Ethernet HWaddr 00:22:64:c2:50:04 inet addr:10.10.10.53 Bcast:10.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::222:64ff:fec2:5004/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2871 errors:0 dropped:0 overruns:0 frame:0 TX packets:3459 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:577957 (577.9 KB) TX bytes:664614 (664.6 KB)
eth1 Link encap:Ethernet HWaddr 00:22:64:c2:50:02 inet addr:10.20.20.53 Bcast:10.20.20.255 Mask:255.255.255.0 inet6 addr: fe80::222:64ff:fec2:5002/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:5 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:418 (418.0 B)
lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:1068 errors:0 dropped:0 overruns:0 frame:0 TX packets:1068 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:82742 (82.7 KB) TX bytes:82742 (82.7 KB)
qbr7b54ec8b-b6 Link encap:Ethernet HWaddr e6:05:8a:28:47:97 inet6 addr: fe80::343d:97ff:fe38:2dc7/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:14 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1616 (1.6 KB) TX bytes:468 (468.0 B)
qbrc48af542-cb Link encap:Ethernet HWaddr 92:5c:94:10:b6:8d inet6 addr: fe80::cc7f:b0ff:fe4a:4af4/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:14 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1616 (1.6 KB) TX bytes:468 (468.0 B)
qvb7b54ec8b-b6 Link encap:Ethernet HWaddr e6:05:8a:28:47:97 inet6 addr: fe80::e405:8aff:fe28:4797/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:6 errors:0 dropped:0 overruns:0 frame:0 TX packets:19 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:468 (468.0 B) TX bytes:2202 (2.2 KB)
qvbc48af542-cb Link encap:Ethernet HWaddr 92:5c:94:10:b6:8d inet6 addr: fe80::905c:94ff:fe10:b68d/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:6 errors:0 dropped:0 overruns:0 frame:0 TX packets:20 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:468 (468.0 B) TX bytes:2280 (2.2 KB)
qvo7b54ec8b-b6 Link encap:Ethernet HWaddr 8e:fa:03:12:dc:7e inet6 addr: fe80::8cfa:3ff:fe12:dc7e/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:19 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2202 (2.2 KB) TX bytes:468 (468.0 B)
qvoc48af542-cb Link encap:Ethernet HWaddr 7e:e5:11:0a:f0:6a inet6 addr: fe80::7ce5:11ff:fe0a:f06a/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:20 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2280 (2.2 KB) TX bytes:468 (468.0 B)
tap7b54ec8b-b6 Link encap:Ethernet HWaddr fe:16:3e:c3:b1:9b inet6 addr: fe80::fc16:3eff:fec3:b19b/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:9 errors:0 dropped:0 overruns:0 frame:0 TX packets:11 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:1434 (1.4 KB) TX bytes:838 (838.0 B)
tapc48af542-cb Link encap:Ethernet HWaddr fe:16:3e:ce:eb:6d inet6 addr: fe80::fc16:3eff:fece:eb6d/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:9 errors:0 dropped:0 overruns:0 frame:0 TX packets:11 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:1434 (1.4 KB) TX bytes:838 (838.0 B)
root@compute:/var/log/quantum# tail -n 200 openvswitch-agent.log File "/usr/lib/python2.7/contextlib.py", line 24, in exit self.gen.next() File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 552, in iter self._iterator.next() File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 648, in iterconsume yield self.ensure(_error_callback, _consume) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 566, in ensure error_callback(e) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 629, in _error_callback raise rpc_common.Timeout() Timeout: Timeout while waiting on RPC response. 2013-09-09 12:00:29 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 56.732728 sec 2013-09-09 12:00:44 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 11.036933 sec 2013-09-09 12:00:52 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 4.284903 sec 2013-09-09 12:05:32 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 52.23289 sec 2013-09-09 12:06:33 ERROR [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Failed reporting state! Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py", line 202, in _report_state self.agent_state) File "/usr/lib/python2.7/dist-packages/quantum/agent/rpc.py", line 66, in report_state topic=self.topic) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/proxy.py", line 80, in call return rpc.call(context, self._get_topic(topic), msg, timeout) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/init.py", line 140, in call return _get_impl().call(CONF, context, topic, msg, timeout) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 798, in call rpc_amqp.get_connection_pool(conf, Connection)) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 613, in call rv = list(rv) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 555, in iter self.done() File "/usr/lib/python2.7/contextlib.py", line 24, in exit self.gen.next() File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 552, in iter self._iterator.next() File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 648, in iterconsume yield self.ensure(_error_callback, _consume) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 566, in ensure error_callback(e) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 629, in _error_callback raise rpc_common.Timeout() Timeout: Timeout while waiting on RPC response. 2013-09-09 12:06:33 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 56.736009 sec 2013-09-09 12:07:34 ERROR [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Failed reporting state! Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py", line 202, in _report_state self.agent_state) File "/usr/lib/python2.7/dist-packages/quantum/agent/rpc.py", line 66, in report_state topic=self.topic) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/proxy.py", line 80, in call return rpc.call(context, self._get_topic(topic), msg, timeout) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/init.py", line 140, in call return _get_impl().call(CONF, context, topic, msg, timeout) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 798, in call rpc_amqp.get_connection_pool(conf, Connection)) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 613, in call rv = list(rv) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 555, in iter self.done() File "/usr/lib/python2.7/contextlib.py", line 24, in exit self.gen.next() File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 552, in iter self._iterator.next() File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 648, in iterconsume yield self.ensure(_error_callback, _consume) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 566, in ensure error_callback(e) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 629, in _error_callback raise rpc_common.Timeout() Timeout: Timeout while waiting on RPC response. 2013-09-09 12:07:34 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 56.73587 sec 2013-09-09 12:07:54 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 16.543438 sec 2013-09-09 12:08:55 ERROR [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Failed reporting state! Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py", line 202, in _report_state self.agent_state) File "/usr/lib/python2.7/dist-packages/quantum/agent/rpc.py", line 66, in report_state topic=self.topic) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/proxy.py", line 80, in call return rpc.call(context, self._get_topic(topic), msg, timeout) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/init.py", line 140, in call return _get_impl().call(CONF, context, topic, msg, timeout) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 798, in call rpc_amqp.get_connection_pool(conf, Connection)) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 613, in call rv = list(rv) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 555, in iter self.done() File "/usr/lib/python2.7/contextlib.py", line 24, in exit self.gen.next() File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 552, in iter self._iterator.next() File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 648, in iterconsume yield self.ensure(_error_callback, _consume) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 566, in ensure error_callback(e) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 629, in _error_callback raise rpc_common.Timeout() Timeout: Timeout while waiting on RPC response. 2013-09-09 12:08:55 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 56.628019 sec 2013-09-09 12:09:56 ERROR [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Failed reporting state! Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py", line 202, in _report_state self.agent_state) File "/usr/lib/python2.7/dist-packages/quantum/agent/rpc.py", line 66, in report_state topic=self.topic) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/proxy.py", line 80, in call return rpc.call(context, self._get_topic(topic), msg, timeout) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/init.py", line 140, in call return _get_impl().call(CONF, context, topic, msg, timeout) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 798, in call rpc_amqp.get_connection_pool(conf, Connection)) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 613, in call rv = list(rv) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 555, in iter self.done() File "/usr/lib/python2.7/contextlib.py", line 24, in exit self.gen.next() File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 552, in iter self._iterator.next() File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 648, in iterconsume yield self.ensure(_error_callback, _consume) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 566, in ensure error_callback(e) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 629, in _error_callback raise rpc_common.Timeout() Timeout: Timeout while waiting on RPC response. 2013-09-09 12:09:56 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 56.7556 sec 2013-09-09 12:10:57 ERROR [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Failed reporting state! Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py", line 202, in _report_state self.agent_state) File "/usr/lib/python2.7/dist-packages/quantum/agent/rpc.py", line 66, in report_state topic=self.topic) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/proxy.py", line 80, in call return rpc.call(context, self._get_topic(topic), msg, timeout) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/init.py", line 140, in call return _get_impl().call(CONF, context, topic, msg, timeout) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 798, in call rpc_amqp.get_connection_pool(conf, Connection)) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 613, in call rv = list(rv) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 555, in iter self.done() File "/usr/lib/python2.7/contextlib.py", line 24, in exit self.gen.next() File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 552, in iter self._iterator.next() File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 648, in iterconsume yield self.ensure(_error_callback, _consume) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 566, in ensure error_callback(e) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 629, in _error_callback raise rpc_common.Timeout() Timeout: Timeout while waiting on RPC response. 2013-09-09 12:10:57 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 56.735974 sec 2013-09-09 12:11:57 ERROR [quantum.plugins.openvswitch.agent.ovs_quantum_agent] Failed reporting state! Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py", line 202, in _report_state self.agent_state) File "/usr/lib/python2.7/dist-packages/quantum/agent/rpc.py", line 66, in report_state topic=self.topic) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/proxy.py", line 80, in call return rpc.call(context, self._get_topic(topic), msg, timeout) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/init.py", line 140, in call return _get_impl().call(CONF, context, topic, msg, timeout) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 798, in call rpc_amqp.get_connection_pool(conf, Connection)) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 613, in call rv = list(rv) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 555, in iter self.done() File "/usr/lib/python2.7/contextlib.py", line 24, in exit self.gen.next() File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/amqp.py", line 552, in iter self._iterator.next() File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 648, in iterconsume yield self.ensure(_error_callback, _consume) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 566, in ensure error_callback(e) File "/usr/lib/python2.7/dist-packages/quantum/openstack/common/rpc/impl_kombu.py", line 629, in _error_callback raise rpc_common.Timeout() Timeout: Timeout while waiting on RPC response. 2013-09-09 12:11:57 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 56.739925 sec 2013-09-09 12:12:21 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 19.87295 sec 2013-09-09 12:13:17 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 51.634017 sec 2013-09-09 12:13:26 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 4.899338 sec 2013-09-09 12:13:41 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 10.896439 sec 2013-09-09 12:14:30 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 45.248542 sec 2013-09-09 12:15:06 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 32.54658 sec 2013-09-09 12:15:31 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 20.489451 sec 2013-09-09 12:16:03 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 28.135182 sec 2013-09-09 12:16:30 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 22.784208 sec 2013-09-09 12:34:50 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 0.552853 sec 2013-09-09 13:39:00 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'get', 'Interface', 'qvoa0786f2d-24', 'external_ids']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'qvoa0786f2d-24', 'external_ids'] Exit code: 1 Stdout: '' Stderr: 'ovs-vsctl: no row "qvoa0786f2d-24" in table Interface\n' 2013-09-09 13:47:03 WARNING [quantum.openstack.common.loopingcall] task run outlasted interval by 0.20803 sec
root@compute:/var/log/nova# ovs-vsctl show 7cc6e610-ffa1-4a6a-81eb-462108284f41 Bridge br-int Port "qvoab4f161a-34" tag: 1 Interface "qvoab4f161a-34" Port "qvoc48af542-cb" tag: 5 Interface "qvoc48af542-cb" Port "qvo5d723748-97" tag: 6 Interface "qvo5d723748-97" Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port br-int Interface br-int type: internal Port "qvo7b54ec8b-b6" tag: 2 Interface "qvo7b54ec8b-b6" Bridge br-tun Port br-tun Interface br-tun type: internal Port patch-int Interface patch-int type: patch options: {peer=patch-tun} ovs_version: "1.4.0+build0"
Sep 9 13:47:08 compute ntpd[23917]: Listening on routing socket on fd #31 for interface updates Sep 9 13:47:15 compute ntpd[23917]: ntpd exiting on signal 15 Sep 9 13:47:16 compute kernel: [12835.000614] bnx2 0000:05:00.0: eth1: NIC Copper Link is Up, 1000 Mbps full duplex Sep 9 13:47:16 compute kernel: [12835.000619] , receive & transmit flow control ON Sep 9 13:47:16 compute kernel: [12835.001657] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready Sep 9 13:47:16 compute kernel: [12835.429828] bnx2 0000:05:00.0: eth1: NIC Copper Link is Down Sep 9 13:47:19 compute kernel: [12838.187469] bnx2 0000:05:00.0: eth1: NIC Copper Link is Up, 1000 Mbps full duplex Sep 9 13:47:19 compute kernel: [12838.187474] , receive & transmit flow control ON Sep 9 13:47:23 compute ntpdate[24021]: step time server 10.10.10.51 offset 0.000011 sec Sep 9 13:47:23 compute ntpd[24159]: ntpd 4.2.6p3@1.2290-o Tue Jun 5 20:12:08 UTC 2012 (1) Sep 9 13:47:23 compute ntpd[24160]: proto: precision = 0.375 usec Sep 9 13:47:23 compute ntpd[24160]: ntp_io: estimated max descriptors: 1024, initial socket boundary: 16 Sep 9 13:47:23 compute ntpd[24160]: Listen and drop on 0 v4wildcard 0.0.0.0 UDP 123 Sep 9 13:47:23 compute ntpd[24160]: Listen and drop on 1 v6wildcard :: UDP 123 Sep 9 13:47:23 compute ntpd[24160]: Listen normally on 2 lo 127.0.0.1 UDP 123 Sep 9 13:47:23 compute ntpd[24160]: Listen normally on 3 eth0 10.10.10.53 UDP 123 Sep 9 13:47:23 compute ntpd[24160]: Listen normally on 4 eth1 10.20.20.53 UDP 123 Sep 9 13:47:23 compute ntpd[24160]: Listen normally on 5 tapc48af542-cb fe80::fc16:3eff:fece:eb6d UDP 123 Sep 9 13:47:23 compute ntpd[24160]: Listen normally on 6 qvo7b54ec8b-b6 fe80::8cfa:3ff:fe12:dc7e UDP 123 Sep 9 13:47:23 compute ntpd[24160]: Listen normally on 7 qvbc48af542-cb fe80::905c:94ff:fe10:b68d UDP 123 Sep 9 13:47:23 compute ntpd[24160]: Listen normally on 8 eth1 fe80::222:64ff:fec2:5002 UDP 123 Sep 9 13:47:23 compute ntpd[24160]: Listen normally on 9 qbrc48af542-cb fe80::cc7f:b0ff:fe4a:4af4 UDP 123 Sep 9 13:47:23 compute ntpd[24160]: Listen normally on 10 qvoc48af542-cb fe80::7ce5:11ff:fe0a:f06a UDP 123 Sep 9 13:47:23 compute ntpd[24160]: Listen normally on 11 qvb7b54ec8b-b6 fe80::e405:8aff:fe28:4797 UDP 123 Sep 9 13:47:23 compute ntpd[24160]: Listen normally on 12 qbr7b54ec8b-b6 fe80::343d:97ff:fe38:2dc7 UDP 123 Sep 9 13:47:23 compute ntpd[24160]: Listen normally on 13 eth0 fe80::222:64ff:fec2:5004 UDP 123 Sep 9 13:47:23 compute ntpd[24160]: Listen normally on 14 tap7b54ec8b-b6 fe80::fc16:3eff:fec3:b19b UDP 123 Sep 9 13:47:23 compute ntpd[24160]: Listen normally on 15 lo ::1 UDP 123 Sep 9 13:47:23 compute ntpd[24160]: peers refreshed Sep 9 13:47:23 compute ntpd[24160]: Listening on routing socket on fd #32 for interface updates Sep 9 13:54:47 compute ovs-vsctl: 00001|stream_unix|ERR|/tmp/stream-unix.30536.0: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 9 13:54:47 compute ovs-vsctl: 00002|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied) Sep 9 13:54:48 compute ovs-vsctl: 00003|stream_unix|ERR|/tmp/stream-unix.30536.1: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 9 13:54:48 compute ovs-vsctl: 00004|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied) Sep 9 13:54:50 compute ovs-vsctl: 00005|stream_unix|ERR|/tmp/stream-unix.30536.2: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 9 13:54:50 compute ovs-vsctl: 00006|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied) Sep 9 13:54:54 compute ovs-vsctl: 00007|stream_unix|ERR|/tmp/stream-unix.30536.3: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 9 13:54:54 compute ovs-vsctl: 00008|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied)
Peter,
this is the relevant network part of Starting network... udhcpc (v1.18.5) started Sending discover... Sending discover... Sending discover... No lease, failing WARN: /etc/rc3.d/S40network failed cloudsetup: checking http://169.254.169.254/20090404/metadata/instanceid wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 1/30: up 9.97. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 2/30: up 10.98. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 3/30: up 11.99. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 4/30: up 13.00. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 5/30: up 14.00. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 6/30: up 15.01. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 7/30: up 16.02. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 8/30: up 17.03. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 9/30: up 18.03. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 10/30: up 19.04. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 11/30: up 20.05. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 12/30: up 21.05. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 13/30: up 22.06. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 14/30: up 23.07. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 15/30: up 24.08. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 16/30: up 25.08. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 17/30: up 26.09. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 18/30: up 27.10. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 19/30: up 28.11. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 20/30: up 29.12. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 21/30: up 30.12. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 22/30: up 31.13. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 23/30: up 32.14. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 24/30: up 33.15. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 25/30: up 34.15. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 26/30: up 35.16. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 27/30: up 36.17. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 28/30: up 37.18. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 29/30: up 38.18. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 30/30: up 39.19. request failed cloudsetup: after 30 fails, debugging cloudsetup: running debug (30 tries reached) ############ debug start ##############
/etc/rc3.d/S45cloudsetup: line 66: /etc/rc.d/init.d/sshd: not found route: fscanf
eth0 Link encap:Ethernet HWaddr FA:16:3E:16:A2:AD inet6 addr: fe80::f816:3eff:fe16:a2ad/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:11 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:838 (838.0 B) TX bytes:1364 (1.3 KiB)
lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface route: fscanf
cat: can't open '/etc/resolv.conf': No such file or directory
/etc/rc3.d/S45cloudsetup: line 66: can't open /etc/resolv.conf: no such file
Linux cirros 3.0.012virtual #20Ubuntu SMP Fri Oct 7 18:19:02 UTC 2011 x86_64 GNU/Linux
Module Size Used by Not tainted vfat 17585 0 fat 61475 1 vfat isofs 40253 0 ip_tables 27473 0 x_tables 29846 1 ip_tables pcnet32 42078 0 8139cp 27412 0 ne2k_pci 13691 0 8390 18856 1 ne2k_pci e1000 108573 0 acpiphp 24080 0
[ 0.851490] acpiphp: Slot [30] registered [ 0.851507] acpiphp: Slot [31] registered [ 0.859827] e1000: Intel(R) PRO/1000 Network Driver version 7.3.21k8NAPI [ 0.859830] e1000: Copyright (c) 19992006 Intel Corporation. [ 0.864701] ne2kpci.c:v1.03 9/22/2003 D. Becker/P. Gortmaker [ 0.868683] 8139cp: 8139cp: 10/100 PCI Ethernet driver v1.3 (Mar 22, 2004) [ 0.873020] pcnet32: pcnet32.c:v1.35 21.Apr.2008 tsbogend@alpha.franken.de [ 0.878876] ip_tables: (C) 20002006 Netfilter Core Team [ 1.256075] Refined TSC clocksource calibration: 3000.076 MHz. [ 11.136115] eth0: no IPv6 routers present
Sep 9 13:53:04 cirros kern.info kernel: [ 0.748870] EXT3fs: barriers not enabled Sep 9 13:53:04 cirros kern.info kernel: [ 0.749036] kjournald starting. Commit interval 5 seconds Sep 9 13:53:04 cirros kern.info kernel: [ 0.749051] EXT3fs (vda1): mounted filesystem with ordered data mode Sep 9 13:53:04 cirros kern.warn kernel: [ 0.777911] EXT3fs (vda1): warning: checktime reached, running e2fsck is recommended Sep 9 13:53:04 cirros kern.info kernel: [ 0.783487] EXT3fs (vda1): using internal journal Sep 9 13:53:04 cirros kern.info kernel: [ 0.788073] usb 11: new full speed USB device number 2 using uhci_hcd Sep 9 13:53:04 cirros kern.info kernel: [ 0.850699] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 Sep 9 13:53:04 cirros kern.info kernel: [ 0.850895] acpiphp: Slot [1] registered Sep 9 13:53:04 cirros kern.info kernel: [ 0.850922] acpiphp: Slot [2] registered Sep 9 13:53:04 cirros kern.info kernel: [ 0.850939] acpiphp: Slot [3] registered Sep 9 13:53:04 cirros kern.info kernel: [ 0.850955] acpiphp: Slot [4] registered Sep 9 13:53:04 cirros kern.info kernel: [ 0.850972] acpiphp: Slot [5] registered Sep 9 13:53:04 cirros kern.info kernel: [ 0.850990] acpiphp: Slot [6] registered Sep 9 13:53:04 cirros kern.info kernel: [ 0.851019] acpiphp: Slot [7] registered Sep 9 13:53:04 cirros kern.info kernel: [ 0.851050] acpiphp: Slot [8] registered Sep 9 13:53:04 cirros kern.info kernel: [ 0.851067] acpiphp: Slot [9] registered Sep 9 13:53:04 cirros kern.info kernel: [ 0.851098] acpiphp: Slot [10] registered Sep 9 13:53:04 cirros kern.info kernel: [ 0.859827] e1000: Intel(R) PRO/1000 Network Driver version 7.3.21k8NAPI Sep 9 13:53:04 cirros kern.info kernel: [ 0.859830] e1000: Copyright (c) 19992006 Intel Corporation. Sep 9 13:53:04 cirros kern.info kernel: [ 0.864701] ne2kpci.c:v1.03 9/22/2003 D. Becker/P. Gortmaker Sep 9 13:53:04 cirros kern.info kernel: [ 0.868683] 8139cp: 8139cp: 10/100 PCI Ethernet driver v1.3 (Mar 22, 2004) Sep 9 13:53:04 cirros kern.info kernel: [ 0.873020] pcnet32: pcnet32.c:v1.35 21.Apr.2008 tsbogend@alpha.franken.de Sep 9 13:53:04 cirros kern.info kernel: [ 0.878876] ip_tables: (C) 20002006 Netfilter Core Team Sep 9 13:53:05 cirros kern.info kernel: [ 1.256075] Refined TSC clocksource calibration: 3000.076 MHz. Sep 9 13:53:15 cirros kern.debug kernel: [ 11.136115] eth0: no IPv6 routers present ############ debug end ############## cloudsetup: failed to read iid from metadata. tried 30 WARN: /etc/rc3.d/S45cloudsetup failed Starting dropbear sshd: generating rsa key... generating dsa key... OK ===== cloudfinal: system completely up in 40.42 seconds ==== wget: can't connect to remote host (169.254.169.254): Network is unreachable wget: can't connect to remote host (169.254.169.254): Network is unreachable wget: can't connect to remote host (169.254.169.254): Network is unreachable instanceid: publicipv4: localipv4 : wget: can't connect to remote host (169.254.169.254): Network is unreachable clouduserdata: failed to read instance id WARN: /etc/rc3.d/S99clouduserdata failed
/ / __ / \/ / / / / // // _// // /\ \ _///// /_/ _// http://launchpad.net/cirros Cirros boot:
Hope you can write me back soon!
Hello Lucio, I've noticed that your openvswitch is 1.4.0 - are you sure you have downloaded it from grizzly's source list?
root@compute:~# apt-cache madison openvswitch-switch openvswitch-switch | 1.4.2+git20120612-9.1~deb7u1 | http://archive.gplhost.com/debian/ grizzly-backports/main amd64 Packages openvswitch-switch | 1.4.0-1ubuntu1.5 | http://pl.archive.ubuntu.com/ubuntu/ precise-updates/universe amd64 Packages openvswitch-switch | 1.4.0-1ubuntu1 | http://pl.archive.ubuntu.com/ubuntu/ precise/universe amd64 Packages
root@compute:~# apt-cache policy openvswitch-switch openvswitch-switch: Installed: 1.4.2+git20120612-9.1~deb7u1 Candidate: 1.4.2+git20120612-9.1~deb7u1 Version table: *** 1.4.2+git20120612-9.1~deb7u1 0 500 http://archive.gplhost.com/debian/ grizzly-backports/main amd64 Packages
root@compute:~# cat /etc/apt/sources.list.d/grizzly.list deb http://archive.gplhost.com/debian grizzly main deb http://archive.gplhost.com/debian grizzly-backports main
root@compute:~# uname -r 3.2.0-23-generic
Regarding these permissions errors - make sure you are using openvswitch from grizzly's source, in my case it's 1.4.2 for 3.2.0-23-generic kernel, reboot all nodes and try to boot this cirros image again. I want to see if these permission denied errors will go away - if not try to grant quantum user full access:
`root@compute:~# cat /etc/sudoers.d/quantum_sudoers Defaults:quantum !requiretty
quantum ALL=NOPASSWD: ALL root@compute:~# service quantum-plugin-openvswitch-agent restart`
Below you can see that it is working:
Starting network... udhcpc (v1.20.1) started Sending discover... Sending select for 192.168.0.2... Lease of 192.168.0.2 obtained, lease time 120 deleting routers route: SIOCDELRT: No such process adding dns 8.8.8.8 cirrosds 'net' up at 2816.25 checking http://169.254.169.254/20090404/instanceid successful after 1/20 tries: up 2816.81. iid=i00000004 found datasource (ec2, net) Starting dropbear sshd: generating rsa key... generating dsa key... OK /run/cirros/datasource/data/userdata was not '#!' or executable === network info === ifinfo: lo,up,127.0.0.1,8,::1 ifinfo: eth0,up,192.168.0.2,24,fe80::f816:3eff:fe45:507a iproute:default via 192.168.0.1 dev eth0 iproute:192.168.0.0/24 dev eth0 src 192.168.0.2 === datasource: ec2 net === instanceid: i00000004 name: N/A availabilityzone: nova localhostname: cirros.novalocal launchindex: 0 === cirros: current=0.3.1 uptime=2854.69 ===
login as: cirros cirros@server's password: $ ifconfig eth0 eth0 Link encap:Ethernet HWaddr FA:16:3E:45:50:7A inet addr:192.168.0.2 Bcast:192.168.0.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe45:507a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:416 errors:0 dropped:0 overruns:0 frame:0 TX packets:437 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:48395 (47.2 KiB) TX bytes:57896 (56.5 KiB)
root@compute:~# ping 192.168.0.2 -I qbr42dd6bd6-d9 PING 192.168.0.2 (192.168.0.2) from 10.28.8.93 qbr42dd6bd6-d9: 56(84) bytes of data. 64 bytes from 192.168.0.2: icmp_req=1 ttl=63 time=44.8 ms 64 bytes from 192.168.0.2: icmp_req=2 ttl=63 time=1.36 ms 64 bytes from 192.168.0.2: icmp_req=3 ttl=63 time=1.36 ms ^C --- 192.168.0.2 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 1.363/15.865/44.869/20.508 ms
root@compute:~# arp -an|grep fa:16:3e:45:50:7a ? (192.168.0.2) at fa:16:3e:45:50:7a [ether] on qbr42dd6bd6-d9
Let me know how it is going
Cheers, Peter
Hello Peter,
I updated the repos and reinstalled "openvswitch-switch openvswitch-datapath-dkms"
root@compute:/home/latechco# apt-cache madison openvswitch-switch openvswitch-switch | 1.4.2+git20120612-9.1~deb7u1 | http://archive.gplhost.com/debian/ grizzly-backports/main amd64 Packages openvswitch-switch | 1.4.0-1ubuntu1.5 | http://us.archive.ubuntu.com/ubuntu/ precise-updates/universe amd64 Packages openvswitch-switch | 1.4.0-1ubuntu1 | http://us.archive.ubuntu.com/ubuntu/ precise/universe amd64 Packages openvswitch | 1.4.0-1ubuntu1 | http://us.archive.ubuntu.com/ubuntu/ precise/universe Sources openvswitch | 1.4.0-1ubuntu1.5 | http://us.archive.ubuntu.com/ubuntu/ precise-updates/universe Sources
I also changed the entry in /etc/sudoers.d/quantum_sudoers:
root@compute:/home/latechco# cat /etc/sudoers.d/quantum_sudoers Defaults:quantum !requiretty
quantum ALL=(ALL) ALL
latechco@compute:/var/log/quantum$ grep dns /var/log/syslog Sep 10 09:51:39 compute kernel: [ 0.662068] Key type dns_resolver registered Sep 10 09:59:32 compute kernel: [ 0.707909] Key type dns_resolver registered Sep 10 10:48:33 compute kernel: [ 0.750071] Key type dns_resolver registered Sep 10 10:55:29 compute kernel: [ 0.686047] Key type dns_resolver registered
root@compute:/home/latechco# grep ovs /var/log/syslog Sep 10 09:51:40 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --no-wait --timeout=5 -- init -- set Open_vSwitch . db-version=6.4.0 Sep 10 09:51:40 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --no-wait --timeout=5 set Open_vSwitch . ovs-version=1.4.0+build0 "external-ids:system-id=\"a67439a6-9aac-4e9e-ac89-a8e1a049c363\"" "system-type=\"Ubuntu\"" "system-version=\"12.04-precise\"" Sep 10 09:59:33 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --no-wait --timeout=5 -- init -- set Open_vSwitch . db-version=6.4.0 Sep 10 09:59:33 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --no-wait --timeout=5 set Open_vSwitch . ovs-version=1.4.0+build0 "external-ids:system-id=\"a67439a6-9aac-4e9e-ac89-a8e1a049c363\"" "system-type=\"Ubuntu\"" "system-version=\"12.04-precise\"" Sep 10 10:48:34 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --no-wait --timeout=5 -- init -- set Open_vSwitch . db-version=6.4.0 Sep 10 10:48:34 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --no-wait --timeout=5 set Open_vSwitch . ovs-version=1.4.0+build0 "external-ids:system-id=\"a67439a6-9aac-4e9e-ac89-a8e1a049c363\"" "system-type=\"Ubuntu\"" "system-version=\"12.04-precise\"" Sep 10 10:48:37 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qvoc48af542-cb -- set Interface qvoc48af542-cb external-ids:iface-id=c48af542-cbba-4aaf-bd48-384a5fc53234 external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:ce:eb:6d external-ids:vm-uuid=4f93ca28-f455-4333-addf-f7071169cd8f Sep 10 10:55:30 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --no-wait --timeout=5 -- init -- set Open_vSwitch . db-version=6.4.0 Sep 10 10:55:30 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --no-wait --timeout=5 set Open_vSwitch . ovs-version=1.4.0+build0 "external-ids:system-id=\"a67439a6-9aac-4e9e-ac89-a8e1a049c363\"" "system-type=\"Ubuntu\"" "system-version=\"12.04-precise\"" Sep 10 10:55:33 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qvoc48af542-cb -- set Interface qvoc48af542-cb external-ids:iface-id=c48af542-cbba-4aaf-bd48-384a5fc53234 external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:ce:eb:6d external-ids:vm-uuid=4f93ca28-f455-4333-addf-f7071169cd8f Sep 10 10:56:18 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qvo866a0175-98 -- set Interface qvo866a0175-98 external-ids:iface-id=866a0175-9814-4ba6-a27a-ea96bda4e499 external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:ad:89:d8 external-ids:vm-uuid=7f43d426-1c3b-4b2c-b02d-9cabdc7b3a0d Sep 10 11:07:55 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --no-wait --timeout=5 -- init -- set Open_vSwitch . db-version=6.4.0 Sep 10 11:07:55 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --no-wait --timeout=5 set Open_vSwitch . ovs-version=1.4.0+build0 "external-ids:system-id=\"a67439a6-9aac-4e9e-ac89-a8e1a049c363\"" "system-type=\"Ubuntu\"" "system-version=\"12.04-precise\"" Sep 10 11:13:10 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --no-wait --timeout=5 -- init -- set Open_vSwitch . db-version=6.4.0 Sep 10 11:13:10 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --no-wait --timeout=5 set Open_vSwitch . ovs-version=1.4.2 "external-ids:system-id=\"a67439a6-9aac-4e9e-ac89-a8e1a049c363\"" "system-type=\"Ubuntu\"" "system-version=\"12.04-precise\"" Sep 10 11:13:20 compute ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl add-br br-int Sep 10 11:13:20 compute ovs-vsctl: 00002|vsctl|ERR|cannot create a bridge named br-int because a bridge named br-int already exists Sep 10 11:14:11 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl del-port br-int qvo866a0175-98 Sep 10 11:15:04 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl -- --may-exist add-port br-int qvo010a1cc5-0c -- set Interface qvo010a1cc5-0c external-ids:iface-id=010a1cc5-0c06-4620-b554-f93efa4a4016 external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:94:8c:40 external-ids:vm-uuid=e6906dc0-df8e-4808-89e9-a4aa45362d3d Sep 10 11:16:47 compute ovs-vsctl: 00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl del-port br-int qvo010a1cc5-0c
latechco@compute:/var/log/quantum$ tail -n 100 openvswitch-agent.log Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:48:34 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'set', 'Interface', 'patch-tun', 'options:peer=patch-int']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'set', 'Interface', 'patch-tun', 'options:peer=patch-int'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:48:34 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'get', 'Interface', 'patch-tun', 'ofport']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'patch-tun', 'ofport'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:48:34 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'add-port', 'br-tun', 'patch-int']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'add-port', 'br-tun', 'patch-int'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:48:34 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'set', 'Interface', 'patch-int', 'type=patch']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'set', 'Interface', 'patch-int', 'type=patch'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:48:34 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'set', 'Interface', 'patch-int', 'options:peer=patch-tun']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'set', 'Interface', 'patch-int', 'options:peer=patch-tun'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:48:34 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'get', 'Interface', 'patch-int', 'ofport']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'patch-int', 'ofport'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:48:34 CRITICAL [quantum] int() argument must be a string or a number, not 'NoneType' 2013-09-10 10:55:30 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', '--', '--if-exists', 'del-port', 'br-int', 'patch-tun']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', '--', '--if-exists', 'del-port', 'br-int', 'patch-tun'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:55:30 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-ofctl', 'del-flows', 'br-int']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-ofctl', 'del-flows', 'br-int'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:55:30 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-ofctl', 'add-flow', 'br-int', 'hard_timeout=0,idle_timeout=0,priority=1,actions=normal']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-ofctl', 'add-flow', 'br-int', 'hard_timeout=0,idle_timeout=0,priority=1,actions=normal'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:55:30 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', '--', '--if-exists', 'del-br', 'br-tun']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', '--', '--if-exists', 'del-br', 'br-tun'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:55:30 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'add-br', 'br-tun']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'add-br', 'br-tun'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:55:30 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'add-port', 'br-int', 'patch-tun']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'add-port', 'br-int', 'patch-tun'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:55:30 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'set', 'Interface', 'patch-tun', 'type=patch']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'set', 'Interface', 'patch-tun', 'type=patch'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:55:30 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'set', 'Interface', 'patch-tun', 'options:peer=patch-int']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'set', 'Interface', 'patch-tun', 'options:peer=patch-int'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:55:30 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'get', 'Interface', 'patch-tun', 'ofport']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'patch-tun', 'ofport'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:55:30 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'add-port', 'br-tun', 'patch-int']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'add-port', 'br-tun', 'patch-int'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:55:30 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'set', 'Interface', 'patch-int', 'type=patch']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'set', 'Interface', 'patch-int', 'type=patch'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:55:30 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'set', 'Interface', 'patch-int', 'options:peer=patch-tun']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'set', 'Interface', 'patch-int', 'options:peer=patch-tun'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:55:30 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'get', 'Interface', 'patch-int', 'ofport']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'patch-int', 'ofport'] Exit code: 1 Stdout: '' Stderr: 'sudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: no tty present and no askpass program specified\nSorry, try again.\nsudo: 3 incorrect password attempts\n' 2013-09-10 10:55:30 CRITICAL [quantum] int() argument must be a string or a number, not 'NoneType'
Still have no luck. No IPs assigned :(
Write me back soon!
Cheers!
Lucio
Lucio,
First check /etc/sudoers
- it should have this exact line commented:
`root@compute:~# grep includedir /etc/sudoers
Change sudoers file for quantum like this: `root@compute:~# cat /etc/sudoers.d/quantum_sudoers Defaults:quantum !requiretty
quantum ALL=NOPASSWD: ALL`
Then enable shell access for quantum's user:
root@compute:~# grep quantum /etc/passwd quantum:x:106:112::/var/lib/quantum:/bin/false root@compute:~# usermod -s /bin/bash quantum
Now we can check sudo rights for quantum:
root@compute:~# su - quantum quantum@compute:~# sudo ovs-vsctl show 6ec2edef-7839-4320-9922-14561a958dd0 Bridge br-int Port br-int Interface br-int type: internal Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Bridge br-tun Port br-tun Interface br-tun type: internal Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port "gre-2" Interface "gre-2" type: gre options: {in_key=flow, out_key=flow, remote_ip="10.28.8.89"} ovs_version: "1.4.2"
Finally disable shell access as before:
quantum@compute:~$ exit root@compute:~# usermod -s /bin/false quantum
Restart compute services:
root@compute:~# service openvswitch-switch restart root@compute:~# service quantum-plugin-openvswitch-agent restart root@compute:~# service nova-compute restart
Check the logs and give me some good news :)
Peter
Hi Peter,
look at this:
root@compute:/home/latechco# su - quantum quantum@compute:~$ ovs-vsctl show
Sep 11 08:38:28|00001|stream_unix|ERR|/tmp/stream-unix.1765.0: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 11 08:38:28|00002|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied) Sep 11 08:38:29|00003|stream_unix|ERR|/tmp/stream-unix.1765.1: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 11 08:38:29|00004|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied) Sep 11 08:38:31|00005|stream_unix|ERR|/tmp/stream-unix.1765.2: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 11 08:38:31|00006|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied) Sep 11 08:38:35|00007|stream_unix|ERR|/tmp/stream-unix.1765.3: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 11 08:38:35|00008|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied) Sep 11 08:38:43|00009|stream_unix|ERR|/tmp/stream-unix.1765.4: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 11 08:38:43|00010|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied) Sep 11 08:38:51|00011|stream_unix|ERR|/tmp/stream-unix.1765.5: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 11 08:38:51|00012|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied) Sep 11 08:38:59|00013|stream_unix|ERR|/tmp/stream-unix.1765.6: connection to /var/run/openvswitch/db.sock failed: Permission denied Sep 11 08:38:59|00014|reconnect|WARN|unix:/var/run/openvswitch/db.sock: connection attempt failed (Permission denied)
After tail -n 100 openvswitch-agent.log there are no entries. No odd entries in any log.
Write me back soon!
Lucio
You forgot about sudo ;)
quantum@compute:~# sudo ovs-vsctl show 6ec2edef-7839-4320-9922-14561a958dd0 Bridge br-int Port br-int Interface br-int type: internal Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Bridge br-tun Port br-tun Interface br-tun type: internal Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port "gre-2" Interface "gre-2" type: gre options: {in_key=flow, out_key=flow, remote_ip="10.28.8.89"} ovs_version: "1.4.2"
Peter
Hi Peter,
you were right:
root@compute:/home/latechco# su - quantum quantum@compute:~$ sudo ovs-vsctl show 7cc6e610-ffa1-4a6a-81eb-462108284f41 Bridge br-int Port "qvoab4f161a-34" tag: 1 Interface "qvoab4f161a-34" Port "qvoc48af542-cb" tag: 5 Interface "qvoc48af542-cb" Port "qvo5d723748-97" tag: 6 Interface "qvo5d723748-97" Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port "qvo9251195f-08" Interface "qvo9251195f-08" Port "qvo34bc315b-b2" tag: 5 Interface "qvo34bc315b-b2" Port "qvo58c0fd6e-eb" tag: 5 Interface "qvo58c0fd6e-eb" Port br-int Interface br-int type: internal Port "qvo7b54ec8b-b6" tag: 2 Interface "qvo7b54ec8b-b6" Bridge br-tun Port br-tun Interface br-tun type: internal Port patch-int Interface patch-int type: patch options: {peer=patch-tun} ovs_version: "1.4.2"
Good news! I can ping the private address (50.50.1.2) from the network node, however not the floating ip (192.168.100.103):
Last login: Tue Sep 10 11:41:02 2013 from 10.10.10.51 latechco@network:~$ ping 50.50.1.2 PING 50.50.1.2 (50.50.1.2) 56(84) bytes of data. 64 bytes from 50.50.1.2: icmp_req=1 ttl=107 time=201 ms 64 bytes from 50.50.1.2: icmp_req=2 ttl=107 time=200 ms 64 bytes from 50.50.1.2: icmp_req=3 ttl=107 time=200 ms 64 bytes from 50.50.1.2: icmp_req=4 ttl=107 time=203 ms 64 bytes from 50.50.1.2: icmp_req=5 ttl=107 time=202 ms 64 bytes from 50.50.1.2: icmp_req=6 ttl=107 time=198 ms 64 bytes from 50.50.1.2: icmp_req=7 ttl=107 time=200 ms 64 bytes from 50.50.1.2: icmp_req=8 ttl=107 time=198 ms ^C --- 50.50.1.2 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 7009ms rtt min/avg/max/mdev = 198.458/200.751/203.039/1.558 ms latechco@network:~$ ping 192.168.100.103 PING 192.168.100.103 (192.168.100.103) 56(84) bytes of data. From 192.168.100.103 icmp_seq=1 Destination Host Unreachable From 192.168.100.103 icmp_seq=2 Destination Host Unreachable From 192.168.100.103 icmp_seq=3 Destination Host Unreachable
Hi Peter,
I can ping the vms private address (50.50.1.2) from the controller node too. But ping to another private ip from another vm (50.50.1.4) just stays still:
latechco@controller:~$ ping 50.50.1.2 PING 50.50.1.2 (50.50.1.2) 56(84) bytes of data. 64 bytes from 50.50.1.2: icmp_req=1 ttl=107 time=200 ms 64 bytes from 50.50.1.2: icmp_req=2 ttl=107 time=206 ms 64 bytes from 50.50.1.2: icmp_req=3 ttl=107 time=200 ms 64 bytes from 50.50.1.2: icmp_req=4 ttl=107 time=200 ms 64 bytes from 50.50.1.2: icmp_req=5 ttl=107 time=202 ms ^C --- 50.50.1.2 ping statistics --- 6 packets transmitted, 5 received, 16% packet loss, time 5002ms rtt min/avg/max/mdev = 200.577/202.273/206.580/2.319 ms latechco@controller:~$ ping 50.50.1.4 PING 50.50.1.4 (50.50.1.4) 56(84) bytes of data.
Write me back soon!
You won't be able to reach the subnet 50.50.1.0/24 outside compute node. Also You need to give source interface for ping because without it You are reaching 50.50.1.2 from the internet:
root@compute:~# ifconfig | grep qbr qbrce872663-e0 Link encap:Ethernet HWaddr 76:ef:13:d7:c5:42
root@compute:~# ping 192.168.0.2 -I qbrce872663-e0 PING 192.168.0.2 (192.168.0.2) from 10.28.8.93 qbrce872663-e0: 56(84) bytes of data. 64 bytes from 192.168.0.2: icmp_req=1 ttl=63 time=34.7 ms 64 bytes from 192.168.0.2: icmp_req=2 ttl=63 time=1.27 ms 64 bytes from 192.168.0.2: icmp_req=3 ttl=63 time=1.38 ms 64 bytes from 192.168.0.2: icmp_req=4 ttl=63 time=1.41 ms --- 192.168.0.2 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3003ms rtt min/avg/max/mdev = 1.277/9.713/34.778/14.471 ms
root@compute:~# arp -an|grep 192.168.0.2 ? (192.168.0.2) at fa:16:3e:f2:64:1f [ether] on qbrce872663-e0
login as: cirros Authenticating with public key from agent ifconfig eth0 Link encap:Ethernet HWaddr FA:16:3E:F2:64:1F inet addr:192.168.0.2 Bcast:192.168.0.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fef2:641f/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:190 errors:0 dropped:0 overruns:0 frame:0 TX packets:220 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:23137 (22.5 KiB) TX bytes:28864 (28.1 KiB)
Show me the output from below commands - you need to have user_one credentials loaded:
export OS_TENANT_NAME=project_one export OS_USERNAME=user_one export OS_PASSWORD=user_one export OS_AUTH_URL="http://192.168.100.51:5000/v2.0/"
1)root@controller:~# quantum floatingip-list
2)root@controller:~# quantum port-list
Peter
Hi Peter.
ping to interfaces didn't work:
root@compute:/home/latechco# ifconfig | grep qbr qbra90f033b-b9 Link encap:Ethernet HWaddr 36:69:5d:88:16:29 qbrc48af542-cb Link encap:Ethernet HWaddr da:a0:db:fa:8b:ff qbre1e1d718-d0 Link encap:Ethernet HWaddr 12:e0:92:26:23:d7 root@compute:/home/latechco# ping 192.168.100.103 -I qbra90f033b-b9 PING 192.168.100.103 (192.168.100.103) from 10.10.10.53 qbra90f033b-b9: 56(84) bytes of data. From 10.10.10.53 icmp_seq=1 Destination Host Unreachable From 10.10.10.53 icmp_seq=5 Destination Host Unreachable From 10.10.10.53 icmp_seq=6 Destination Host Unreachable From 10.10.10.53 icmp_seq=7 Destination Host Unreachable From 10.10.10.53 icmp_seq=8 Destination Host Unreachable ^C --- 192.168.100.103 ping statistics --- 9 packets transmitted, 0 received, +5 errors, 100% packet loss, time 7999ms pipe 4 root@compute:/home/latechco# ping 192.168.100.103 -I qbrc48af542-cb PING 192.168.100.103 (192.168.100.103) from 10.10.10.53 qbrc48af542-cb: 56(84) bytes of data. From 10.10.10.53 icmp_seq=1 Destination Host Unreachable From 10.10.10.53 icmp_seq=2 Destination Host Unreachable From 10.10.10.53 icmp_seq=3 Destination Host Unreachable ^C --- 192.168.100.103 ping statistics --- 4 packets transmitted, 0 received, +3 errors, 100% packet loss, time 3013ms pipe 3 root@compute:/home/latechco# ping 192.168.100.103 -I qbre1e1d718-d0 PING 192.168.100.103 (192.168.100.103) from 10.10.10.53 qbre1e1d718-d0: 56(84) bytes of data. From 10.10.10.53 icmp_seq=1 Destination Host Unreachable From 10.10.10.53 icmp_seq=2 Destination Host Unreachable From 10.10.10.53 icmp_seq=3 Destination Host Unreachable ^C --- 192.168.100.103 ping statistics --- 5 packets transmitted, 0 received, +3 errors, 100% packet loss, time 4022ms pipe 3
root@compute:/home/latechco# arp -an|grep 192.168.100.103
? (192.168.100.103) at
root@controller:/home/latechco# source creds_proj_one root@controller:/home/latechco# quantum floatingip-list +--------------------------------------+------------------+---------------------+--------------------------------------+ | id | fixed_ip_address | floating_ip_address | port_id | +--------------------------------------+------------------+---------------------+--------------------------------------+ | 3726d2c1-63ab-452b-9fc6-f065767356dc | 50.50.1.2 | 192.168.100.103 | a90f033b-b9d7-420d-8b81-6facf29f83b9 | | 5066e6fd-6125-4b66-978f-6c64cf293383 | 50.50.1.4 | 192.168.100.104 | e1e1d718-d00a-4281-a14c-8612e065cdfe | +--------------------------------------+------------------+---------------------+--------------------------------------+ root@controller:/home/latechco# quantum port-list +--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+ | id | name | mac_address | fixed_ips | +--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+ | 44efb7a7-40eb-4377-bb87-9fb77fc09b3e | | fa:16:3e:a5:ad:65 | {"subnet_id": "9658838a-116e-408f-9727-b600cd700894", "ip_address": "192.168.100.104"} | | 775f9dd2-ca17-4967-9ab9-e8264bf7fd8b | | fa:16:3e:9c:6b:2e | {"subnet_id": "56f67daa-17f3-4084-8aa2-4e0e5ff3bcce", "ip_address": "50.50.1.3"} | | 8ec32483-1098-4c5c-8686-0f7bd3dd37bf | | fa:16:3e:07:6a:81 | {"subnet_id": "9658838a-116e-408f-9727-b600cd700894", "ip_address": "192.168.100.105"} | | a6d76170-cce4-4f79-a2b8-42fa23c6bcbe | | fa:16:3e:c3:99:a1 | {"subnet_id": "9658838a-116e-408f-9727-b600cd700894", "ip_address": "192.168.100.103"} | | a90f033b-b9d7-420d-8b81-6facf29f83b9 | | fa:16:3e:85:66:5d | {"subnet_id": "56f67daa-17f3-4084-8aa2-4e0e5ff3bcce", "ip_address": "50.50.1.2"} | | b27df820-eeca-448c-87bc-35ddedf8e7de | | fa:16:3e:94:36:69 | {"subnet_id": "56f67daa-17f3-4084-8aa2-4e0e5ff3bcce", "ip_address": "50.50.1.1"} | | e1e1d718-d00a-4281-a14c-8612e065cdfe | | fa:16:3e:ae:09:3e | {"subnet_id": "56f67daa-17f3-4084-8aa2-4e0e5ff3bcce", "ip_address": "50.50.1.4"} | | f0456333-eb07-4db2-a2f4-6045db0a15cf | | fa:16:3e:19:26:96 | {"subnet_id": "9658838a-116e-408f-9727-b600cd700894", "ip_address": "192.168.100.102"} | +--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+
Thanks again!
Write me back soon!
Remember to use creds for user_one and show me
root@controller:~# nova list
root@controller:~# nova secgroup-list
root@controller:~# nova secgroup-list-rules default
We need to check whether You added rules for icmp traffic ;)
If this won't help - delete all instances, reboot all nodes and boot cirros again.
I want to see if it gets IP address this time from console output. Good luck!
Peter
Hi Peter,
here's the data before the three nodes reboot:
root@controller:/home/latechco# source creds_proj_one
root@controller:/home/latechco# nova list +--------------------------------------+---------+--------+-----------------------------------------+ | ID | Name | Status | Networks | +--------------------------------------+---------+--------+-----------------------------------------+ | e9d540c8-81f8-4531-8d26-9d11fd9c9d72 | Lucio | ACTIVE | net_proj_one=50.50.1.2, 192.168.100.103 | | 7724e02c-c980-4d6b-8824-bbc708c9567f | Lucio 2 | ACTIVE | net_proj_one=50.50.1.4, 192.168.100.104 | +--------------------------------------+---------+--------+-----------------------------------------+
root@controller:/home/latechco# nova secgroup-list +---------+-------------+ | Name | Description | +---------+-------------+ | default | default | +---------+-------------+
root@controller:/home/latechco# nova secgroup-list-rules default +-------------+-----------+---------+-----------+--------------+ | IP Protocol | From Port | To Port | IP Range | Source Group | +-------------+-----------+---------+-----------+--------------+ | icmp | -1 | -1 | 0.0.0.0/0 | | | tcp | 22 | 22 | 0.0.0.0/0 | | +-------------+-----------+---------+-----------+--------------+
Here's the data after the three nodes reboot:
Starting network... udhcpc (v1.18.5) started Sending discover... Sending discover... Sending discover... No lease, failing WARN: /etc/rc3.d/S40network failed cloudsetup: checking http://169.254.169.254/20090404/metadata/instanceid wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 1/30: up 9.99. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 2/30: up 11.00. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 3/30: up 12.00. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 4/30: up 13.01. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 5/30: up 14.02. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 6/30: up 15.02. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 7/30: up 16.03. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 8/30: up 17.04. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 9/30: up 18.04. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 10/30: up 19.05. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 11/30: up 20.06. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 12/30: up 21.06. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 13/30: up 22.07. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 14/30: up 23.08. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 15/30: up 24.08. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 16/30: up 25.09. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 17/30: up 26.10. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 18/30: up 27.10. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 19/30: up 28.11. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 20/30: up 29.11. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 21/30: up 30.12. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 22/30: up 31.13. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 23/30: up 32.13. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 24/30: up 33.14. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 25/30: up 34.15. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 26/30: up 35.15. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 27/30: up 36.16. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 28/30: up 37.17. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 29/30: up 38.17. request failed wget: can't connect to remote host (169.254.169.254): Network is unreachable cloudsetup: failed 30/30: up 39.18. request failed cloudsetup: after 30 fails, debugging
Still no luck :(
Greetings,
Lucio
Peter,
thank you very much for your support.
Today I just reinstalled the three servers and had an issue with openvswitch on the Network node.
After all what I did was not to upgrade the Network node which was causing me issues with openvswitch and finally I can ping and ssh to my cirros instance.
Thank you very much again!
Lucio
Nice to hear that Lucio - I'm glad I could help :) I think we can close this issue now.
Cheers, Peter
@moss2k13 , So do you suggest to upgrade all nodes ubuntu with : deb http://archive.gplhost.com/debian grizzly main deb http://archive.gplhost.com/debian grizzly-backports main
at the moment I have
openvswitch-switch | 1.4.0-1ubuntu1.5 | http://gb.archive.ubuntu.com/ubuntu/ precise-updates/universe amd64 Packages openvswitch-switch | 1.4.0-1ubuntu1 | http://gb.archive.ubuntu.com/ubuntu/ precise/universe amd64 Packages openvswitch | 1.4.0-1ubuntu1 | http://gb.archive.ubuntu.com/ubuntu/ precise/universe Sources openvswitch | 1.4.0-1ubuntu1.5 | http://gb.archive.ubuntu.com/ubuntu/ precise-updates/universe Sources
I have 2 compute / 1 net / 1 cont:
I can ping vm each other, I can ssh from outside , inside the vm I can ping google and bbc but I can't do apt-get update from example a vm with ubuntu cloud image or not able to surf from a vm instace with ubuntu desktop . so I think is a problem with DNs .
grep dns /var/log/syslog loki is my net node ! from net node : infinity Sep 24 14:32:03 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a 50-50-1-3 Sep 24 14:32:43 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:32:43 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:33:03 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a Sep 24 14:33:03 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a 50-50-1-3 Sep 24 14:33:37 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:33:37 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:34:03 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a Sep 24 14:34:03 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a 50-50-1-3 Sep 24 14:34:21 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:34:21 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:35:03 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a Sep 24 14:35:03 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a 50-50-1-3 Sep 24 14:35:05 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:35:05 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:35:51 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:35:51 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:36:03 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a Sep 24 14:36:03 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a 50-50-1-3 Sep 24 14:36:39 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:36:39 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:37:03 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a Sep 24 14:37:03 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a 50-50-1-3 Sep 24 14:37:29 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:37:29 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:38:03 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a Sep 24 14:38:03 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a 50-50-1-3 Sep 24 14:38:16 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:38:16 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:39:03 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a Sep 24 14:39:03 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a 50-50-1-3 Sep 24 14:39:07 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:39:07 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:39:58 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:39:58 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:40:03 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a Sep 24 14:40:03 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a 50-50-1-3 Sep 24 14:40:54 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:40:54 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:41:03 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a Sep 24 14:41:03 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a 50-50-1-3 Sep 24 14:41:45 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:41:45 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:42:03 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a Sep 24 14:42:03 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a 50-50-1-3 Sep 24 14:42:29 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:42:29 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:43:03 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a Sep 24 14:43:03 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a 50-50-1-3 Sep 24 14:43:21 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:43:21 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:44:04 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a Sep 24 14:44:04 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a 50-50-1-3 Sep 24 14:44:04 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:44:04 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:44:56 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:44:56 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:45:04 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a Sep 24 14:45:04 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a 50-50-1-3 Sep 24 14:45:45 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:45:45 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:46:04 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a Sep 24 14:46:04 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a 50-50-1-3 Sep 24 14:46:34 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:46:34 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4 Sep 24 14:47:04 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a Sep 24 14:47:04 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.3 fa:16:3e:32:fa:5a 50-50-1-3 Sep 24 14:47:17 loki dnsmasq-dhcp[19298]: DHCPREQUEST(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf Sep 24 14:47:17 loki dnsmasq-dhcp[19298]: DHCPACK(tap689b75b7-f5) 50.50.1.4 fa:16:3e:cb:f7:cf 50-50-1-4
also quantum port-list +--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+ | id | name | mac_address | fixed_ips | +--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+ | 01c154df-4573-4c7f-a276-9965eb2cdd37 | | fa:16:3e:cb:f7:cf | {"subnet_id": "3f2c9eef-62fb-4698-9887-a7cc0ff5f361", "ip_address": "50.50.1.4"} | | 0dc95fec-89c0-405b-895b-ec2362fa5392 | | fa:16:3e:4e:22:e8 | {"subnet_id": "ea685e5e-103c-4649-8984-3c283f7d6f97", "ip_address": "192.168.221.103"} | | 12c93ea2-d1af-4f86-b7de-587f5153856e | | fa:16:3e:32:fa:5a | {"subnet_id": "3f2c9eef-62fb-4698-9887-a7cc0ff5f361", "ip_address": "50.50.1.3"} | | 659af233-a77a-49fe-b607-79f4586faf14 | | fa:16:3e:cf:b3:53 | {"subnet_id": "3f2c9eef-62fb-4698-9887-a7cc0ff5f361", "ip_address": "50.50.1.1"} | | 689b75b7-f5e8-4b9b-8518-c73ad54b9b7e | | fa:16:3e:10:21:7b | {"subnet_id": "3f2c9eef-62fb-4698-9887-a7cc0ff5f361", "ip_address": "50.50.1.2"} | | 83bd8240-a1f4-4ce3-9e4d-f9e1c1eacea9 | | fa:16:3e:22:19:2a | {"subnet_id": "ea685e5e-103c-4649-8984-3c283f7d6f97", "ip_address": "192.168.221.104"} | | a0c04933-4a58-4176-81d3-4329550bbeda | | fa:16:3e:46:8b:45 | {"subnet_id": "ea685e5e-103c-4649-8984-3c283f7d6f97", "ip_address": "192.168.221.102"} | | f65bd825-63f5-48d9-b6ec-9638edb183e2 | | fa:16:3e:cb:98:01 | {"subnet_id": "ea685e5e-103c-4649-8984-3c283f7d6f97", "ip_address": "192.168.221.105"} | +--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+
so internal net 50.50.1.0/24 and ext 192.168.221.100/24 when I do arp -an|grep fa:16:3e:cb:f7:cf ? (50.50.1.3) at fa:16:3e:32:fa:5a [ether] on qbr12c93ea2-d1 sometime is working but not everytime but this never work ping 50.50.1.3 -I qbr12c93ea2-d1
but from my netwok node : ip netns qrouter-9f7482ce-a0a7-45ed-b978-37dbf2dd3bc5 qdhcp-f92614a6-bbd0-404c-a59b-86eed7428298 ip netns exec qrouter-9f7482ce-a0a7-45ed-b978-37dbf2dd3bc5 ping 50.50.1.3 PING 50.50.1.4 (50.50.1.4) 56(84) bytes of data. 64 bytes from 50.50.1.3: icmp_req=1 ttl=64 time=22.0 ms 64 bytes from 50.50.1.3: icmp_req=2 ttl=64 time=0.353 ms
from compute 1 :ovs-vsctl show 43f78e6a-5a48-4baa-9986-9d9a526fb534 Bridge br-int Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port "qvo01c154df-45" tag: 1 Interface "qvo01c154df-45" Port br-int Interface br-int type: internal Bridge br-tun Port br-tun Interface br-tun type: internal Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port "gre-2" Interface "gre-2" type: gre options: {in_key=flow, out_key=flow, remote_ip="10.0.5.232"} Port "gre-1" Interface "gre-1" type: gre options: {in_key=flow, out_key=flow, remote_ip="10.0.5.230"} ovs_version: "1.4.0+build0"
from compute 2 : ovs-vsctl show 82a85a82-b20e-4b5b-9dc3-1d61b2f20f4a Bridge br-tun Port br-tun Interface br-tun type: internal Port "gre-3" Interface "gre-3" type: gre options: {in_key=flow, out_key=flow, remote_ip="10.0.5.233"} Port "gre-1" Interface "gre-1" type: gre options: {in_key=flow, out_key=flow, remote_ip="10.0.5.230"} Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Bridge br-int Port "qvo12c93ea2-d1" tag: 1 Interface "qvo12c93ea2-d1" Port "qvo4d9c511c-a3" tag: 4095 Interface "qvo4d9c511c-a3" Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port br-int Interface br-int type: internal ovs_version: "1.4.0+build0"
from net node : ovs-vsctl show 8f5dd9f1-77ec-4bec-a72f-06dde7fe5218 Bridge br-tun Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port "gre-3" Interface "gre-3" type: gre options: {in_key=flow, out_key=flow, remote_ip="10.0.5.233"} Port "gre-2" Interface "gre-2" type: gre options: {in_key=flow, out_key=flow, remote_ip="10.0.5.232"} Port br-tun Interface br-tun type: internal Bridge br-ex Port "qg-a0c04933-4a" Interface "qg-a0c04933-4a" type: internal Port "eth1" Interface "eth1" Port br-ex Interface br-ex type: internal Bridge br-int Port "tap689b75b7-f5" tag: 1 Interface "tap689b75b7-f5" type: internal Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port "qr-659af233-a7" tag: 1 Interface "qr-659af233-a7" type: internal Port br-int Interface br-int type: internal ovs_version: "1.4.0+build0"
I have notice that : all cat /var/log/quantum/openvswitch-agent.log is clean but just the node that I restart the service "quantum-plugin-openvswitch-agent"and the other two node : get some error :
ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'add-port', 'br-tun', 'gre-3']. Exception: Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'add-port', 'br-tun', 'gre-3'] Exit code: 1 Stdout: '' Stderr: 'ovs-vsctl: cannot create a port named gre-3 because a port named gre-3 already exists on bridge br-tun\n'
then depend wherre I restart the service I get gre-1 or gre-2
what else I need to check ?
Hello All, I am struggling a week...to configure the openstack in 4 nodes but cannot ping instances, I can create instances router etc but VMs does not get an ip
Sending discover... Sending discover... Sending discover... No lease, failing WARN: /etc/rc3.d/S40-network failed cirros-ds 'net' up at 181.08 checking http://169.254.169.254/2009-04-04/instance-id failed 1/20: up 181.09. request failed failed 2/20: up 183.16. request failed failed 3/20: up 185.17. request failed failed 4/20: up 187.17. request failed failed 5/20: up 189.18. request failed failed 6/20: up 191.18. request failed failed 7/20: up 193.19. request failed failed 8/20: up 195.20. request failed failed 9/20: up 197.20. request failed
Is it related with ovswitch? or quantom.. I dont have any errors in the log files.. Is it related to the kernel?
please I need some help!
Hello,
This issue has been closed Stelio - investigate similar issues, below I will give you some hints:
1)make sure you enabled dhcp during subnet creation,
2)make sure you don't have errors in /var/log/quantum/openvswitch-agent.log
- check https://github.com/mseknibilel/OpenStack-Grizzly-Install-Guide/issues/133#issuecomment-27088805
3)make sure you have gre tunnels up and running using ovs-vsctl show
command
If after that you still have problems you had to missed something during installation - mseknibilel's guide works.
Good luck! Peter
Hi all,
I followed this how to: https://github.com/mseknibilel/OpenStack-Grizzly-Install-Guide/blob/OVS_MultiNode/OpenStack_Grizzly_Install_Guide.rst exaclty as it says.
However I can't ping the vm, nor ssh it. Instead I can ping the interface of the router and the default gateway.
Any help will be appreciated as I've been a week trying to connect to my vm.
Greetings,
Lucio