search

sap-oc / crowbar-openstack

Openstack deployment for Crowbar
3 stars 1 forks source link

Nova/Neutron: Enable rootwrap daemon mode #26

Open matelakat opened 7 years ago

matelakat commented 7 years ago

Original commit message:

This provides a speedup (benchmarks that do a lot of neutron operations
go down from 24 minutes to 11 minutes, providing an almost twofold
improvement). The improvement is actually like eightfold on ARM.

(cherry picked from commit ef29f92efe8d8dbc269c3ea127985a9f2cc16922)

Fixes sap-oc/crowbar-openstack#1

Sudoers File for Neutron

Specfile

install -D -m 440 %{SOURCE3} %{buildroot}%{_sysconfdir}/sudoers.d/openstack-neutron

Contents

The rootwrap-daemon line is new:

Defaults:neutron !syslog, !pam_session, !use_pty
neutron ALL = (root) NOPASSWD: /usr/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf
neutron ALL = (root) NOPASSWD: /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf *

Sudoers File for Nova

Specfile

install -D -m 440 %{SOURCE5} %{buildroot}%{_sysconfdir}/sudoers.d/openstack-nova

Contents

The rootwrap-daemon line is new:

Defaults:nova !syslog, !pam_session, !use_pty
nova ALL = (root) NOPASSWD: /usr/bin/nova-rootwrap-daemon /etc/nova/rootwrap.conf
nova ALL = (root) NOPASSWD: /usr/bin/nova-rootwrap /etc/nova/rootwrap.conf *
matelakat commented 7 years ago

I still need to verify this.

aspiers commented 7 years ago

I'm testing this now.

aspiers commented 7 years ago

Here's a sample of the speed difference. Individual rootwrap commands take roughly 10ms via the daemon:

2017-03-21 21:04:11.767 29740 DEBUG neutron.agent.linux.utils [...] Running command (rootwrap daemon): ['ip', 'link', 'set', 'qg-fccd10fb-6f', 'address', 'fa:16:3e:5d:d2:01'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:101
2017-03-21 21:04:11.775 29740 DEBUG neutron.agent.linux.utils [...] Running command (rootwrap daemon): ['ip', '-o', 'netns', 'list'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:101
2017-03-21 21:04:11.781 29740 DEBUG neutron.agent.linux.utils [...] Running command (rootwrap daemon): ['ip', 'link', 'set', 'qg-fccd10fb-6f', 'netns', 'qrouter-3fdf4656-d57d-42d5-acfb-787093d56b57'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:101
2017-03-21 21:04:11.801 29740 DEBUG neutron.agent.linux.utils [...] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-3fdf4656-d57d-42d5-acfb-787093d56b57', 'ip', 'link', 'set', 'qg-fccd10fb-6f', 'mtu', '1500'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:101
2017-03-21 21:04:11.808 29740 DEBUG neutron.agent.linux.utils [...] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-3fdf4656-d57d-42d5-acfb-787093d56b57', 'ip', 'link', 'set', 'qg-fccd10fb-6f', 'up'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:101
2017-03-21 21:04:11.815 29740 DEBUG neutron.agent.linux.utils [...] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-3fdf4656-d57d-42d5-acfb-787093d56b57', 'ip', '-o', 'link', 'show', 'qg-fccd10fb-6f'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:101
2017-03-21 21:04:11.824 29740 DEBUG neutron.agent.linux.utils [...] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-3fdf4656-d57d-42d5-acfb-787093d56b57', 'ip', 'addr', 'show', 'qg-fccd10fb-6f', 'permanent'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:101
2017-03-21 21:04:11.834 29740 DEBUG neutron.agent.linux.utils [...] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-3fdf4656-d57d-42d5-acfb-787093d56b57', 'ip', '-4', 'addr', 'add', '192.168.238.142/24', 'scope', 'global', 'dev', 'qg-fccd10fb-6f', 'brd', '192.168.238.255'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:101
2017-03-21 21:04:11.844 29740 DEBUG neutron.agent.linux.utils [...] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-3fdf4656-d57d-42d5-acfb-787093d56b57', 'ip', '-4', 'route', 'replace', 'default', 'via', '192.168.238.1', 'dev', 'qg-fccd10fb-6f'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:101
2017-03-21 21:04:11.851 29740 DEBUG neutron.agent.linux.utils [...] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-3fdf4656-d57d-42d5-acfb-787093d56b57', 'sysctl', '-w', 'net.ipv6.conf.qg-fccd10fb-6f.accept_ra=2'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:101
2017-03-21 21:04:11.860 29740 DEBUG neutron.agent.linux.utils [...] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-3fdf4656-d57d-42d5-acfb-787093d56b57', 'ip', '-4', 'route', 'list', 'dev', 'qg-fccd10fb-6f', 'scope', 'link'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:101
2017-03-21 21:04:11.904 29740 DEBUG neutron.agent.linux.utils [...] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-3fdf4656-d57d-42d5-acfb-787093d56b57', 'ip', '-6', 'route', 'list', 'dev', 'qg-fccd10fb-6f', 'scope', 'link'] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:101
2017-03-21 21:04:11.918 29740 DEBUG neutron.agent.linux.utils [...] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-3fdf4656-d57d-42d5-acfb-787093d56b57', 'find', '/sys/class/net', '-maxdepth', '1', '-type', 'l', '-printf', '%f '] execute_rootwrap_daemon /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:101

vs. 100ms without:

2017-03-21 21:20:51.867 350 DEBUG neutron.agent.linux.utils [...] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'link', 'set', 'qr-3f68630e-b9', 'address', 'fa:16:3e:e6:41:75'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:85
2017-03-21 21:20:51.973 350 DEBUG neutron.agent.linux.utils [...] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', '-o', 'netns', 'list'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:85
2017-03-21 21:20:52.085 350 DEBUG neutron.agent.linux.utils [...] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'link', 'set', 'qr-3f68630e-b9', 'netns', 'qrouter-bdbbbfb7-ff13-4414-87f4-e8663331a28b'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:85
2017-03-21 21:20:52.257 350 DEBUG neutron.agent.linux.utils [...] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-bdbbbfb7-ff13-4414-87f4-e8663331a28b', 'ip', 'link', 'set', 'qr-3f68630e-b9', 'mtu', '1458'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:85
2017-03-21 21:20:52.418 350 DEBUG neutron.agent.linux.utils [...] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-bdbbbfb7-ff13-4414-87f4-e8663331a28b', 'ip', 'link', 'set', 'qr-3f68630e-b9', 'up'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:85
2017-03-21 21:20:52.606 350 DEBUG neutron.agent.linux.utils [...] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-bdbbbfb7-ff13-4414-87f4-e8663331a28b', 'ip', '-o', 'link', 'show', 'qr-3f68630e-b9'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:85
2017-03-21 21:20:52.721 350 DEBUG neutron.agent.linux.utils [...] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-bdbbbfb7-ff13-4414-87f4-e8663331a28b', 'ip', 'addr', 'show', 'qr-3f68630e-b9', 'permanent'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:85
2017-03-21 21:20:52.828 350 DEBUG neutron.agent.linux.utils [...] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-bdbbbfb7-ff13-4414-87f4-e8663331a28b', 'ip', '-4', 'addr', 'add', '192.168.123.1/24', 'scope', 'global', 'dev', 'qr-3f68630e-b9', 'brd', '192.168.123.255'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:85
2017-03-21 21:20:52.975 350 DEBUG neutron.agent.linux.utils [...] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-bdbbbfb7-ff13-4414-87f4-e8663331a28b', 'ip', '-4', 'route', 'list', 'dev', 'qr-3f68630e-b9', 'scope', 'link'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:85
2017-03-21 21:20:53.053 350 DEBUG neutron.agent.linux.utils [...] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-bdbbbfb7-ff13-4414-87f4-e8663331a28b', 'ip', '-6', 'route', 'list', 'dev', 'qr-3f68630e-b9', 'scope', 'link'] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:85
2017-03-21 21:20:53.133 350 DEBUG neutron.agent.linux.utils [...] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-bdbbbfb7-ff13-4414-87f4-e8663331a28b', 'find', '/sys/class/net', '-maxdepth', '1', '-type', 'l', '-printf', '%f '] create_process /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:85

Migration of 10 routers with the daemon takes ~43s:

root@controller1:~ # time neutron-ha-tool --l3-agent-evacuate d52-54-77-77-77-01
2017-03-21 21:00:00,613 requests.packages.urllib3.connectionpool INFO     Starting new HTTP connection (1): cluster-services.vp8.cloud.suse.de
2017-03-21 21:00:00,819 neutron-ha-tool INFO     Performing L3 Agent Evacuation from host d52-54-77-77-77-01
2017-03-21 21:00:01,004 neutron-ha-tool INFO     Querying agent_id=77b3edb6-9066-4447-af92-6b2f6fe69677 for routers to migrate away
2017-03-21 21:00:01,267 neutron-ha-tool INFO     Refreshing router count per agent cache
2017-03-21 21:00:01,290 neutron-ha-tool INFO     Migrating router=bdbbbfb7-ff13-4414-87f4-e8663331a28b from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:00:01,662 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=bdbbbfb7-ff13-4414-87f4-e8663331a28b to be ACTIVE
2017-03-21 21:00:05,153 neutron-ha-tool INFO     Migrating router=079bf0b9-22e6-48c8-829c-881e4ff8efe3 from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:00:05,509 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=079bf0b9-22e6-48c8-829c-881e4ff8efe3 to be ACTIVE
2017-03-21 21:00:08,882 neutron-ha-tool INFO     Migrating router=5cc10af6-b496-40ad-a805-c79fec4c3296 from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:00:09,327 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=5cc10af6-b496-40ad-a805-c79fec4c3296 to be ACTIVE
2017-03-21 21:00:12,682 neutron-ha-tool INFO     Migrating router=a4773172-27cd-40b8-a589-5e0b5a768a4d from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:00:13,016 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=a4773172-27cd-40b8-a589-5e0b5a768a4d to be ACTIVE
2017-03-21 21:00:16,359 neutron-ha-tool INFO     Migrating router=4a02ff92-3416-4b24-8f7f-3968968b9fb3 from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:00:16,699 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=4a02ff92-3416-4b24-8f7f-3968968b9fb3 to be ACTIVE
2017-03-21 21:00:18,958 neutron-ha-tool INFO     Migrating router=af899582-6d14-4584-82fd-f0bd2dced806 from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:00:19,351 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=af899582-6d14-4584-82fd-f0bd2dced806 to be ACTIVE
2017-03-21 21:00:23,224 neutron-ha-tool INFO     Migrating router=b041091b-fb0c-40b7-b0ff-82237b3afaf2 from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:00:23,557 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=b041091b-fb0c-40b7-b0ff-82237b3afaf2 to be ACTIVE
2017-03-21 21:00:26,878 neutron-ha-tool INFO     Migrating router=31621795-3b51-4299-b7d6-bc80462cccbf from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:00:27,187 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=31621795-3b51-4299-b7d6-bc80462cccbf to be ACTIVE
2017-03-21 21:00:31,077 neutron-ha-tool INFO     Migrating router=e11d1179-2e7a-4cf6-895c-0c660519f306 from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:00:31,439 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=e11d1179-2e7a-4cf6-895c-0c660519f306 to be ACTIVE
2017-03-21 21:00:34,886 neutron-ha-tool INFO     Migrating router=3f41c314-dd25-4a41-b507-6a2b2e4b78fe from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:00:35,191 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=3f41c314-dd25-4a41-b507-6a2b2e4b78fe to be ACTIVE
2017-03-21 21:00:39,187 neutron-ha-tool INFO     Migrating router=3fdf4656-d57d-42d5-acfb-787093d56b57 from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:00:39,481 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=3fdf4656-d57d-42d5-acfb-787093d56b57 to be ACTIVE
2017-03-21 21:00:43,337 neutron-ha-tool INFO     11 routers were evacuated from L3 agent d52-54-77-77-77-01

real    0m43.342s
user    0m0.812s
sys     0m0.104s

and ~84s without:

root@controller1:~ # time neutron-ha-tool --l3-agent-evacuate d52-54-77-77-77-01
2017-03-21 21:24:46,732 requests.packages.urllib3.connectionpool INFO     Starting new HTTP connection (1): cluster-services.vp8.cloud.suse.de
2017-03-21 21:24:46,917 neutron-ha-tool INFO     Performing L3 Agent Evacuation from host d52-54-77-77-77-01
2017-03-21 21:24:47,096 neutron-ha-tool INFO     Querying agent_id=77b3edb6-9066-4447-af92-6b2f6fe69677 for routers to migrate away
2017-03-21 21:24:47,276 neutron-ha-tool INFO     Refreshing router count per agent cache
2017-03-21 21:24:47,290 neutron-ha-tool INFO     Migrating router=bdbbbfb7-ff13-4414-87f4-e8663331a28b from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:24:47,612 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=bdbbbfb7-ff13-4414-87f4-e8663331a28b to be ACTIVE
2017-03-21 21:24:54,994 neutron-ha-tool INFO     Migrating router=079bf0b9-22e6-48c8-829c-881e4ff8efe3 from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:24:55,476 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=079bf0b9-22e6-48c8-829c-881e4ff8efe3 to be ACTIVE
2017-03-21 21:25:02,150 neutron-ha-tool INFO     Migrating router=5cc10af6-b496-40ad-a805-c79fec4c3296 from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:25:02,534 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=5cc10af6-b496-40ad-a805-c79fec4c3296 to be ACTIVE
2017-03-21 21:25:09,793 neutron-ha-tool INFO     Migrating router=a4773172-27cd-40b8-a589-5e0b5a768a4d from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:25:10,132 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=a4773172-27cd-40b8-a589-5e0b5a768a4d to be ACTIVE
2017-03-21 21:25:16,699 neutron-ha-tool INFO     Migrating router=4a02ff92-3416-4b24-8f7f-3968968b9fb3 from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:25:17,002 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=4a02ff92-3416-4b24-8f7f-3968968b9fb3 to be ACTIVE
2017-03-21 21:25:23,632 neutron-ha-tool INFO     Migrating router=af899582-6d14-4584-82fd-f0bd2dced806 from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:25:24,018 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=af899582-6d14-4584-82fd-f0bd2dced806 to be ACTIVE
2017-03-21 21:25:30,806 neutron-ha-tool INFO     Migrating router=b041091b-fb0c-40b7-b0ff-82237b3afaf2 from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:25:31,557 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=b041091b-fb0c-40b7-b0ff-82237b3afaf2 to be ACTIVE
2017-03-21 21:25:38,896 neutron-ha-tool INFO     Migrating router=31621795-3b51-4299-b7d6-bc80462cccbf from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:25:39,297 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=31621795-3b51-4299-b7d6-bc80462cccbf to be ACTIVE
2017-03-21 21:25:46,581 neutron-ha-tool INFO     Migrating router=e11d1179-2e7a-4cf6-895c-0c660519f306 from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:25:46,949 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=e11d1179-2e7a-4cf6-895c-0c660519f306 to be ACTIVE
2017-03-21 21:25:54,805 neutron-ha-tool INFO     Migrating router=3f41c314-dd25-4a41-b507-6a2b2e4b78fe from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:25:55,220 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=3f41c314-dd25-4a41-b507-6a2b2e4b78fe to be ACTIVE
2017-03-21 21:26:02,419 neutron-ha-tool INFO     Migrating router=3fdf4656-d57d-42d5-acfb-787093d56b57 from agent=77b3edb6-9066-4447-af92-6b2f6fe69677 to agent=fe1906c7-f8ea-47aa-b8c1-7fa6676a1c81
2017-03-21 21:26:02,771 neutron-ha-tool INFO     Wait for the ports and floating IPs of router_id=3fdf4656-d57d-42d5-acfb-787093d56b57 to be ACTIVE
2017-03-21 21:26:10,111 neutron-ha-tool INFO     11 routers were evacuated from L3 agent d52-54-77-77-77-01

real    1m23.947s
user    0m1.016s
sys     0m0.104s

So even though each rootwrap command is now roughly 10x faster, the router migration speed only doubled. It seems that the bottleneck now is waiting for the ports to become ACTIVE, although this seems to be mainly about waiting for rootwrap commands to complete, and to a lesser extent waiting for the neutron-server to notice the state changes.

aspiers commented 7 years ago

Patched oslo.rootwrap package is now in my OBS branch: https://build.opensuse.org/package/rdiff/home:aspiers:branches:Cloud:OpenStack:Liberty/python-oslo.rootwrap?opackage=python-oslo.rootwrap&oproject=Cloud%3AOpenStack%3ALiberty&rev=2

matelakat commented 7 years ago

@aspiers done

aspiers commented 7 years ago

I've now submitted the necessary changes to the nova and neutron packages:

mkoderer commented 7 years ago

Changes LGTM - we should test all together