tomoe
commented
11 years ago this was due to nova-compute's periodical tasks.
Closed ryu25ish closed 11 years ago
tomoe
commented
11 years ago this was due to nova-compute's periodical tasks.
From Takayuki:
In summary, to create a single VM instance, quantum-server handled 16 requests taking 7.66 secs, meanwhile api handle 38 requests sent from quantum-server taking 6.82 secs. That is, quantum-server spent 89% of response time to waiting for api. Can you believe this?
I have been working on the large latencies of quantum-server lately. While I was struggling it for a week or so, I came across several concurrency issues that has been fixed or worked around.
Then, now I believe quantum-server's latencies is bound by our api's latencies. Here is the (long) list of all the requests that quantum-server and api handled to create _asingle VM instance.
Takayuki
The leftmost column shows how long it took to respond in secs.
quantum-server: 0.798865 "GET //v2.0/networks.json HTTP/1.1" 200 634 0.491398 "POST //v2.0/subnets.json HTTP/1.1" 500 215 1.497892 "GET //v2.0/subnets.json HTTP/1.1" 200 888 0.319304 "GET //v2.0/routers.json HTTP/1.1" 200 378 0.100125 "PUT //v2.0/routers/d2120936-5940-458c-96e5-226dd5c662e7/add_router_interface.json HTTP/1.1" 400 230 0.377872 "GET /v2.0/networks.json?tenant_id=bca66a16c8bc46f29957fbb4468bdb68&shared=False HTTP/1.1" 200 380 0.184456 "GET /v2.0/networks.json?shared=True HTTP/1.1" 200 124 1.877901 "POST /v2.0/ports.json HTTP/1.1" 201 549 0.343977 "GET /v2.0/ports.json?tenant_id=bca66a16c8bc46f29957fbb4468bdb68&device_id=5fcc7b9d-712a-4f18-b814-4a0ba41c9db0 HTTP/1.1" 200 547 0.508770 "GET /v2.0/subnets.json?id=986e9f01-f12d-4b54-96c0-1111b15445c3 HTTP/1.1" 200 510 0.019115 "GET /v2.0/ports.json?network_id=3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7&device_owner=network%3Adhcp HTTP/1.1" 200 121 0.272698 "GET /v2.0/ports.json?tenant_id=bca66a16c8bc46f29957fbb4468bdb68&device_id=5fcc7b9d-712a-4f18-b814-4a0ba41c9db0 HTTP/1.1" 200 547 0.291828 "GET /v2.0/networks.json?tenant_id=bca66a16c8bc46f29957fbb4468bdb68&shared=False HTTP/1.1" 200 380 0.146555 "GET /v2.0/networks.json?shared=True HTTP/1.1" 200 124 0.409711 "GET /v2.0/subnets.json?id=986e9f01-f12d-4b54-96c0-1111b15445c3 HTTP/1.1" 200 510 0.019507 "GET /v2.0/ports.json?network_id=3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7&device_owner=network%3Adhcp HTTP/1.1" 200 121 7.65 (total)
api: 0.167 "GET /midolmanj-mgmt/bridges?tenant_id=bca66a16c8bc46f29957fbb4468bdb68 HTTP/1.1" 200 746 0.287 "GET /midolmanj-mgmt/bridges/19ad0957-b2d4-49c1-a917-a07b4b036f4b HTTP/1.1" 200 742 0.179 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 0.230 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 0.154 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 0.237 "GET /midolmanj-mgmt/bridges/19ad0957-b2d4-49c1-a917-a07b4b036f4b HTTP/1.1" 200 742 0.330 "GET /midolmanj-mgmt/bridges/19ad0957-b2d4-49c1-a917-a07b4b036f4b/dhcp HTTP/1.1" 200 424 0.195 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 0.171 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7/dhcp HTTP/1.1" 200 409 0.221 "GET /midolmanj-mgmt/bridges/19ad0957-b2d4-49c1-a917-a07b4b036f4b HTTP/1.1" 200 742 0.282 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 0.220 "GET /midolmanj-mgmt/routers/d2120936-5940-458c-96e5-226dd5c662e7 HTTP/1.1" 200 874 0.175 "GET /midolmanj-mgmt/bridges?tenant_id=bca66a16c8bc46f29957fbb4468bdb68 HTTP/1.1" 200 746 0.239 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 0.165 "GET /midolmanj-mgmt/bridges?tenant_id=bca66a16c8bc46f29957fbb4468bdb68 HTTP/1.1" 200 746 0.257 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 0.149 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 0.143 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 0.263 "POST /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7/ports HTTP/1.1" 201 - 0.188 "GET /midolmanj-mgmt/ports/2080c5ec-d303-4fcf-9218-51ace9f1a593 HTTP/1.1" 200 563 0.101 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 0.172 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7/dhcp HTTP/1.1" 200 409 0.173 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7/dhcp HTTP/1.1" 200 409 0.187 "POST /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7/dhcp/172.16.3.0_24/hosts HTTP/1.1" 201 - 0.151 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7/dhcp/172.16.3.0_24/hosts/fa-16-3e-b8-fa-45 HTTP/1.1" 200 215 0.127 "GET /midolmanj-mgmt/ports/2080c5ec-d303-4fcf-9218-51ace9f1a593 HTTP/1.1" 200 563 0.175 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 0.167 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 0.154 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7/dhcp HTTP/1.1" 200 409 0.144 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 0.085 "GET /midolmanj-mgmt/ports/2080c5ec-d303-4fcf-9218-51ace9f1a593 HTTP/1.1" 200 797 0.153 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 0.141 "GET /midolmanj-mgmt/bridges?tenant_id=28a27ec5e74f4ca399dd8006d7e8aa9c HTTP/1.1" 200 1492 0.132 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 0.128 "GET /midolmanj-mgmt/bridges?tenant_id=28a27ec5e74f4ca399dd8006d7e8aa9c HTTP/1.1" 200 1492 0.124 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 0.118 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7/dhcp HTTP/1.1" 200 409 0.136 "GET /midolmanj-mgmt/bridges/3f0b7cb3-7209-46a5-b271-9a8afc2ec3c7 HTTP/1.1" 200 744 6.82 (total)