weave proxy is too slow

weaveworks / weave

Simple, resilient multi-host containers networking and more.

https://www.weave.works

Apache License 2.0

6.62k stars 670 forks source link

weave proxy is too slow #1439

Closed kesavkolla closed 9 years ago

kesavkolla commented 9 years ago

I just upgraded to latest weave release v1.1.0. I'm running weave on ubuntu 15.04 server.

I launched weave using weave launch command. Here is the weave status

       Version: v1.1.0

       Service: router
      Protocol: weave 1..2
          Name: 12:ad:f3:59:9d:b2(xxxxx)
    Encryption: disabled
 PeerDiscovery: enabled
       Targets: 0
   Connections: 0
         Peers: 1

       Service: ipam
     Consensus: achieved
         Range: [10.32.0.0-10.48.0.0)
 DefaultSubnet: 10.32.0.0/12

       Service: dns
        Domain: weave.local.
           TTL: 1
       Entries: 4

       Service: proxy
       Address: unix:///var/run/weave.sock

I set my environment with eval $(weave env)

Now whenever I execute docker exec -it <containername> bash -l it takes long time to return. It takes almost a minute to get the exec command to work.

Earlier version never had this problem hope it's not a regression.

-Kesav

rade commented 9 years ago

Needless to say this works just fine for me. I am running ubuntu 14.04.1 and docker 1.7.1. What docker version are you running? Also, check the timings in docker logs weaveproxy and how they correlate with the time you execute the command. When I do the exec I instantly get...

INFO: 2015/09/16 05:53:15.136078 POST /v1.19/containers/foo/exec
INFO: 2015/09/16 05:53:15.137783 Exec in container 90cdeb1a6d7a1f58d99976d7a6b7f50cf39afa50bcdf7e4a06c8265ca6331fd2 with WEAVE_CIDR ""
INFO: 2015/09/16 05:53:15.139347 POST /v1.19/exec/312054c5c64964c61a4eb17ba34857c2eba5e148878e6054e2bef9cdfb92c6b6/start
INFO: 2015/09/16 05:53:15.140501 POST /v1.19/exec/312054c5c64964c61a4eb17ba34857c2eba5e148878e6054e2bef9cdfb92c6b6/resize?h=24&w=80

bboreham commented 9 years ago

Also check what Docker file system driver you are using (with docker info) - this can make a massive difference to the speed of creating containers. Though I've never seen it take a minute.

See http://blog.weave.works/2015/07/14/i-quadrupled-the-speed-of-our-container-test-rig-in-a-day/

kesavkolla commented 9 years ago

Here is the output of my docker info

Containers: 8
Images: 167
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 183
 Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.19.0-28-generic
Operating System: Ubuntu 15.04
CPUs: 8
Total Memory: 31.38 GiB
ID: V5Y6:3A6H:D2RH:CCXD:7GQZ:2PIL:D63T:IP2A:LXJQ:3NMG:M5KC:LYDX
WARNING: No swap limit support

I'm already using aufs.

It's the exec command which is taking lot of time

rade commented 9 years ago

@kesavkolla see my earlier comment for additional data that might help track this down.

rade commented 9 years ago

@kesavkolla if you have some more data as per https://github.com/weaveworks/weave/issues/1439#issuecomment-140634874 please post it. Until then I am closing this.