search

kubicorn / kubicorn

Simple, cloud native infrastructure for Kubernetes.
http://kubicorn.io
Apache License 2.0
1.69k stars 196 forks source link

Extremely high ping between Droplets when pinged over VPN #171

Closed xmudrii closed 7 years ago

xmudrii commented 7 years ago

If I login into two Droplets created using a default bootstrap script, take IP address and ping one Droplet form another, I'll get very high ping.

Here are details: VPN details (ipconfig) from the master Droplet:

tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet addr:10.7.30.67  P-t-P:10.7.30.67  Mask:255.255.0.0
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1400  Metric:1
          RX packets:723 errors:0 dropped:0 overruns:0 frame:0
          TX packets:723 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:60145 (60.1 KB)  TX bytes:62263 (62.2 KB)

tunl0     Link encap:IPIP Tunnel  HWaddr
          inet addr:192.168.220.0  Mask:255.255.255.255
          UP RUNNING NOARP  MTU:1440  Metric:1
          RX packets:4 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:336 (336.0 B)  TX bytes:336 (336.0 B)

VPN details from the node:

tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet addr:10.7.250.88  P-t-P:10.7.250.88  Mask:255.255.0.0
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1400  Metric:1
          RX packets:151 errors:0 dropped:0 overruns:0 frame:0
          TX packets:214 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:15851 (15.8 KB)  TX bytes:16907 (16.9 KB)

tunl0     Link encap:IPIP Tunnel  HWaddr
          inet addr:192.168.153.128  Mask:255.255.255.255
          UP RUNNING NOARP  MTU:1440  Metric:1
          RX packets:4 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:336 (336.0 B)  TX bytes:336 (336.0 B)

Ping Master->Node

PING 10.7.250.88 (10.7.250.88) 56(84) bytes of data.
64 bytes from 10.7.250.88: icmp_seq=1 ttl=64 time=92334 ms
64 bytes from 10.7.250.88: icmp_seq=2 ttl=64 time=91325 ms
64 bytes from 10.7.250.88: icmp_seq=3 ttl=64 time=90325 ms
64 bytes from 10.7.250.88: icmp_seq=4 ttl=64 time=89326 ms
64 bytes from 10.7.250.88: icmp_seq=5 ttl=64 time=88325 ms

Ping Node->Master

root@xmudrii-k8s2-node-0:~# ping 10.7.30.67
PING 10.7.30.67 (10.7.30.67) 56(84) bytes of data.
64 bytes from 10.7.30.67: icmp_seq=1 ttl=64 time=81339 ms
64 bytes from 10.7.30.67: icmp_seq=2 ttl=64 time=80346 ms
64 bytes from 10.7.30.67: icmp_seq=3 ttl=64 time=79346 ms
64 bytes from 10.7.30.67: icmp_seq=4 ttl=64 time=78347 ms
64 bytes from 10.7.30.67: icmp_seq=5 ttl=64 time=77347 ms

It's also not possible to SSH or it takes very long time to establish a connection.

kubectl works fast though. I'm not sure is this indeed or not (e.g. due to firewall, etc), so I'm reporting it here.

krisnova commented 7 years ago

So I think this is a case of meshbird being a quick and dirty solution to the VPN mesh we are trying to create..

I vote we check out the following two projects:

and write bootstrap scripts for both of them for DO. This gives us an ability to do side-by-side testing and figure out which VPN mesh we want to use.

I think the reason it's so slow is because meshbird is building the tunnel over the public IP address.. which completely defeats the purpose of running digital ocean private networking..

krisnova commented 7 years ago

Can we please:

  1. Open up an issue for the OpenVPN bootstrap script
  2. Open up an issue for the Weave mesh bootstrap script
  3. Point all the issues together by dropping links in comments
xmudrii commented 7 years ago

I agree with you. Originally, I was trying to setup the following https://www.digitalocean.com/community/tutorials/how-to-install-tinc-and-set-up-a-basic-vpn-on-ubuntu-14-04

But I was stuck on two things: 1) Set subnet depending on the server https://www.digitalocean.com/community/tutorials/how-to-install-tinc-and-set-up-a-basic-vpn-on-ubuntu-14-04#tinc-configuration (as you see - every Droplet has it's own subnet) 2) Copying keys between Droplets

xmudrii commented 7 years ago

@kris-nova created the issues #177 and #178.