Using weave when one of peers is behind NAT

[karser]

What you expected to happen?

So I have 2 servers, let's call them A1 (behind NAT) and A2 (has publicly available IP address). My goal is to achieve 100% peers compatibility as if the server A1 had public IP.

How to reproduce it?

I setup a script on A1 which forwards TCP port though ssh connection

A1# ssh -R 16783:127.0.0.1:6783 A2

So that I can reach A1 from A2 now:

A2# telnet 127.0.0.1 16783
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
weave^]

Great, let's add A1 server to the A2 peers:

cat /etc/sysconfig/weave
PEERS="127.0.0.1:16783"

Let's see if A2 sees A1 as a peer:

A2# weave status peers
22:bb:2e:ae:0b:62(A2)
   -> 127.0.0.1:16783       3e:f5:a1:0d:22:4a(A1)        pending
3e:f5:a1:0d:22:4a(A1)
   <- 127.0.0.1:51520       22:bb:2e:ae:0b:62(A2)               pending

A2# weave status connections
-> 127.0.0.1:16783       pending     encrypted   fastdp 3e:f5:a1:0d:22:4a(A1) encrypted=truemtu=1376

So far it seems to be working. Let's finally test pings:

A1# docker run -ti --net=weave -e WEAVE_CIDR=192.168.0.1/8 --ip=192.168.0.1 -h a1.weave.local $(weave dns-args) weaveworks/ubuntu
A2# docker run -ti --net=weave -e WEAVE_CIDR=192.168.0.2/8 --ip=192.168.0.2 -h a2.weave.local $(weave dns-args) weaveworks/ubuntu

A2# ping a1.weave.local
PING a1.weave.local (192.168.0.1) 56(84) bytes of data.
From a2.weave.local (192.168.0.2) icmp_seq=1 Destination Host Unreachable
From a2.weave.local (192.168.0.2) icmp_seq=2 Destination Host Unreachable
From a2.weave.local (192.168.0.2) icmp_seq=3 Destination Host Unreachable
^C

#vice versa
A1# ping a2.weave.local
PING a2.weave.local (192.168.0.1) 56(84) bytes of data.
From a1.weave.local (192.168.0.2) icmp_seq=1 Destination Host Unreachable
From a1.weave.local (192.168.0.2) icmp_seq=2 Destination Host Unreachable
^C

Here are the questions I have:

1. What other ports I need to forward? 6783, 6784 for TCP and UDP?

Weave uses ports 6783 TCP / 6784 UDP for fastdp and 6783 TCP / 6783 UDP for sleeve

I'm going to forward UDP/6783 using this approach http://zarb.org/~gc/html/udp-in-ssh-tunneling.html but the question 2 stops me.

2. I was able to specify a single port in PEERS="127.0.0.1:16783". How can I specify the other forwarded ports?

3. I understand that A2->A1 ping requires UDP traffic forwarding. But why A1->A2 ping doesn't work? A2 server has public IP and ports are not restricted.

Please, answer my questions or push me to the right direction.

Versions:

$ weave version weave script 2.6.0

$ docker version 18.09.3

$ uname -a Linux 4.4.127-mainline-rev1 #1 SMP Sun Apr 8 10:38:32 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

## Logs:
A2 logs: https://gist.github.com/karser/f323ffb3ac3d875b62c571a46ec5206e

[murali-reddy]

Please see https://www.weave.works/docs/net/latest/faq/ for the details on the ports to be opened. If you are using encryption then ESP traffic should be permitted as well.

Have you tried without port forwarding by directly launching weave on A1 with public IP of A2 as peer? Please see https://www.weave.works/docs/net/latest/overview/#nat-traversal Weave can work though NAT, though it will will use only sleeve.

[karser]

Thank you for your response @murali-reddy

Have you tried without port forwarding by directly launching weave on A1 with public IP of A2 as peer?

That's exactly how I launched it. The A2 server has public IP, so it's in direct visibility.

A2# cat /etc/sysconfig/weave
PEERS="127.0.0.1:16783"

A1# cat /etc/sysconfig/weave
PEERS="<A2 PUBLIC IP>"

The overview of net-traversal is great, but how can I use it in practice? What IP should I specify on A2 server to reach A1 without port forwarding? Now it's 127.0.0.1:16783?

[murali-reddy]

What IP should I specify on A2 server to reach A1 without port forwarding? Now it's 127.0.0.1:16783?

You can keep the connection in one direction only. Meaning just launch weave on A2 without any peers. Pods across A1, A2 can communicate in sleeve mode