search

troglobit / pimd

PIM-SM/SSM multicast routing for UNIX and Linux
http://troglobit.com/projects/pimd/
BSD 3-Clause "New" or "Revised" License
194 stars 86 forks source link

Multicast via OpenVPN tunnel #77

Open jbysewski opened 8 years ago

jbysewski commented 8 years ago

Hi troglobit,

I have the following situation where I'm attempting to get multicast traffic from Client1 to Client2 (or any other client in their respective subnets).

+--------------------+                   +-------------------------+
|                    |                   |                         |
|      Client1       |                   |         Router1         |
| eth0: 192.168.0.99 |        LAN        | br0(eth0): 192.168.0.1  |
|                    |------------------>| wwan0:         <GSM-IP> |
|                    |                   | OpenVPN(tap0): 10.8.0.2 |
|                    |                   |                         |
+--------------------+                   +-------------------------+
                                                     |
                                                     | Internet
                                                     |
                                                     v
                                         +--------------------------+
                                         |                          |
                                         |         Server           |
                                         |                          |
                                         | eth0: <public ip>        |
                                         | OpenVPN(tap0): 10.8.0.1  |
                                         |                          |
                                         +--------------------------+
                                                     |
                                                     | Internet
                                                     |
                                                     v
+--------------------+                   +-------------------------+
|                    |                   |                         |
|      Client2       |                   |        Router2          |
| eth0: 192.168.1.99 |        LAN        | br0(eth0): 192.168.1.1  |
|                    |<------------------| wwan0:         <GSM-IP> |
|                    |                   | OpenVPN(tap0): 10.8.0.3 |
|                    |                   |                         |
+--------------------+                   +-------------------------+

I run pimd in default configuration on Router1, Server and Router2 but it seems I can't get multicast traffic from Client1 beyond Router1.

Preparation

C1> ip route add 224.0.0.0/4 dev eth0

Also tried the following on R1 and S:

echo 1 > /proc/sys/net/ipv4/ip_forward
echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter

Test 1 - ping

C1> ping -I eth0 -t 5 225.1.2.3

R1> tcpdump -n -i br0 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0, link-type EN10MB (Ethernet), capture size 65535 bytes
10:26:25.487971 IP 192.168.0.99 > 225.1.2.3: ICMP echo request, id 5673, seq 1, length 64
10:26:26.497753 IP 192.168.0.99 > 225.1.2.3: ICMP echo request, id 5673, seq 2, length 64
10:26:27.507753 IP 192.168.0.99 > 225.1.2.3: ICMP echo request, id 5673, seq 3, length 64

R1> tcpdump -n -i tap0 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap0, link-type EN10MB (Ethernet), capture size 65535 bytes
<nothing>

S> tcpdump -n -i tap0 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap0, link-type EN10MB (Ethernet), capture size 65535 bytes
<nothing>

Test 2 - mcjoin

C1> mcjoin -s -d -t 5
Sending packet on signal 14, msg: Sender PID 5726, MC group 225.1.2.3 ... count: 1
Sending packet on signal 14, msg: Sender PID 5726, MC group 225.1.2.3 ... count: 2
Sending packet on signal 14, msg: Sender PID 5726, MC group 225.1.2.3 ... count: 3

R1> mcjoin -i br0 -d
Added iface br0, idx 8
GROUP 0xe1010203 (225.1.2.3)
joined group 225.1.2.3 on br0 ...
Count 0, Our PID 18132 Got PID: 0, dst 225.1.2.3 group 225.1.2.3 msg: Sender PID 5726, MC group 225.1.2.3 ... count: 1
.
Count 1, Our PID 18132 Got PID: 0, dst 225.1.2.3 group 225.1.2.3 msg: Sender PID 5726, MC group 225.1.2.3 ... count: 2
.
Count 2, Our PID 18132 Got PID: 0, dst 225.1.2.3 group 225.1.2.3 msg: Sender PID 5726, MC group 225.1.2.3 ... count: 3

R1> mcjoin -i tap0 -d
Added iface br0, idx 8
GROUP 0xe1010203 (225.1.2.3)
joined group 225.1.2.3 on br0 ...
Count 0, Our PID 18132 Got PID: 0, dst 225.1.2.3 group 225.1.2.3 msg: Sender PID 5726, MC group 225.1.2.3 ... count: 10
.
Count 1, Our PID 18132 Got PID: 0, dst 225.1.2.3 group 225.1.2.3 msg: Sender PID 5726, MC group 225.1.2.3 ... count: 11
.
Count 2, Our PID 18132 Got PID: 0, dst 225.1.2.3 group 225.1.2.3 msg: Sender PID 5726, MC group 225.1.2.3 ... count: 12

S> mcjoin -i tap0 -d
Added iface tap0, idx 7
GROUP 0xe1010203 (225.1.2.3)
joined group 225.1.2.3 on tap0 ...
<nothing>

But if we start mcjoin as a sender on R1 the multicast traffic from C1 starts to come through - but only while R1 is also sending.

R1> mcjoin -s -d
Sending packet on signal 14, msg: Sender PID 19464, MC group 225.1.2.3 ... count: 1
Sending packet on signal 14, msg: Sender PID 19464, MC group 225.1.2.3 ... count: 2
Sending packet on signal 14, msg: Sender PID 19464, MC group 225.1.2.3 ... count: 3

S> mcjoin -i tap0 -d
Added iface tap0, idx 7
GROUP 0xe1010203 (225.1.2.3)
joined group 225.1.2.3 on tap0 ...
Count 0, Our PID 11466 Got PID: 0, dst 225.1.2.3 group 225.1.2.3 msg: Sender PID 19464, MC group 225.1.2.3 ... count: 1
.
Count 1, Our PID 11466 Got PID: 0, dst 225.1.2.3 group 225.1.2.3 msg: Sender PID 5726, MC group 225.1.2.3 ... count: 30
.
Count 2, Our PID 11466 Got PID: 0, dst 225.1.2.3 group 225.1.2.3 msg: Sender PID 19464, MC group 225.1.2.3 ... count: 2
.
Count 3, Our PID 11466 Got PID: 0, dst 225.1.2.3 group 225.1.2.3 msg: Sender PID 5726, MC group 225.1.2.3 ... count: 31
.
Count 4, Our PID 11466 Got PID: 0, dst 225.1.2.3 group 225.1.2.3 msg: Sender PID 19464, MC group 225.1.2.3 ... count: 3
.
Count 5, Our PID 11466 Got PID: 0, dst 225.1.2.3 group 225.1.2.3 msg: Sender PID 5726, MC group 225.1.2.3 ... count: 32
.

I could do multicast iperf from R1 -> S or from C1 -> R1 but not get it from C1 -> S.

Additional info

R1> ifconfig
br0       Link encap:Ethernet  HWaddr 00:60:64:DD:A1:AA
          inet addr:192.168.0.1  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::260:64ff:fedd:a1aa/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9427 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4561 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:5093188 (4.8 MiB)  TX bytes:1788183 (1.7 MiB)

eth0      Link encap:Ethernet  HWaddr 00:60:64:DD:A1:AA
          inet6 addr: fe80::260:64ff:fedd:a1aa/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9429 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4568 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:5263228 (5.0 MiB)  TX bytes:1788717 (1.7 MiB)

pimreg    Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          UP RUNNING NOARP  MTU:1472  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

tap0      Link encap:Ethernet  HWaddr 4A:82:33:63:50:57
          inet addr:10.8.0.2  Bcast:10.8.0.255  Mask:255.255.255.0
          inet6 addr: fe80::4882:33ff:fe63:5057/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:334 errors:0 dropped:0 overruns:0 frame:0
          TX packets:402 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:19716 (19.2 KiB)  TX bytes:43458 (42.4 KiB)

wwan0     Link encap:Ethernet  HWaddr 22:06:89:29:9B:47
          inet addr:<wwan_ip>  Bcast:<wwan_broadcast>  Mask:255.255.255.252
          inet6 addr: fe80::2006:89ff:fe29:9b47/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1460  Metric:1
          RX packets:1661 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4834 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:189841 (185.3 KiB)  TX bytes:721152 (704.2 KiB)

S> ifconfig
ens3      Link encap:Ethernet  HWaddr 06:6a:70:00:00:31
          inet addr:<public_ip>  Bcast:<public_broadcast>  Mask:255.255.252.0
          inet6 addr: <...> Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:119119 errors:0 dropped:0 overruns:0 frame:0
          TX packets:72255 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:23099591 (23.0 MB)  TX bytes:11702271 (11.7 MB)

pimreg    Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          UP RUNNING NOARP  MTU:1472  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

tap0      Link encap:Ethernet  HWaddr f2:e7:fc:a8:58:00
          inet addr:10.8.0.1  Bcast:10.8.0.255  Mask:255.255.255.0
          inet6 addr: fe80::f0e7:fcff:fea8:5800/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3756 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1098 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:3427986 (3.4 MB)  TX bytes:62928 (62.9 KB)

R1> pimd -r
Virtual Interface Table ======================================================
Vif  Local Address    Subnet              Thresh  Flags      Neighbors
---  ---------------  ------------------  ------  ---------  -----------------
  0  192.168.0.1      192.168                  1  PIM        192.168.0.99
  1  <wwan_ip>        <wwan_subnet>/30         1  DR NO-NBR
  2  10.8.0.2         10.8/24                  1  DR PIM     10.8.0.1
  3  192.168.0.1      register_vif0            1

 Vif  SSM Group        Sources

Multicast Routing Table ======================================================
----------------------------------- (S,G) ------------------------------------
Source           Group            RP Address       Flags
---------------  ---------------  ---------------  ---------------------------
10.8.0.2         225.1.2.3        192.168.0.99     SPT CACHE SG
Joined   oifs: ...j
Pruned   oifs: ....
Leaves   oifs: ....
Asserted oifs: ....
Outgoing oifs: ...o
Incoming     : ..I.

TIMERS:  Entry    JP    RS  Assert VIFS:  0  1  2  3
            45    10     0       0        0  0  0  0
Source           Group            RP Address       Flags
---------------  ---------------  ---------------  ---------------------------
192.168.0.99     225.1.2.3        192.168.0.99     SPT SG
Joined   oifs: ....
Pruned   oifs: ....
Leaves   oifs: ....
Asserted oifs: ....
Outgoing oifs: ....
Incoming     : I...

TIMERS:  Entry    JP    RS  Assert VIFS:  0  1  2  3
           205    60     0       0        0  0  0  0
--------------------------------- (*,*,G) ------------------------------------
Number of Groups: 1
Number of Cache MIRRORs: 1
------------------------------------------------------------------------------

S> pimd -r
Virtual Interface Table ======================================================
Vif  Local Address    Subnet              Thresh  Flags      Neighbors
---  ---------------  ------------------  ------  ---------  -----------------
  0  <public_ip>      <public_subnet>/22       1  DR NO-NBR
  1  10.8.0.1         10.8/24                  1  PIM        10.8.0.2
  2  <public_ip>      register_vif0            1

 Vif  SSM Group        Sources

Multicast Routing Table ======================================================
----------------------------------- (S,G) ------------------------------------
Source           Group            RP Address       Flags
---------------  ---------------  ---------------  ---------------------------
10.8.0.2         225.1.2.3        <public_ip>      SG
Joined   oifs: ...
Pruned   oifs: ...
Leaves   oifs: ...
Asserted oifs: ...
Outgoing oifs: ...
Incoming     : .I.

TIMERS:  Entry    JP    RS  Assert VIFS:  0  1  2
            40    15     0       0        0  0  0
--------------------------------- (*,*,G) ------------------------------------
Number of Groups: 1
Number of Cache MIRRORs: 0
------------------------------------------------------------------------------

I have tried the same configuration with OpenVPN configured to tun-networking. Any help would be much appreciated.

troglobit commented 8 years ago

Hi @jbysewski, awesome report, and I'm happy to see someone using my little mcjoin tool for debugging! :smiley:

Now, PIM, unlike DVMRP (mrouted), does not flood multicast to all routers. So C2 needs to run mcjoin (i.e. an IGMP join for the requested multicast) which will receive R2 and realize it needs to send a PIM Join upstream, towards the elected rendez-vous point for that multicast group.

For PIM to work, there needs to be a unicast routing table setup first. This can be done with RIP, OSPF, or static unicast routes. That's where the protocol independent in PIM comes from. Easily tested by checking that C1 can ping C2 and vice versa.

Very observant of you to notice you need to change the TTL, that's usually FAQ n:o 1.

In the pimd -r output on R1 there is a strange listing for a PIM neighbor in 192.168.0.99. Are you sure you are not running pimd on C1?

jbysewski commented 8 years ago

Are you sure you are not running pimd on C1?

It seems I had a pimd running on C1.

I redid the experiment with a bit different results, but I don't understand why it's different now (but I restarted the GSM routers). The multicast ping got to R2 - but not to C2 as desired. I also tried a higher ttl of 10 without success.

C1> ping -I eth0 -t 5 225.1.2.3

R2> tcpdump -n -i tap0 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap0, link-type EN10MB (Ethernet), capture size 65535 bytes
21:49:02.788180 IP 192.168.0.99 > 225.1.2.3: ICMP echo request, id 5320, seq 211, length 64
21:49:03.818336 IP 192.168.0.99 > 225.1.2.3: ICMP echo request, id 5320, seq 212, length 64
21:49:04.828742 IP 192.168.0.99 > 225.1.2.3: ICMP echo request, id 5320, seq 213, length 64

C2> tcpdump -n -i eth0 icmp
<nothing>

R1> pimd -r
Virtual Interface Table ======================================================
Vif  Local Address    Subnet              Thresh  Flags      Neighbors
---  ---------------  ------------------  ------  ---------  -----------------
  0  192.168.0.1      192.168                  1  DR NO-NBR
  1  xxx.xx.xxx.xx    xxx.xx.xxx.xx/29         1  DR NO-NBR
  2  10.8.0.2         10.8/24                  1  PIM        10.8.0.3
                                                             10.8.0.1
  3  192.168.0.1      register_vif0            1

 Vif  SSM Group        Sources

Multicast Routing Table ======================================================
----------------------------------- (*,G) ------------------------------------
Source           Group            RP Address       Flags
---------------  ---------------  ---------------  ---------------------------
INADDR_ANY       225.1.2.3        192.168.0.1      WC RP
Joined   oifs: ....
Pruned   oifs: ....
Leaves   oifs: ..l.
Asserted oifs: ....
Outgoing oifs: ..o.
Incoming     : ...I

TIMERS:  Entry    JP    RS  Assert VIFS:  0  1  2  3
             0    20     0       0        0  0  0  0
----------------------------------- (S,G) ------------------------------------
Source           Group            RP Address       Flags
---------------  ---------------  ---------------  ---------------------------
192.168.0.99     225.1.2.3        192.168.0.1      SPT CACHE SG
Joined   oifs: ....
Pruned   oifs: ....
Leaves   oifs: ..l.
Asserted oifs: ....
Outgoing oifs: ..o.
Incoming     : I...

TIMERS:  Entry    JP    RS  Assert VIFS:  0  1  2  3
           170    25     0       0        0  0  0  0
--------------------------------- (*,*,G) ------------------------------------
Number of Groups: 1
Number of Cache MIRRORs: 1
------------------------------------------------------------------------------

S> pimd -r
Virtual Interface Table ======================================================
Vif  Local Address    Subnet              Thresh  Flags      Neighbors
---  ---------------  ------------------  ------  ---------  -----------------
  0  xxx.xxx.xxx.xx   xxx.xxx.xxx/22           1  DR NO-NBR
  1  10.8.0.1         10.8/24                  1  PIM        10.8.0.3
                                                             10.8.0.2
  2  xxx.xxx.xxx.xx   register_vif0            1

 Vif  SSM Group        Sources

Multicast Routing Table ======================================================
--------------------------------- (*,*,G) ------------------------------------
Number of Groups: 0
Number of Cache MIRRORs: 0
------------------------------------------------------------------------------

R2> pimd -r
Virtual Interface Table ======================================================
Vif  Local Address    Subnet              Thresh  Flags      Neighbors
---  ---------------  ------------------  ------  ---------  -----------------
  0  192.168.1.1      192.168.1                1  DR NO-NBR
  1  10.8.0.3         10.8/24                  1  DR PIM     10.8.0.2
                                                             10.8.0.1
  2  192.168.1.1      register_vif0            1

 Vif  SSM Group        Sources

Multicast Routing Table ======================================================
--------------------------------- (*,*,G) ------------------------------------
Number of Groups: 0
Number of Cache MIRRORs: 0
------------------------------------------------------------------------------

I suppose that the three instances of pimd see each other as neighbors is a good thing. Your multicast tutorial has the following phrase

Could be your underlying routing protocol (RIP/OSPF) does not know the reverse path to the source. Make sure the sender’s network is listed in the routing table on the receiving sides routing table.

Could that be a hint? Currently I fail to understand what else to do.

troglobit commented 8 years ago

First thing to test, whatever you do, make unicast ping between C1 and C2 work. Without that, you're toast.

Second, start mcjoin on C2 so the nearest router understands it needs to request multicast.

troglobit commented 8 years ago

Did you give up, or get it working?

jbysewski commented 8 years ago

None of the above. :-) I'm very occupied with family stuff, beside other things my wife's pregnant with our second child and we've reached the due date. I'll work on this as soon as time permits and report back.

troglobit commented 8 years ago

I see! :smiley:

That's great news, take care of your family and we'll pick this one up when time permits again. Cheers!

Voronenko commented 5 years ago

Anyone can point to a guide - if I see multicasts from needed devices on each of the interfaces eth0 and tap0 of the same host, but can't forward one from tap0 to eth0.

What would be the correct steps?

troglobit commented 5 years ago

@Voronenko use tcpdump or Wireshark to verify the TTL in the IP header of the multicast traffic. It must be > 1 for the traffic to be forwarded from one interface to another. Then check the IP forwarding setting of the kernel, on each of the involved interfaces.

jp-t commented 5 years ago

@Voronenko maybe, also check that the MTU on both interfaces is large enough to convey the packets. It looks like tap0 is a tunnel, which usually shrinks the MTU. (though it seems that your data flows in the direction of smaller mtu to larger mtu)

Voronenko commented 5 years ago

@troglobit @jp-t Thank you all for a response. I suspect it is oftopic, but perhaps can be converted to some example.

So: I tried to affect TTL in a following manner - either increase TTL by 2 when goes to needed destination

iptables -t mangle -A PREROUTING -d 224.0.0.50 -j TTL --ttl-inc 2

as well as hardcore abnormal TTL 128 or 255

iptables -t mangle -A OUTPUT -d 224.0.0.50 -j TTL --ttl-set 128

So right now TTLs are quite high - but I see notifications only from local devices on each network respectively:

tcpdump -v -i tap0 dst host 224.0.0.50 and port 9898
tcpdump: listening on tap0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:40:39.840906 IP (tos 0x0, ttl 255, id 8450, offset 0, flags [none], proto UDP (17), length 163)
    192.168.1.10.4321 > 224.0.0.50.9898: UDP, length 135
tcpdump -v -i eth0 dst host 224.0.0.50 and port 9898
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:41:45.474687 IP (tos 0x0, ttl 255, id 47139, offset 0, flags [none], proto UDP (17), length 164)
    192.168.2.214.4321 > 224.0.0.50.9898: UDP, length 136

Re forwarding:

sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

Re "on each of the involved interfaces":

Do you mean kind of

    iptables -A FORWARD -i tap0 -o eth0 -j ACCEPT
    iptables -A FORWARD -i tap0 -o eth0 -m state --state ESTABLISHED,RELATED \
             -j ACCEPT

(note: not sure if masquerade is needed)

    iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

Should I tune pimd.conf from defaults?

troglobit commented 5 years ago

The 224.0.0.* is link-local only, and should never be routed. I'd recommend bridging networks instead if that is what you want. (This is a very common question actually, which never ends well for those attempting to route it. Also very probably why the TTL is 1)

Voronenko commented 5 years ago

For notes of everyone interested, who would came here.

ip link add br0 type bridge
ip link set tap0 master br0
ip link set eth0 master br0

POC - both multicasts on single interface


sudo tcpdump -i br0 dst host 224.0.0.50 and port 9898
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:09:51.823632 IP 192.168.1.10.4321 > 224.0.0.50.9898: UDP, length 135
21:09:55.045138 IP 192.168.2.214.4321 > 224.0.0.50.9898: UDP, length 136

Now , if eth0 is in the same network as br0, it most likely will get another address, thus you might want to tune to live box without double addresses.