Open aseaudi opened 2 years ago
I created a P2P Vxlan tunnel between 2 sonic switches using the Loopback interfaces as SRC and DST, but it didn't work.
I found out it was because Sonic by default creates iptables rule to DROP INPUT traffic for Loopbacks.
I added a specific allow rule to ACCEPT INPUT DST UDP port 4789, then vxlan traffic worked.
Why does Sonic add those DROP rules ?
config vxlan add vtep 172.16.66.1 172.16.66.2 config vxlan evpn_nvo add nvo vtep config vxlan map add vtep 10 10
config vxlan add vtep 172.16.66.2 172.16.66.1 config vxlan evpn_nvo add nvo vtep config vxlan map add vtep 10 10
root@sonic:~# show vxlan interface VTEP Information: VTEP Name : vtep, SIP : 172.16.66.1 NVO Name : nvo, VTEP : vtep Source interface : Loopback0 root@sonic:~# show vxlan tunnel vxlan tunnel name source ip destination ip tunnel map name tunnel map mapping(vni -> vlan) ------------------- ----------- ---------------- ----------------- --------------------------------- vtep 172.16.66.1 172.16.66.2 map_10_Vlan10 10 -> Vlan10 root@sonic:~# show vxlan vlan +--------+-------+ | VLAN | VNI | +========+=======+ | Vlan10 | 10 | +--------+-------+ Total count : 1 root@sonic:~# iptables-save # Generated by iptables-save v1.8.7 on Sun Jan 16 12:58:32 2022 *mangle :PREROUTING ACCEPT [0:0] :INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [0:0] :POSTROUTING ACCEPT [0:0] -A PREROUTING -d 172.16.66.2/32 -p tcp -m tcp --tcp-flags SYN SYN -j TCPMSS --set-mss 1460 -A PREROUTING -d 172.16.55.1/32 -p tcp -m tcp --tcp-flags SYN SYN -j TCPMSS --set-mss 1460 -A POSTROUTING -s 172.16.66.2/32 -p tcp -m tcp --tcp-flags SYN SYN -j TCPMSS --set-mss 1460 -A POSTROUTING -s 172.16.55.1/32 -p tcp -m tcp --tcp-flags SYN SYN -j TCPMSS --set-mss 1460 COMMIT # Completed on Sun Jan 16 12:58:32 2022 # Generated by iptables-save v1.8.7 on Sun Jan 16 12:58:32 2022 *filter :INPUT ACCEPT [107472:21601742] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [7131427:639555513] -A INPUT -s 127.0.0.1/32 -i lo -j ACCEPT -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT -A INPUT -p icmp -m icmp --icmp-type 0 -j ACCEPT -A INPUT -p icmp -m icmp --icmp-type 3 -j ACCEPT -A INPUT -p icmp -m icmp --icmp-type 11 -j ACCEPT -A INPUT -p udp -m udp --dport 67:68 -j ACCEPT -A INPUT -p udp -m udp --dport 546:547 -j ACCEPT -A INPUT -p tcp -m tcp --dport 179 -j ACCEPT -A INPUT -p tcp -m tcp --sport 179 -j ACCEPT -A INPUT -d 172.16.66.2/32 -j DROP <<<< DEFAULT DROP ALL LOOPBACK TRAFFIC ADDED BY SONIC -A INPUT -d 172.16.55.1/32 -j DROP -A INPUT -d 192.168.11.1/32 -j DROP -A INPUT -d 172.16.11.0/32 -j DROP -A INPUT -m ttl --ttl-lt 2 -j ACCEPT COMMIT # Completed on Sun Jan 16 12:58:32 2022 root@sonic:~#
The vxlan tunnel is created, however we cannot ping over the tunnel
In order to make the tunnel work, I have to add specific INPUT UDP DST PORT 4789 to overcome the default DROP rule created by SONIC for all Loopbacks
root@sonic:~# iptables -I INPUT 10 -p udp -m udp --dport 4789 -j ACCEPT
After that I can ping from host 1 to host 2 over the tunnel
I should have ping working, and I should not DROP all incoming traffic for the Loopback
show version
root@sonic:~# show version SONiC Software Version: SONiC.master.60337-dirty-20211220.183808 Distribution: Debian 11.2 Kernel: 5.10.0-8-2-amd64 Build commit: 0d327abe9 Build date: Mon Dec 20 18:46:04 UTC 2021 Built by: AzDevOps@sonic-build-workers-00101X Platform: x86_64-kvm_x86_64-r0 HwSKU: Force10-S6000 ASIC: vs ASIC Count: 1 Serial Number: N/A Model Number: N/A Hardware Revision: N/A Uptime: 13:08:01 up 1 day, 5:25, 1 user, load average: 0.16, 0.14, 0.18 Docker images: REPOSITORY TAG IMAGE ID SIZE docker-dhcp-relay latest a6a3100137c8 436MB docker-platform-monitor latest 34ab0c7b9bce 689MB docker-platform-monitor master.60337-dirty-20211220.183808 34ab0c7b9bce 689MB docker-sonic-mgmt-framework latest f38327843b43 578MB docker-sonic-mgmt-framework master.60337-dirty-20211220.183808 f38327843b43 578MB docker-teamd latest 823108733c9b 436MB docker-teamd master.60337-dirty-20211220.183808 823108733c9b 436MB docker-syncd-vs latest 723cb3fc6cc0 435MB docker-syncd-vs master.60337-dirty-20211220.183808 723cb3fc6cc0 435MB docker-snmp latest de43b8b8acef 465MB docker-snmp master.60337-dirty-20211220.183808 de43b8b8acef 465MB docker-sonic-telemetry latest 15965d93c8a5 511MB docker-sonic-telemetry master.60337-dirty-20211220.183808 15965d93c8a5 511MB docker-orchagent latest 05e9299feb1d 455MB docker-orchagent master.60337-dirty-20211220.183808 05e9299feb1d 455MB docker-sflow latest a46e3275c303 436MB docker-sflow master.60337-dirty-20211220.183808 a46e3275c303 436MB docker-router-advertiser latest 12cf802b8ea1 423MB docker-router-advertiser master.60337-dirty-20211220.183808 12cf802b8ea1 423MB docker-nat latest 937e9f11accc 438MB docker-nat master.60337-dirty-20211220.183808 937e9f11accc 438MB docker-lldp latest ab2869ad3fda 463MB docker-lldp master.60337-dirty-20211220.183808 ab2869ad3fda 463MB docker-macsec latest 4924107df9a4 439MB docker-macsec master.60337-dirty-20211220.183808 4924107df9a4 439MB docker-mux latest 55af47e1752b 475MB docker-mux master.60337-dirty-20211220.183808 55af47e1752b 475MB docker-fpm-frr latest bd2e3744a4b6 454MB docker-fpm-frr master.60337-dirty-20211220.183808 bd2e3744a4b6 454MB docker-gbsyncd-vs latest 38f368627c24 431MB docker-gbsyncd-vs master.60337-dirty-20211220.183808 38f368627c24 431MB docker-database latest 5b39fa071593 423MB docker-database master.60337-dirty-20211220.183808 5b39fa071593 423MB root@sonic:~#
show techsupport
(paste your output here or download and attach the file here )
NOTE: I made a small change in the CLI to enable to add the remote ip address when the Vxlan is created.
Note: From discussion on the mailing list, this happens on sonic-vs - not physical hardware. Likely because the VXLAN tunnel is emulated and hits the control plane ACLs.
Vxlan is not supported on sonic-vs.
Description
I created a P2P Vxlan tunnel between 2 sonic switches using the Loopback interfaces as SRC and DST, but it didn't work.
I found out it was because Sonic by default creates iptables rule to DROP INPUT traffic for Loopbacks.
I added a specific allow rule to ACCEPT INPUT DST UDP port 4789, then vxlan traffic worked.
Why does Sonic add those DROP rules ?
Steps to reproduce the issue:
config vxlan add vtep 172.16.66.1 172.16.66.2 config vxlan evpn_nvo add nvo vtep config vxlan map add vtep 10 10
config vxlan add vtep 172.16.66.2 172.16.66.1 config vxlan evpn_nvo add nvo vtep config vxlan map add vtep 10 10
Describe the results you received:
The vxlan tunnel is created, however we cannot ping over the tunnel
In order to make the tunnel work, I have to add specific INPUT UDP DST PORT 4789 to overcome the default DROP rule created by SONIC for all Loopbacks
root@sonic:~# iptables -I INPUT 10 -p udp -m udp --dport 4789 -j ACCEPT
After that I can ping from host 1 to host 2 over the tunnel
Describe the results you expected:
I should have ping working, and I should not DROP all incoming traffic for the Loopback
Output of
show version
:Output of
show techsupport
:Additional information you deem important (e.g. issue happens only occasionally):
NOTE: I made a small change in the CLI to enable to add the remote ip address when the Vxlan is created.