drop vrrp advertisement packets if not coming from my peer ip address

subhajit-cdot commented 3 months ago

Hi, I have a simple requirement of dropping vrrp adv multicast packets if not coming from my peer ip address whose ip address I will know beforehand. I am not able to make it work using iptables/ebtables, it seems keepalived opens RAW socket and packets are going directly to keepalived daemon bypassing iptables/ebtables. Here are below rules I have configured and it's counter increasing, still keepalived receiving packets.

So, for testing I have used 2 system connected and running VRRP .. active/backup selection done. Now I tried to drop packets from my peer, and it is just to check if keepalived getting packets or not. The below rule counters are increasing, Rule: iptables -t mangle -A PREROUTING -s -d 224.0.0.18 -j DROP

Counter: iptables -t mangle -nvL Chain PREROUTING (policy ACCEPT 201K packets, 31M bytes) pkts bytes target prot opt in out source destination
5316 213K DROP all -- 192.168.122.4 224.0.0.18

but keepalived statistics showing packets received in daemon .. kill -USR2 $(cat /var/run/keepalived.pid) cat /tmp/keepalived.stats

Similarly, I tried with ebtables .. to drop packets based on mac .. but failed

Rule: ebtables-legacy -t broute --insert BROUTING -i ens3 --dst 01:00:5e:00:00:12 -j redirect --redirect-target DROP.

$netstat -anp | grep kee raw 0 0 0.0.0.0:112 0.0.0.0: 7 5320/keepalived
raw 0 0 0.0.0.0:112 0.0.0.0: 7 5320/keepalived
raw 0 0 0.0.0.0:112 0.0.0.0:* 7 5320/keepalived
unix 3 [ ] STREAM CONNECTED 69898 5320/keepalived
unix 2 [ ] DGRAM 73108 5319/keepalived

$ip maddr show 1: lo inet 224.0.0.1 inet6 ff02::1 inet6 ff01::1 2: ens3 link 33:33:00:00:00:01 link 01:00:5e:00:00:01 link 33:33:ff:9a:76:19 link 01:00:5e:00:00:12 inet 224.0.0.1 inet6 ff02::1:ff9a:7619 inet6 ff02::1 inet6 ff01::1

It seems keepalived opened raw socket is picking packets before packets are hitting these tables.

What could be the solution to address the above requirement?

Thanks Subhajit

pqarmitage commented 3 months ago

I have added a rule iptables -I INPUT -p 112 -s 192.168.122.4 -d 224.0.0.18 -j DROP and it successfully drops received VRRP adverts from 192.168.122.4.

So I suppose what you want is a rule: iptables -I INPUT -p 112 ! -s 192.168.122.4 -d 224.0.0.18 -j DROP or for nft: ip saddr != 192.168.122.4 ip daddr 224.0.0.18 ip protocol vrrp drop

subhajit-cdot commented 2 months ago

In my system packets are not coming to INPUT chain for vrrp protocol or multicast address .. packet counters are not increasing .. can it be related to vrrp version or configuration ? I can see packets in MANGLE table PREROUTING chain ... DROP counter is also increasing .. but keepalived is getting packets. So I thought may be keepalived is picking packets from RAW socket bypassing tcp/ip kernel stack here.

subhajit-cdot commented 2 months ago

So, I have tested with a vanilla ubuntu vm .. without running keepalived . I am still not getting packets in INPUT chain for vrrp packets having multicast ip .. below are the details

$ tcpdump -i enp1s0 vrrp -nex -vv tcpdump: listening on enp1s0, link-type EN10MB (Ethernet), snapshot length 262144 bytes 09:41:19.397539 52:54:00:9a:76:19 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 60: (tos 0xc0, ttl 255, id 55350, offset 0, flags [none], proto VRRP (112), length 40) 192.168.122.5 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 100, authtype none, intvl 1s, length 20, addrs: 192.168.122.3 0x0000: 45c0 0028 d836 0000 ff70 c7ae c0a8 7a05 0x0010: e000 0012 2101 6401 0001 4050 c0a8 7a03 0x0020: 0000 0000 0000 0000 0000 0000 0000 09:41:20.353285 52:54:00:9a:76:19 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 60: (tos 0xc0, ttl 255, id 55351, offset 0, flags [none], proto VRRP (112), length 40) 192.168.122.5 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 100, authtype none, intvl 1s, length 20, addrs: 192.168.122.3 0x0000: 45c0 0028 d837 0000 ff70 c7ad c0a8 7a05 0x0010: e000 0012 2101 6401 0001 4050 c0a8 7a03 0x0020: 0000 0000 0000 0000 0000 0000 0000 ^C 2 packets captured 2 packets received by filter 0 packets dropped by kernel $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 22.04.3 LTS Release: 22.04 Codename: jammy $ uname -a Linux enodeb-ubuntu 6.8.0-40-generic #40~22.04.3-Ubuntu SMP PREEMPT_DYNAMIC Tue Jul 30 17:30:19 UTC 2 x86_64 x86_64 x86_64 GNU/Linux $ iptables -nvL Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination

$ iptables -I INPUT -p 112 -s 192.168.122.5 -d 224.0.0.18 -j DROP

$ iptables -nvL Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination
0 0 DROP 112 -- 192.168.122.5 224.0.0.18

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination
$ iptables -nvL Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination
0 0 DROP 112 -- 192.168.122.5 224.0.0.18

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination

$ tcpdump -i enp1s0 vrrp -nex -vv tcpdump: listening on enp1s0, link-type EN10MB (Ethernet), snapshot length 262144 bytes 09:42:34.519128 52:54:00:9a:76:19 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 60: (tos 0xc0, ttl 255, id 55425, offset 0, flags [none], proto VRRP (112), length 40) 192.168.122.5 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 100, authtype none, intvl 1s, length 20, addrs: 192.168.122.3 0x0000: 45c0 0028 d881 0000 ff70 c763 c0a8 7a05 0x0010: e000 0012 2101 6401 0001 4050 c0a8 7a03 0x0020: 0000 0000 0000 0000 0000 0000 0000 09:42:35.468167 52:54:00:9a:76:19 > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 60: (tos 0xc0, ttl 255, id 55426, offset 0, flags [none], proto VRRP (112), length 40) 192.168.122.5 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 100, authtype none, intvl 1s, length 20, addrs: 192.168.122.3 0x0000: 45c0 0028 d882 0000 ff70 c762 c0a8 7a05 0x0010: e000 0012 2101 6401 0001 4050 c0a8 7a03 0x0020: 0000 0000 0000 0000 0000 0000 0000

pqarmitage commented 2 months ago

If you are not running keepalived in your VM, then presumably the system has not joined the multicast group. While the VRRP adverts are being seen by tcpdump (which presumably will have put the network adapter in promiscuous mode) I would expect the kernel to drop the multicast packets fairly early on since nothing is listening to that multicast group.

I think you need to be running keepalived for the test to be meaningful.

subhajit-cdot commented 2 months ago

Yes thanks for pointing out .. I have tested and its working fine in ubuntu22.04 VM with keepalived v2.2.4 .. but not in my debian buster VM having keepalived v1.2.24 Differences $ netstat -anp | grep kee raw 0 0 0.0.0.0:112 0.0.0.0:* 7 129539/keepalived
*raw 0 0 224.0.0.18:112 0.0.0.0: 7 129539/keepalived**

$ ip maddr show 1: lo inet 224.0.0.251 inet 224.0.0.1 inet6 ff02::fb inet6 ff02::1 inet6 ff01::1 2: enp1s0 link 01:00:5e:00:00:01 link 33:33:00:00:00:01 link 33:33:ff:4c:da:ef link 33:33:00:00:00:fb link 01:00:5e:00:00:fb link 01:00:5e:00:00:16 link 01:00:5e:00:00:fc link 01:00:5e:7f:ff:fa link 01:00:5e:00:00:12 inet 224.0.0.18 inet 239.255.255.250 inet 224.0.0.252 inet 224.0.0.22 inet 224.0.0.251 users 2 inet 224.0.0.1 users 2 inet6 ff02::fb inet6 ff02::1:ff4c:daef inet6 ff02::1 inet6 ff01::1

$ keepalived -v Keepalived v2.2.4 (08/21,2021)

Where I am getting issue ... The differences are

ip mroute show not showing 224.0.0.18 and RAW socket is opened at any ip address

$ systemctl status keepalived.service ● keepalived.service - Keepalived Daemon (LVS and VRRP) Loaded: loaded (/lib/systemd/system/keepalived.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2024-08-28 18:18:19 IST; 20h ago Main PID: 7187 (keepalived) Tasks: 5 (limit: 4915) CPU: 16.874s CGroup: /system.slice/keepalived.service ├─7187 /usr/sbin/keepalived --snmp --log-facility=7 --log-detail --dump-conf -x --use-file /etc/keepalived/keepalived.conf --release-vips --snmp-agent-socket tcp:localhost:705:1 └─7190 /usr/sbin/keepalived --snmp --log-facility=7 --log-detail --dump-conf -x --use-file /etc/keepalived/keepalived.conf --release-vips --snmp-agent-socket tcp:localhost:705:1

Aug 28 18:36:20 node Keepalived_vrrp[7190]: IFLA_CARRIER present for interface ens8 and the value is 1 Aug 28 18:50:39 node Keepalived_vrrp[7190]: IFLA_CARRIER present for interface vrfcfwSecSeg and the value is 1 Aug 28 18:50:39 node Keepalived_vrrp[7190]: IFLA_CARRIER present for interface vrfcfwSecSeg and the value is 1 Aug 28 18:50:39 node Keepalived_vrrp[7190]: IFLA_CARRIER present for interface vrfcfwSecSeg and the value is 1 Aug 28 18:50:39 node Keepalived_vrrp[7190]: Checking VIPs for ens3 (ens3) Aug 28 18:50:39 node Keepalived_vrrp[7190]: ens3-1 is address owner? no Aug 28 18:50:39 node Keepalived_vrrp[7190]: Checking VIPs for dp0s8.212 (dp0s8.212) Aug 28 18:50:39 node Keepalived_vrrp[7190]: dp0s8.212-4 is address owner? no Aug 28 18:50:39 node Keepalived_vrrp[7190]: Checking VIPs for dp0s10.213 (dp0s10.213) Aug 28 18:50:39 node Keepalived_vrrp[7190]: dp0s10.213-5 is address owner? no

$ sudo ip mroute show

$ sudo ip maddr show 1: lo inet 224.0.0.1 inet6 ff02::1 inet6 ff01::1 2: ens3 link 33:33:00:00:00:01 link 01:00:5e:00:00:01 link 33:33:ff:3b:90:0e link 01:00:5e:00:00:12 inet 224.0.0.1 inet6 ff02::1:ff3b:900e inet6 ff02::1 inet6 ff01::1

$ sudo netstat -anp | grep kee raw 0 0 0.0.0.0:112 0.0.0.0: 7 7190/keepalived
raw 0 0 0.0.0.0:112 0.0.0.0: 7 7190/keepalived
raw 0 0 0.0.0.0:112 0.0.0.0:* 7 7190/keepalived
unix 3 [ ] STREAM CONNECTED 73434 7190/keepalived
unix 2 [ ] DGRAM 69571 7187/keepalived

$ keepalived -v Keepalived v1.2.24 (08/03,2024)

$ iptables -nvL Chain INPUT (policy ACCEPT 446 packets, 118K bytes) pkts bytes target prot opt in out source destination
0 0 DROP 112 -- 192.168.122.5 224.0.0.18

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 429 packets, 46032 bytes) pkts bytes target prot opt in out source destination

$tcpdump -i ens3 vrrp -n tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on ens3, link-type EN10MB (Ethernet), capture size 262144 bytes 14:52:29.319475 IP 192.168.122.5 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 100, authtype none, intvl 1s, length 20 14:52:30.146265 IP 192.168.122.5 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 100, authtype none, intvl 1s, length 20

all packets are received by keepalived and no iptable hit is seen. If I put the same rule in MANGLE table PREROUTING chain .. the counter increases for vrrp packet .. still packets are getting received by vrrp.

Thanks Subhajit

pqarmitage commented 2 months ago

keepalived v1.2.24 did not bind the receive socket to the multicast address whereas v2.2.4 does (it was added in version 2.1.0).

I suggest you update to a recent version of keepalived on your Debian Buster system and see if that resolves your issue. Of course, it might be that Linux 4.19 (Buster) just behaves differently from Linux 5.15 (Ubuntu 22.04).

I note that Buster is now EOL, and so if you really want to continue with Buster, it is going to be down to you to resolve whatever issues you come across.

acassen / keepalived

drop vrrp advertisement packets if not coming from my peer ip address #2467