Multicast traffic problem

[alamintech]

ISSUE TYPE

• Improvement Request

COMPONENT NAME

UI

CLOUDSTACK VERSION

ACS 4.18.0.0

CONFIGURATION

OS / ENVIRONMENT

AlmaLinux8 MGMT+KVM

SUMMARY

Needed multicast traffic to send or receive from user vm to vm.

STEPS TO REPRODUCE

I am not sure how to reproduce

EXPECTED RESULTS

Esxi system have this option like IGMP As like below link https://prnt.sc/e_f5V9S3xgox

ACTUAL RESULTS

[levindecaro]

@alamintech , can you describe your network setup (eg: physical network, L2, VLAN, interface) and screen dump of your guest network / vpc etc. I couldn't see any problem using multicast in KVM environment in SAME guest network.

[alamintech]

Thanks for your reply, How can I test it from my existing environment? I think your are right I am wrong. Windows vm to vm or Linux vm to vm(How can test it? is it properly working)

My network like below:

Thank you!

[alamintech]

I am using this guide, https://salmannaqvi.com/2016/11/14/simple-multicast-testing-tool-for-windows/

[levindecaro]

@alamintech From your illustration, you are using a shared L2 networking for guest network right?, did you try the do multicasting on the host first, some switch default blocks multicast. U can try https://github.com/troglobit/mcjoin/ and tcpdump see if you can capture multicast on receiver ends.

[levindecaro]

@alamintech one thing u can try is turn off multicast snooping on bridge interface. Run the vm run on same KVM host first to ensure bridge interface is forwarding multicast.

eg: echo 0 > /sys/class/net/cloudbr0/bridge/multicast_snooping echo 2 > /sys/class/net/cloudbr0/bridge/multicast_router

[alamintech]

@alamintech one thing u can try is turn off multicast snooping on bridge interface. Run the vm run on same KVM host first to ensure bridge interface is forwarding multicast.

eg: echo 0 > /sys/class/net/cloudbr0/bridge/multicast_snooping echo 2 > /sys/class/net/cloudbr0/bridge/multicast_router

I follow your step, now needed any service restart ? I see same result no traffic

[alamintech]

This is my kvm server, I see multicast start ip address but not ping from this server. Please see my image below.

[levindecaro]

@alamintech I'd tried your setup, can't see any problem with all default setting, it looks doesn't related ACS nor KVM. Suggest you to try wireshark/tcpdump tools to diagnostic the problem at the first place, it can be windows firewall setting..etc.

[NuxRo]

@levindecaro @alamintech it should have been specified in the original posting that we are dealing with a Security Group zone! It's the SGs who are stopping that traffic, it would have saved time on all the testing above.

@alamintech if it helps, the up and coming 4.18.1 will allow you to create an empty L2 network in a SG zone, you could connect both your VMs into such a net and do your thing there.

[alamintech]

My setup is basic zone, not impact SG.

[levindecaro]

@alamintech , try this on your host

iptables -I FORWARD -m pkttype --pkt-type multicast -j ACCEPT

[alamintech]

iptables not enable from my kvm host.

[alamintech]

now ACS 4.18.1 version is running, This version is enable IGMP by default? or any other settings needed. Thanks.

[alamintech]

Linux server is ok multicast, but when use windows 7 then not REV and SEND packet. Please help me

[alamintech]

I see windows 7 can send multicast packet to outside, but not incoming traffic come to vm.

[alamintech]

Laptop(as Server) => VM(as Client)- not ok VM(as Server) =>Laptop(as Client) -ok

[levindecaro]

@alamintech since it is just windows 7 problem as you told, have you read this https://support.microsoft.com/en-gb/topic/multicast-packets-are-dropped-in-windows-7-or-in-windows-server-2008-r2-fdb2004f-1eca-e894-eb7e-ac0db9a9182d ?

[alamintech]

But If I use EsXi Server vm then no issue, auto IGMP is enable from EsXi, I will send to video all environment.

[alamintech]

Please see my only kvm server using multicast traffic send and receive. (Without ACS)

https://drive.google.com/file/d/1kLWSm6hzjoUj7J6wuGkQW73ppZH18H-x/view?usp=sharing

I will send next reply ACS video..

[alamintech]

Please see my ACS Advance zone vm with SG enable, packet send is ok, but no incoming traffic to vm.

https://drive.google.com/file/d/1jvh01B1Yzn1m_XyssR36VNb3DTEkRHdB/view?usp=sharing

Please see this video.

Thanks.

[weizhouapache]

@alamintech it looks the incoming traffic is blocked , probably by firewall. can you share the output of commands iptables-save and ipset list on the kvm host ?

[alamintech]

[root@kvm ~]# ipset list Name: i-2-3-VM Type: hash:ip Revision: 4 Header: family inet hashsize 1024 maxelem 65536 Size in memory: 168 References: 5 Number of entries: 1 Members: 172.22.0.63

Name: i-2-3-VM-6 Type: hash:net Revision: 6 Header: family inet6 hashsize 1024 maxelem 65536 Size in memory: 1272 References: 9 Number of entries: 1 Members: fe80::1c00:3bff:fe00:3

Name: i-2-5-VM Type: hash:ip Revision: 4 Header: family inet hashsize 1024 maxelem 65536 Size in memory: 168 References: 5 Number of entries: 1 Members: 172.22.0.65

Name: i-2-5-VM-6 Type: hash:net Revision: 6 Header: family inet6 hashsize 1024 maxelem 65536 Size in memory: 1272 References: 9 Number of entries: 1 Members: fe80::1c00:95ff:fe00:5

[alamintech]

[root@kvm ~]# iptables-save

Generated by iptables-save v1.8.4 on Thu Oct 5 14:12:15 2023

*raw :PREROUTING ACCEPT [5182138:20322138213] :OUTPUT ACCEPT [913654:448793991] COMMIT

Completed on Thu Oct 5 14:12:15 2023

Generated by iptables-save v1.8.4 on Thu Oct 5 14:12:15 2023

*filter :INPUT ACCEPT [595195:4335269819] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [913654:448794023] :s-1-VM - [0:0] :BF-cloudbr0 - [0:0] :BF-cloudbr0-OUT - [0:0] :BF-cloudbr0-IN - [0:0] :v-2-VM - [0:0] :r-4-VM - [0:0] :i-2-3-VM - [0:0] :i-2-3-VM-eg - [0:0] :i-2-3-def - [0:0] :i-2-5-VM - [0:0] :i-2-5-VM-eg - [0:0] :i-2-5-def - [0:0] -A INPUT -p tcp -m tcp --dport 49152:49216 -j ACCEPT -A INPUT -p tcp -m tcp --dport 5900:6100 -j ACCEPT -A INPUT -p tcp -m tcp --dport 16514 -j ACCEPT -A INPUT -p tcp -m tcp --dport 16509 -j ACCEPT -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT -A FORWARD -o cloudbr0 -m physdev --physdev-is-bridged -j BF-cloudbr0 -A FORWARD -i cloudbr0 -m physdev --physdev-is-bridged -j BF-cloudbr0 -A FORWARD -o cloudbr0 -j DROP -A FORWARD -i cloudbr0 -j DROP -A s-1-VM -m physdev --physdev-in vnet2 --physdev-is-bridged -j RETURN -A s-1-VM -m physdev --physdev-in vnet1 --physdev-is-bridged -j RETURN -A s-1-VM -j ACCEPT -A BF-cloudbr0 -m state --state RELATED,ESTABLISHED -j ACCEPT -A BF-cloudbr0 -m physdev --physdev-is-in --physdev-is-bridged -j BF-cloudbr0-IN -A BF-cloudbr0 -m physdev --physdev-is-out --physdev-is-bridged -j BF-cloudbr0-OUT -A BF-cloudbr0 -m physdev --physdev-out ens9f0np0 --physdev-is-bridged -j ACCEPT -A BF-cloudbr0-OUT -m physdev --physdev-out vnet2 --physdev-is-bridged -j s-1-VM -A BF-cloudbr0-OUT -m physdev --physdev-out vnet1 --physdev-is-bridged -j s-1-VM -A BF-cloudbr0-OUT -m physdev --physdev-out vnet4 --physdev-is-bridged -j v-2-VM -A BF-cloudbr0-OUT -m physdev --physdev-out vnet5 --physdev-is-bridged -j v-2-VM -A BF-cloudbr0-OUT -m physdev --physdev-out vnet6 --physdev-is-bridged -j r-4-VM -A BF-cloudbr0-OUT -m physdev --physdev-out vnet8 --physdev-is-bridged -j i-2-3-def -A BF-cloudbr0-OUT -m physdev --physdev-out vnet9 --physdev-is-bridged -j i-2-5-def -A BF-cloudbr0-IN -m physdev --physdev-in vnet2 --physdev-is-bridged -j s-1-VM -A BF-cloudbr0-IN -m physdev --physdev-in vnet1 --physdev-is-bridged -j s-1-VM -A BF-cloudbr0-IN -m physdev --physdev-in vnet4 --physdev-is-bridged -j v-2-VM -A BF-cloudbr0-IN -m physdev --physdev-in vnet5 --physdev-is-bridged -j v-2-VM -A BF-cloudbr0-IN -m physdev --physdev-in vnet6 --physdev-is-bridged -j r-4-VM -A BF-cloudbr0-IN -m physdev --physdev-in vnet8 --physdev-is-bridged -j i-2-3-def -A BF-cloudbr0-IN -m physdev --physdev-in vnet9 --physdev-is-bridged -j i-2-5-def -A v-2-VM -m physdev --physdev-in vnet4 --physdev-is-bridged -j RETURN -A v-2-VM -m physdev --physdev-in vnet5 --physdev-is-bridged -j RETURN -A v-2-VM -j ACCEPT -A r-4-VM -m physdev --physdev-in vnet6 --physdev-is-bridged -j RETURN -A r-4-VM -j ACCEPT -A i-2-3-VM -m state --state NEW -j ACCEPT -A i-2-3-VM -j DROP -A i-2-3-VM-eg -m state --state NEW -j RETURN -A i-2-3-VM-eg -j DROP -A i-2-3-def -m state --state RELATED,ESTABLISHED -j ACCEPT -A i-2-3-def -p udp -m physdev --physdev-in vnet8 --physdev-is-bridged -m udp --sport 68 --dport 67 -j ACCEPT -A i-2-3-def -p udp -m physdev --physdev-out vnet8 --physdev-is-bridged -m udp --sport 67 --dport 68 -j ACCEPT -A i-2-3-def -p udp -m physdev --physdev-in vnet8 --physdev-is-bridged -m udp --sport 67 -j DROP -A i-2-3-def -m physdev --physdev-in vnet8 --physdev-is-bridged -m set ! --match-set i-2-3-VM src -j DROP -A i-2-3-def -m physdev --physdev-out vnet8 --physdev-is-bridged -m set ! --match-set i-2-3-VM dst -j DROP -A i-2-3-def -p udp -m physdev --physdev-in vnet8 --physdev-is-bridged -m set --match-set i-2-3-VM src -m udp --dport 53 -j RETURN -A i-2-3-def -p tcp -m physdev --physdev-in vnet8 --physdev-is-bridged -m set --match-set i-2-3-VM src -m tcp --dport 53 -j RETURN -A i-2-3-def -m physdev --physdev-in vnet8 --physdev-is-bridged -m set --match-set i-2-3-VM src -j i-2-3-VM-eg -A i-2-3-def -m physdev --physdev-out vnet8 --physdev-is-bridged -j i-2-3-VM -A i-2-5-VM -m state --state NEW -j ACCEPT -A i-2-5-VM -j DROP -A i-2-5-VM-eg -m state --state NEW -j RETURN -A i-2-5-VM-eg -j DROP -A i-2-5-def -m state --state RELATED,ESTABLISHED -j ACCEPT -A i-2-5-def -p udp -m physdev --physdev-in vnet9 --physdev-is-bridged -m udp --sport 68 --dport 67 -j ACCEPT -A i-2-5-def -p udp -m physdev --physdev-out vnet9 --physdev-is-bridged -m udp --sport 67 --dport 68 -j ACCEPT -A i-2-5-def -p udp -m physdev --physdev-in vnet9 --physdev-is-bridged -m udp --sport 67 -j DROP -A i-2-5-def -m physdev --physdev-in vnet9 --physdev-is-bridged -m set ! --match-set i-2-5-VM src -j DROP -A i-2-5-def -m physdev --physdev-out vnet9 --physdev-is-bridged -m set ! --match-set i-2-5-VM dst -j DROP -A i-2-5-def -p udp -m physdev --physdev-in vnet9 --physdev-is-bridged -m set --match-set i-2-5-VM src -m udp --dport 53 -j RETURN -A i-2-5-def -p tcp -m physdev --physdev-in vnet9 --physdev-is-bridged -m set --match-set i-2-5-VM src -m tcp --dport 53 -j RETURN -A i-2-5-def -m physdev --physdev-in vnet9 --physdev-is-bridged -m set --match-set i-2-5-VM src -j i-2-5-VM-eg -A i-2-5-def -m physdev --physdev-out vnet9 --physdev-is-bridged -j i-2-5-VM COMMIT

Completed on Thu Oct 5 14:12:15 2023

Generated by iptables-save v1.8.4 on Thu Oct 5 14:12:15 2023

*nat :PREROUTING ACCEPT [6464:506599] :INPUT ACCEPT [2469:156500] :POSTROUTING ACCEPT [11120:744935] :OUTPUT ACCEPT [7426:413470] COMMIT

Completed on Thu Oct 5 14:12:15 2023

Generated by iptables-save v1.8.4 on Thu Oct 5 14:12:15 2023

*mangle :PREROUTING ACCEPT [5182144:20322138417] :INPUT ACCEPT [712195:4345441269] :FORWARD ACCEPT [4495090:15978470369] :OUTPUT ACCEPT [913688:448796727] :POSTROUTING ACCEPT [5403355:16426889245] COMMIT

Completed on Thu Oct 5 14:12:15 2023

[weizhouapache]

@alamintech for troubleshooting, can you retry after adding two new rules ?

iptables -I i-2-3-def -j ACCEPT
iptables -I i-2-5-def -j ACCEPT

You can remove them after testing by

iptables -D i-2-3-def -j ACCEPT
iptables -D i-2-5-def -j ACCEPT

[alamintech]

yes, now traffic IN and OUT, needed permanently solve, which rules adding for this. Thanks.

[weizhouapache]

@alamintech the firewall rules are generated by /usr/share/cloudstack-common/scripts/vm/network/security_group.py you can modify the file on kvm host

@@ -1198,6 +1198,7 @@ def add_network_rules(vm_name, vm_id, vm_ip, vm_ip6, signature, seqno, vmMac, ru

         vmchain = iptables_chain_name(vm_name)

+        execute('iptables -A ' + vmchain + ' -m pkttype --pkt-type multicast -j ACCEPT')
         execute('iptables -A ' + vmchain + ' -j DROP')
         execute('ip6tables -A ' + vmchain + ' -j DROP')

it is not tested

[levindecaro]

@weizhouapache

forward chain also need to grant for mutlicast, otherwise multicast won't work between vm.

[alamintech]

@weizhouapache

your code is working fine, Thank you so much!

[DaanHoogland]

@weizhouapache , is this code needed in main (or other branches) ? And can we close this issue?

[weizhouapache]

@weizhouapache , is this code needed in main (or other branches) ? And can we close this issue?

@DaanHoogland I think we need to fix it. I am not sure when the rule needs to be added, maybe when

• protocol is all , or
• protocol number is 92 (Multicast Transport Protocol, MTP)

There is a similar issue with GRE protocol https://lists.apache.org/thread/1rcllrzdroh6jpf5hz4wfyll69ocvsn6