Multicast Relay on Raspberry Pi - HomeKit Issues (Detailed Breakdown)

VeniceNerd commented 2 years ago

SPOILER EDIT

It seems that I was able to get everything working with SETUP 6. I am going to leave the entire post up as it may be a good guide for others.

However, I am still experiencing issues with one device that will only sporadically work via Multicast-Relay. I would still like to get to the bottom of this. Details can be found in SETUP 1 and SETUP 6. I am also trying to understand a mystery of ifFilter outlined in SETUP 7. Any help on those would still be appreciated!

ORIGINAL POST

I have been trying to get Multicast Relay to work for almost two years. I still can not get it to work properly and it's driving me insane. I am hoping that the community may have some answers for me. To not waste anyone's time I'm going to try to be very detailed.

OBJECTIVE

I have three VLANS. VLAN 10, 20, and 30. I would like to send mDNS traffic from VLAN 30 to both VLAN 10 and VLAN 20. I do not want VLAN 10 and VLAN 20 to send mDNS traffic to each other.

NETWORK SETUP

Unifi Dream Machine Pro SE UniFi OS v2.5.11 Network App v7.2.94

NETWORKS:

10.1.1.0 - Main Network
10.1.10.0 - VLAN 10 (Home)
10.1.20.0 - VLAN 20 (IOT Devices)
10.1.30.0 - VLAN 30 (Office)

SETTINGS:

Settings → Networks: Multicast DNS disabled
Settings → Networks: IGM Snooping disabled
Settings → Wifi → Each Wifi Network: Multicast Enhancement disabled
Settings → Wifi → Each Wifi Network: Multicast and Broadcast Control disabled

DEVICES The following mDNS enabled devices are on each VLAN. I will only look at "_airplay._tcp", "_hap._tcp", and "_home-assistant._tcp" devices in the Discovery App. I will use these to test the various setups.

VLAN10 Living Room ATV (_airplay._tcp) - 10.1.10.10:7000 - LAN

VLAN20 Bedroom ATV (_airplay._tcp) - 10.1.20.96:7000 - WLAN

VLAN30 Homebridge (_hap._tcp) - 10.1.30.3:51407 - LAN Hue BCN (_hap._tcp) - 10.1.3.10 - LAN Sensibo (_hap._tcp) - 10.1.30.25:80 - WLAN Shapes (_hap._tcp) - 10.1.30.21:6517 - WLAN Barcelona (_home-assistant._tcp) - 10.1.30.2:8123 - LAN

OBSERVATIONS: Using the Discovery App I can confirm that no mDNS devices from any VLANS show up in the other VLANS. They are only visible in their own VLAN. This is what I would expect.

RASPBERRY PI SETUP

I have decided to run MulticastRelay on a Raspberry Pi instead of the DMPSE itself. I don't want things to get messed up during UnifiOS upgrades and I also couldn't find great resources to get this running on the SE.

RASPBERRY PI Raspberry Pi 4 Bullseye 32bit (without Desktop)

NSTALL DOCKER

"curl -sSL https://get.docker.com | sh" → to install docker
"sudo usermod -aG docker ${USER}" → to add permission to pi user.

VLAN CONFIGURATION

"sudo apt-get install vlan" → to install VLAN support
Created "vlan" file in /etc/network/interfaces.d with the following contents:

auto eth0.10
iface eth0.10 inet manual
  vlan-raw-device eth0

auto eth0.20
iface eth0.20 inet manual
  vlan-raw-device eth0

auto eth0.30
iface eth0.30 inet manual
  vlan-raw-device eth0

auto eth0.60
iface eth0.60 inet manual
  vlan-raw-device eth0

"Sudo reboot" → restart pi
"Ifconfig" → to confirm the VLAN networks show up:

MULTICAST RELAY SETUPS

Next I am going to show the various ways I tried to achieve my objective.

SETUP 1 - BASIC CONFIGURATION

The most basics Multicast-Relay setup to test functionality. Here I simply ask Multicast Relay to transmit information between all three VLANS.

INSTALL COMMAND

docker run -it -d \
--network=host \
--name multicast-relay \
--restart=always \
-e OPTS="--verbose" \ 
-e INTERFACES="eth0.10 eth0.20 eth0.30" \
docker.io/scyto/multicast-relay

OBSERVATION

While being connected to VLAN10 discovery app now shows the following devices:

Here I notice a small issue, as my "Sensibo" device does not show up. I can see it, as soon as I switch to VLAN30:

I have seen the Sensibo device show up sporadically on VLAN10 but when I'm connected to VLAN30 (where it resides) it permanently shows up in Discovery App. It also sporadically stops responding in HomeKit. When using the build in "Multicast DNS" service of the Dream Machine Pro this device is also broadcast to all VLANS. For some reason it does not work with multicast-relay.

Is there any way to troubleshoot this?

LOGS I let the container run for about 15 minutes to create the log file: log1.txt

SETUP 2 - USING IFFILTER OPTION

Next I will try to use ifFilter.json to broadcast VLAN 30 to VLAN 10 and VLAN 20.

INSTALL COMMAND

docker run -it -d \
--network=host \
--name multicast-relay \
--restart=always \
--mount type=bind,src=/home/pi/multicast-relay,dst=/multicast-relay-config \ 
-e OPTS="--verbose --ifFilter=/multicast-relay-config/ifFilter.json" \
-e INTERFACES="eth0.10 eth0.20 eth0.30" \
docker.io/scyto/multicast-relay

ifFilter.json Content

{
"10.1.10.0/24": ["eth0.10"],
"10.1.20.0/24": ["eth0.20"],
"10.1.30.0/24": ["eth0.10", "eth0.20"]
}

→ I am asking VLAN 10 and VLAN 20 to just broadcast to themselves. If I don't include VLANS 10 and 20 in the ifFilter they will broadcast to all networks according to @alsmith.

OBSERVATION This is where things get really strange. VLAN10 and VLAN20 don't seem to talk to each other so that is good. However, only the Homebridge from VLAN30 show up in VLAN10 (via Discovery App):

LOGS log2.txt

From the logs I notice a few things:

No devices from VLAN 10 or 20 are transmitting. This is as expected.
My Hue (10.1.30.10) almost exclusively transmits SSDP data now. It no longer transmits any mDNS data. After I restarted the Hue bridge around 15:33h and 15:55h some mdns data seems to have been transmitted briefly. It then went back to just SSDP. In Setup 1 it continuously keeps sending mDNS data.
Homebridge (10.1.30.3) is the only device that regularly keeps sending mDNS info
Sensibo (10.1.30.25) only sent mDNS info once.

Thoughts

What seems to happen in Setup 2 is that all of a sudden mDNS data from most of my VLAN30 devices is no longer being transmitted. Only one device (Homebride 10.1.30.3) continuously keeps transmitting mDNS data. Why would including the ifFilter option have any impact on this?'

I am wondering if it is perhaps a problem that the HomeKit Hub (Apple TV 10.1.1.10 on VLAN 10) can't send mDNS information back to VLAN30 in this setup. I don't know enough about HomeKit to know if mDNS must be sent both ways.

SETUP 6 I am now trying something that I've tried last year but it didn't work back then. However, it appears to work now.

INSTALL COMMAND

docker run -it -d \
--network=host \
--name multicast-relay \
--restart=always \
--mount type=bind,src=/home/pi/multicast-relay,dst=/multicast-relay-config \ 
-e OPTS="--verbose --ifFilter=/multicast-relay-config/ifFilter.json" \
-e INTERFACES="eth0.10 eth0.20 eth0.30" \
docker.io/scyto/multicast-relay

IF FILTER

{ "10.1.10.0/24": ["eth0.30"], "10.1.20.0/24": ["eth0.30"], "10.1.30.0/24": ["eth0.10", "eth0.20"] }

RESULTS This seems to work! I can not see mDNS traffic between VLANS 10 and 20 (as intended) but mDNS information is being transmitted between VLAN30 to VLAN10 and VLAN 30 to VLAN20 (bi directional). Last year this did not work as traffic from VLAN 10 and 20 would be sent to VLAN 30 and then VLAN30 turned around and sent it back to VLANS 10 and 20 basically ending in a scenario where VLAN 10 and VLAN 20 would end up sharing mDNS information. I don't know why this works now but it works.

However, the Sensibo devices is still intermittent. This is the same issue I'm seeing in Setup 1 above.

LOGS log6.txt

SETUP 7 At this point I just want to make sure I understand everything better so I can troubleshoot in the future. So I am changing the iFFilter from Setup 6 to:

{
"10.1.10.10/32": ["eth0.30"],
"10.1.10.0/24": ["eth0.10"],
"10.1.20.0/24": ["eth0.20"],
"10.1.30.0/24": ["eth0.10", "eth0.20"]
}

Here I am basically trying to combine Setup 2 and 6. I am telling VLAN 10 to only transmit to VLAN 10 and VLAN 20 only to VLAN 20. This did not work in Setup 2. However, here I am also adding this line:

"10.1.10.10/32": ["eth0.30"]

With that I am trying to tell the relay to take JUST relay traffic from my AppleTV HomeKit up on 10.1.10.10 and send that to VLAN 30.

However, from the logs I can see that no information from 10.1.10.10 is being sent: log7.txt

So I still don't think I understand the ifFilter fully. Why doesn't this work?

SUMMARY

I was able to solve this with Setup 6. However, even in this setup one of my devices (Sensibo) only seems to transmit mDNS traffic sporadically. It comes and goes. So there is still something going on here that needs to be figured out.

I also don't think I understand ifFilter fully. In Setup 7 I am trying to send all traffic from VLAN 10 to itself BUT send traffic from one specific ip address (10.1.10.10) to VLAN 30. This does not seem to work. Why?

So the following questions remain:

Why is one of my devices 10.1.30.25 only working sporadically in Setup 1 and Setup 6 when everything else seems to work? It shows up in the Discovery App when connected to its native VLAN 30 but only shows up sporadically on the VLANS I relay to. The Sensibo device also works perfectly when using the Multicast DNS service of the Dream Machine.
Why does the ifFilter logic in Setup 7 not work? What am I doing wrong here?

neles86 commented 1 year ago

Hey. Your breakdown helped me and all is working. Quick question. Is there any advantage of using the Multicast relay mDNS vs Unifi mDNS?

VeniceNerd commented 1 year ago

Hey. Your breakdown helped me and all is working. Quick question. Is there any advantage of using the Multicast relay mDNS vs Unifi mDNS?

I’m glad it helped. I eventually gave up since it never 100% worked for me.

I guess the theoretical benefit is that you can be more granular. The Unifi mDns broadcasts to all selected VLANS.

scyto commented 1 year ago

nice write up, thanks!

one thing i will say is many clients don't purely use multicast traffic to do connections they sometimes use unicast for certain functions too (this is the case with sonos).

so your issue may go beyond normal mDNS multicast to other multicast group protocols that do discover like SSDP or even to unicast protocols

k

VeniceNerd commented 1 year ago

nice write up, thanks!

one thing i will say is many clients don't purely use multicast traffic to do connections they sometimes use unicast for certain functions too (this is the case with sonos).

so your issue may go beyond normal mDNS multicast to other multicast group protocols that do discover like SSDP or even to unicast protocols

k

Yes this is absolutely possible. I just know when I turn on the Unifi mdns all my devices work. When I use multicast-relay it’s intermittent. Maybe Unifi mdns does more than just mdns?

Carl-Higby commented 1 year ago

Wait so did you ever get this running ? I have a UDM Pro and trying to set up a very similar network it does not work. I can not get Sonos to work across vlans for the life of me. Lots of older info that doesn't seem to apply anymore. Any help would be greatly appreciated.

neles86 commented 1 year ago

Wait so did you ever get this running ? I have a UDM Pro and trying to set up a very similar network it does not work. I can not get Sonos to work across vlans for the life of me. Lots of older info that doesn't seem to apply anymore. Any help would be greatly appreciated.

I got this running on my raspberry pi 4 using steps mentioned in the write up. I specifically did this for my Sonos devices across vlans. I had the multicast relay working on my old UDM Pro (pre-v2) using podman. I now have a UXG pro and since this doesn't apply anymore the RPI4 route was the way to go.

scyto commented 1 year ago

nice write up, thanks! one thing i will say is many clients don't purely use multicast traffic to do connections they sometimes use unicast for certain functions too (this is the case with sonos). so your issue may go beyond normal mDNS multicast to other multicast group protocols that do discover like SSDP or even to unicast protocols k

Yes this is absolutely possible. I just know when I turn on the Unifi mdns all my devices work. When I use multicast-relay it’s intermittent. Maybe Unifi mdns does more than just mdns?

mdns isn't all multicast (its just a discovery protocol)

multicast relay relays specific groups of multicast (you can add additional multicast groups, it's also a relay not a router or proxy)

UI's mdns toggle IIRC does more than just mdns too...

i am hoping the new IGMP proxy feature of OS3 removes the need for any of this container (i only built it because i was annoyed by the daily occurrence of people asking about sonos, lol)

scyto commented 1 year ago

Wait so did you ever get this running ? I have a UDM Pro and trying to set up a very similar network it does not work. I can not get Sonos to work across vlans for the life of me. Lots of older info that doesn't seem to apply anymore. Any help would be greatly appreciated.

the way the sonos works is (IIRC) the client uses multicast to find the server, the server then makes a unicast connection back to the client (this is first time only) as such you need to make sure you have enough firewall ports open from the VLAN where the sonos device is back to the LAN (aka a firewall rule that lets the sonos contact any client it wants.... at which point you have to ask WTF was the point fo putting it on a VLAN... lol) so you should always start with zero traffic being blocked between VLANS/LAN get stuff working and then build up rules

(note the one time discover unicast issue can make this container look unreliable - it isn't, it is the sonos behavior changes)

of course you may have a different issue entirely!

scyto / multicast-relay