WSD between Vlan's

[Ramalama2]

Hi, im sorry, but i can't fix it myself and gaved up now the debugging :-( And im pretty sure that it is not an bug either, but however, in short:

My Setup:

Server (with VMs + opnsense) <--> Dumb Switch (has only multiple vlans but doesn't route between them) <--> Win10 PC Or Simpler: Samba-Srv (Vlan 25) --> (Vlan25) OpnSense (Vlan 27) --> Dumb Switch --> W10 PC

On the OpnSense i do the Multicast reflection with "udpbroadcastrelay" This Service listens on port 3702 and reflects everything between this both Multicast Domains (of Vlan25+27)

This seems working, have testet it with socat + "wsdd -vv" Without that "udpbroadcastrelay" i receive nothing on the samba server. And with it, i get the XML and wsdd seems even to respond.

But yeah, nothing shows up in the windows explorer.

If i move the PC to Vlan 25 (where samba-srv is)... Well, its working all perfect xD

But i don't get what i miss, i mean the reflection seems working and the firewall is completely open for IPv4+6 between Vlan25 & 27.

However, Logs from socat + logs from wsdd are attached +3 screenshots from the opnsense. Sorry to bother, i know that it is a layer 8 problem, but probably you have some ideas :-)

Thanks :-) -> Logs: Logs.txt

[christgau]

Interesting problem. Here's what I can tell from the logs and your screenshots

1. wsdd sends out its "Hello" and "Bye" message on startup and shutdown for IPv4 and IPv6. That's a good thing and intended.
2. However the "Hello" message should already be followed by an HTTP POST request from the Windows machine (from its IPv6 link local address or its IPv4 address). This is not the case.
3. In case you do a refresh in the Network View of the Explorer wsdd receives a "Probe" message from Windows via IPv4 (not via IPv6) and it sends out a "ProbeMatch" reply. This is again intended. However, after the "ProbeMatch" the Windows machine should send a "Resolve" message which I can't see in the log.

In both cases ("Hello" and "Probe") the message interchange appears to be interrupted. It looks as if the flow from the wsdd host to Windows is interrupted. The opposite direction appears to work since the Probe message is received by wsdd (at least for IPv4).

For me it looks like the firewall setup is not correct. I cannot judge on the Opnsense configuration if it really allows all traffic for either address family. Please also note that you must allow TCP traffic for port 5357 on the wsdd machine as well.

[Ramalama2]

Well, you brought me to the idea to make a complete "working" log first. Then compare what i miss.

As far i understood after this probe, the windows computer should contact wsdd on port 5357. But this doesn't happens, because either windows doesn't receive the reply proble from wsdd or 5357 somewhere blocked to wsdd.

So i need to debug this 🙈

Can be igmp snooping on a switch a problem? Because every switch has it and it controls multicast... But dunno how it works between vlans 🙈 However will try to turn it off either.

Thank you very much. You gaved me good hints!

[christgau]

As far i understood after this probe, the windows computer should contact wsdd on port 5357.

No.

There are two message flows to distinguish here:

Announcement at Startup with Hello

1. wsdd send "Hello" message to multicast address (obviously udp) which contains the IP address of the host
2. Windows (or another WSD host implementation) performs so called metadata exchange via tcp/http on port 5357.

General Discovery

1. Windows sends a "Probe" message to multicast
2. wsdd replies with "ProbeMatch" to the unicast address the "Probe" message came from (udp)
3. Step 1 and 2 repeat but with "Resolve" and "ResolveMatch" messages. The IP address of the wsdd host is transferred in the Match message. Again udp.
4. Metadata exchange as described above (tcp/http)

You can find a picture for that here: https://docs.microsoft.com/en-us/windows/win32/wsdapi/discovery-and-metadata-exchange-message-patterns Note that the "Hello" flow is not fully shown there.

Can be igmp snooping on a switch a problem?

That could actually be a problem. I don't know how multicast group membership is handled between VLANs... or (to put it in another way) if a membership to the same group is even possible when hosts are on different VLANs and there is "converter" in between.

However will try to turn it off either.

Good idea to remove simplify the problem.

Thank you very much. You gaved me good hints!

🇩🇪 Gerne 😉

[Ramalama2]

Hehe, hab schon mitbekommen das du deutschsprachig bist 😂

Back to english. I have disabled igmp snooping on all switches in between, doesn't made any change 🙈

However, i tested today a bit, but don't have enough energy left to find the problem. Will continue tomorrow 🙈

It's definitively something with the opnsense. Opnsense has other plugins, like mdns-reflector or igmp proxy, but those doesn't work at all 🙈

However, i will find out, get it running and post the solution.

Thanks and good evening ✌️😂

[Ramalama2]

Tryed today a virtual setup with ipfire/openwrt. Then all on hardware, because i thinked it's a proxmox bug. But well, it isn't opnsense or proxmox.

And i tryed a full virtual setup on proxmox (win10+openwrt/opnsense+wsdd), to get the physical switches out of the way 🤣

Nothing helped, and everything in the stupid firewall logs is green (allowed).

Seems for me like the windows host somehow ignores or doesn't reply to the probe, if it doesn't comes from a network that windows doesn't know 🤣 (from gw)

Well, i need somehow to debug what windows does exactly, but I don't know how 🙈 The only way is port mirroring and wireshark, but ugh, that's hardcore 🤣🤣🤣

Is there maybe any linux way?

Like in theory i could let talk wsdd with wsdd on different vlans? Or anything else that would make my tests easier? Wish i could have simply a tool, where i can see exactly what's wrong, but in the end i see me with wireshark and inspecting all the pakets and still failing on this 🤣🤣🤣

[christgau]

And i tryed a full virtual setup on proxmox (win10+openwrt/opnsense+wsdd), to get the physical switches out of the way

Have you tried to eliminate openwrt/opensense an putting Win10 on the same VLAN. This would ensure that the setup actually works. If not the issue might not be firewall

Seems for me like the windows host somehow ignores or doesn't reply to the probe, if it doesn't comes from a network that windows doesn't know 🤣 (from gw)

It could be that Windows actually ignores messages from networks it does not "know". Are the Windows host and the wsdd machine on the same IP network (layer 3) although they are on different VLANs (layer 2)? Or to put in another way: What are the IP addresses (plus "netmask") of the Server/wsdd machine and the Windows host?

[Ramalama2]

And i tryed a full virtual setup on proxmox (win10+openwrt/opnsense+wsdd), to get the physical switches out of the way

Have you tried to eliminate openwrt/opensense an putting Win10 on the same VLAN. This would ensure that the setup actually works. If not the issue might not be firewall

Seems for me like the windows host somehow ignores or doesn't reply to the probe, if it doesn't comes from a network that windows doesn't know 🤣 (from gw)

It could be that Windows actually ignores messages from networks it does not "know". Are the Windows host and the wsdd machine on the same IP network (layer 3) although they are on different VLANs (layer 2)? Or to put in another way: What are the IP addresses (plus "netmask") of the Server/wsdd machine and the Windows host?

Sorry, i didn't seen that you replyed 🤦‍♂️

However, in short:

Vlan 24 - 192.168.24.0/30 (Transfer Wan for pppoe) Vlan 25 - 192.168.25.0/24 (Servers) Vlan 26 - 192.168.26.0/24 (Voip) Vlan 27 - 192.168.27.0/24 (Users) Vlan 28 - 192.168.28.0/24 (Wlan) Vlan 29 - 192.168.29.0/24 (Vpn)

This is quite simple home setup, just sounds super complex 😂

The samba server is on (25) The w10 client is on (27)

Well if both are on the same network (L2) / aka same vlan, wsdd works perfectly. I see the linux server even before everything else in the windows network discovery 🤣

If they aren't on same vlan (different networks / L3), wsdd doesn't work.

Usually there are multicast reflectors, that reflect multicast from one network to another. Thats what im using (uddbroadcastrelay) with opnsense.

All what it does is: Listening on udp port 239.255.255.250:3702 on both networks (vlan 25 & 27) and reflects everything between those 2 networks. (It's a bit more complex for sure, but im a noob)

However, with the udpbroadcastrelay, wsdd sees the probe from windows and is replying to it.

Without udpbroadcastrelay, there is nothing, not even a probe. This makes absolutely sense, because both vlans are own broadcast/multicast domains.

For example if you ping 255.255.255.255 on vlan 25, nothing of vlan 27 would answer, cause 2 different networks 🙈

But aside from broadcast/unicast/multicast, which are naturally layer 2 cant work without "reflectors" between 2 networks, layer 3 communications (ip) can reach everything between both networks. There is no nat or anything between. It just need to get routed between both networks through a route 🤣

So basically, i need to make multicast/unicast/broadcast working between wsdd and the w10 clients, because for everything else they can communicate with each other.

Probably there is more involved in wsdd, other as 239.255.255.250 on port 3702. Or does windows checks for a mac address? Cause the package from other network, will have the right ip, but the macaddress would be the one from the gw.

Sorry if my explanation sounds like for idiots 🤦‍♂️ Just wanted to be detaily 🤦‍♂️🤣

[Ramalama2]

Ah one other thing.

Udpbroadcastrelay doesn't support ipv6 🤦‍♂️

But all my networks are properly ipv4 & v6 configured.

Means on layer 3 everything with ipv6 is reachable by everyone between the networks. Only multicast isn't. Cause the udpbroadcastrelay can't ipv6 at this moment. Unicast/broadcast works different on ipv6, so i don't know, they are probably working between networks.

However, i tested this either, disabled on w10 the ipv6 stack and it doesn't change anything 🙈🤣

[christgau]

Only multicast isn't. Cause the udpbroadcastrelay can't ipv6 at this moment.

OK. Then let's stick with IPv4 in the discussion.

Vlan 25 - 192.168.25.0/24 (Servers) Vlan 27 - 192.168.27.0/24 (Users) [...] The samba server is on (25) The w10 client is on (27)

Well if both are on the same network (L2) / aka same vlan, wsdd works perfectly. I see the linux server even before everything else in the windows network discovery

Do they share the same IPv4 address range in that case or do the hosts have are different address ranges as listed above, so from different IPv4 subnets. I ask because if they are from different subnets the routing must work correctly.

Let's recap a little: When the hosts are on different VLANs, we see that multicast traffic arrives from Windows (Probe) and - as pointed out above - the flow is interrupted after the ProbeMatch is sent (at least the wsdd log lets me conclude that). So the question is: Where or why does this message disappear? Up to now, we don't know if the ProbeMatch arrives on the Windows host. It doesn't look so, because the subsequent Resolve message does not appear in the wsdd log, but - again - we don't know if ProbeMatch really arrives. I would suggest to check for this fact now. Maybe you can install wireshark on a Windows machine and check for UDP traffic that was sent from port 3702. You may use the filter udp src port 3702 as a filter. You may also extent the filter to udp src port 3702 or udp dst port 3702 to see the whole udp-based conversation between wsdd and Windows. Make sure you use the latest wsdd release.

In you initial post you mentioned that

the firewall is completely open for IPv4+6 between Vlan25 & 27.

However, looking the screenshot from the config again it appears to me that UDP/IPv4 traffic is only allowed for specific port unrelated to wsdd. Further, only traffic coming from the user net is allowed, but what about traffic from user net to the server net? I ask this again because I have no experience with Opnsense but from my nft/iptables knowledge I would assume no packets to the server net are allowed given a reject policy.

[Ramalama2]

Maybe i make it a bit easier :-)

Wireshark: W10-MW = Client = 192.168.27.15 linux-srv = wsdd = 192.168.25.4 druck_mfc7360 = Stupid wss printer = 192.168.27.9 (it's on same lan as the pc, so the printer works, didn't tested him in vlan, could do it xD)

Wireshark cut, of exactly one refresh in windows explorer: One_Discovery_Refresh.zip

So the message arrives from wsdd xD

[christgau]

Maybe i make it a bit easier :-)

Only a little 😉 For me, the last two rules only allow traffic from the same network, e.g. source address is from LAN User (192.168.27.0/24), but what about the other networks? Regardless of this, the reason for the issue maybe really is Windows since the ProbeMatch message arrives at Windows.

Wireshark: W10-MW = Client = 192.168.27.15 linux-srv = wsdd = 192.168.25.4 druck_mfc7360 = Stupid wss printer = 192.168.27.9

After seeing the dump and excluding the firewall as a potential blocker of the ProbeMatch message, my impression is that Windows is actually involved here. The whole WSD thing is for service discovery on your local network. The Linux server and the Windows client are on different networks from the Windows perspective. Thus, wsdd replies to a Probe request from a network that Windows assumes is not the local one - and from IP address perspective this is actually true. Remember that you already had to do a handstand by reflecting the multicast traffic to another, i.e. non-local, network. Apparently this works but windows appears to be "smart" enough to check that a reply is not coming from the local network.

I fear that nothing can be done from the wsdd side here.

([the printer] is on same lan as the pc, so the printer works, didn't tested him in vlan, could do it xD)

That would be interesting. If it really works on the Linux subnet, another pcap dump would be really appreciated. 😉

Btw, wsdd has a discovery mode which can be used to search for Windows or Linux hosts running wsdd. However, wsdd is (currently) not doing such "smart" checks whether the address is from the local net or not. If you place a Linux machine in your VLAN 27/Users net and launch the discovery mode you may see the Linux hosts from the other network. To be fair: This won't really help you to convince Windows that is should detect the hosts as well. But it would be interesting to see if it works.

[Ramalama2]

I will do another pcap, no problem, but a bit later, cause i think i know what the problem is xD

I understand now what you mean, don't compare the rules above with iptables :-)

The picture above is LAN_USER, imagine LAN_USER as an interface port, that means "LAN_USER net" as source on LAN_USER interface, its same as this: this----: Allow from Interface LAN_USER from source "LAN_USER net" to ANY is this-: Allow from Interface 192.168.27.1 from source "192.168.27.0/24" to ANY

in theory, i can exchange "LAN_USER net" with a wildcard, since the source on that interface can anyway be only 192.168.27.xx... And that something else has a different ip on that network, is impossible or a static misconfiguration anyway :-)

However, the Rules in the LAN_SRV section are identical, just with "LAN_SRV net"... which i can replace with an wildcard wither xD

Did the wildcard now for you, that you feel better, but it doesn't changes anything xD

about the issue that windows ignores the probereply, you might have absolutely right, im googling right now, maybe i find something xD

And i found this here: http://ws4d.org/dpws-explorer/ looks exactly what i want for my testings and playings :-)

Im cloning a vm right now, to play with that tool, cause i don't want to install java on my pc xD

However, need a bit time xD

[Ramalama2]

Okay in short, i found a major issue on freebsd (opnsense/pfsense). Which gooes back to freebsd bugs with q35 passthrough (passing through sr-iov intel virtual ethernet nics). This is not an issue on linux. But i had to fight with the last 2 days, to halfway fix it. (It's still not fixed, will be fixed once freebsd 13 is out)

However that has nothing with this here todo, just that i have multiple problems, and not able to test everything fast out 🙈

However, i had half success, original windows servers can't see each other either on different vlans. Printer is not discoverable either between 2 vlans.

I was able to get it working in a virtual environment, with some dirty hacks and macvtap bridge. But that's inside proxmox and basically i mirrored a port from one vlan inside another vlan, that's like a short of multicast networks 😂 However, that's not replicatable in real environment 😂

But i will dig today later.

The tool above didn't worked btw. It works but not for wss. There are some security implementations and whatever 🤦‍♂️😂

[christgau]

However, i had half success, original windows servers can't see each other either on different vlans. Printer is not discoverable either between 2 vlans.

Ok, looks like Windows is behaving consistently to ignore hosts from other VLANs/"non-local" IP addresses. Overall, wsdd is obviously not the cause of your issue 😀

The tool above didn't worked btw.

In the Windows SDK there is tool named wsddebug_client which can be used to test implementations and scan for devices.

I'm closing this issue since wsdd is not the cause here (see above), but I'm happy to discuss it further...

[Ramalama2]

I told you in the beginning, that's it's not a wsdd bug already 🙈

Wsdd works pretty perfect. If i get my windows clients working and wsdd won't, then it is a bug, but otherwise 😂

However, thank you very much for all the help, if i find a solution, i will post it here. About the debug client, i have it already in a test vm, but the client is a bit hard to work with, you need the scheme etc and it only sends like one probe and doesn't listens for any reply, so not a real help.

But there is a way to enable debugging, exactly for wsd and do a windows explorer refresh. Just the output (the logfile) is something microsoft specific crap that you can't really read 🙈 https://docs.microsoft.com/en-us/windows/win32/wsdapi/enabling-wsdapi-tracing

However, additionally to all that, i didn't took really some hours in the last days, to sit and try everything out and test etc. Cause of yeah, i have to work either and am a bit time limited.

However, i will get it solved at some point. Will reply then. Thank you very much for all the help!

Cheers ✌️

[christgau]

I told you in the beginning, that's it's not a wsdd bug already 🙈

Yes, but you never know... 😉

if i find a solution, i will post it here.

Great.

About the debug client, i have it already in a test vm, but the client is a bit hard to work with, you need the scheme etc and it only sends like one probe and doesn't listens for any reply, so not a real help.

Oh it works quite well for me. Just launch it from the command line (or directly from Explorer/shell) and put the thing into discovery or metadata mode (mode discovery). By using the probe (probe type [dpws]:Device) and resolve (resolve urn:uuid:UUID) commands you can then perform the message exchange steps one after another. If you are in metadata mode, the resolve command also triggers the metadata exchange after a ResolveMatch is received.

But there is a way to enable debugging, exactly for wsd and do a windows explorer refresh. Just the output (the logfile) is something microsoft specific crap that you can't really read 🙈 https://docs.microsoft.com/en-us/windows/win32/wsdapi/enabling-wsdapi-tracing

Thank you very much for all the help!

You're welcome.

[Ramalama2]

Hey @christgau , after an eternity, i found out what the exact issue is 😂

If the discovery package comes from an unknown network, or not from the same network as the windows client is, windows won't reply and ignore it simply.

There may be a way to get it still working, but that requires some changes in udpbroadcastrelay. In short udpbroadcastrelay needs to act like a proxy instead of a relay. (That's in case for opnsense/pfsense)

That means simply, that there is currently no solution to this.

Cheers and thanks for your work! ✌️