Device, published by RSSDP, is invisible on local network

[sensboston]

If it's a duplicate, probably it's still not resolved yet.

I'm using UPnP-Pentest-Toolkit to test Rssdp.Samples (console program from master branch), also built the same program but with nuget package (to be 100% sure) - no "Sample RSSDP Device" visible on local network (and same happened with my real DLNA media renderer program). By the way, I can find published device with SSDP code (another UPnP C# code from github) but looks like it's still messy for the standard SSDP browsers/devices.

I tried on the same PC running Windows 10, and on different PCs, Win10 and Win7 - same result, device is invisible...

Any advice please? Could you tell me, how to make your published device (from sample) visible in the UPnP-Pentest-Toolkit? (it has a precompiled binaries located here). WinUPnPFun utility works 100% correct with the other UPnP devices on my local network (better than Intel's utility - they can't find my smart TV).

P.S. I have a troubles to build https://github.com/nccgroup/UPnP-Pentest-Toolkit , to see what's happened inside (this assembly uses native C++ code). BTW, I'll keep trying to get debug build working and see, what's wrong with the RSSDP.

[Yortw]

Hi,

I'm not familiar with UPnP-Pentest-Toolkit, I can investigate but not sure when (at work at the moment and have a busy week ahead).

One common problem with other tools is they won't show devices they can't get the Device Description Document for. The sample console app publishes the device with a faked DDD URL, but it does not actually host an end point for retrieving the DDD from. It's possible that is your problem. If you can stand up an end point that actually returns a correctly formatted DDD with the same device name/id etc. then you may find UPnP-Pentest-Toolkit finds the device.

[sensboston]

UPnP-Pentest-Toolkit is a nice tool, working out of the box.

As for device description, please take a look to my real code (for DLNA media renderer based on your library):

        void PublishNewDevice()
        {
            if (_devicePublisher != null) _devicePublisher.Dispose();

            // Create a device publisher
            _devicePublisher = new SsdpDevicePublisher(new SsdpCommunicationsServer(new SocketFactory(_localIP))); //, NullLogger.Instance);

            // Create the device(s) we want to publish.
            _rootDevice = new SsdpRootDevice()
            {
                CacheLifetime = TimeSpan.FromMinutes(30),
                FriendlyName = _config.Settings.DeviceName,
                Manufacturer = "SeNSSoFT",
                ManufacturerUrl = new Uri("http://senssoft.com"),
                ModelNumber = "001",
                ModelName = "DLNA renderer for Windows PC",
                ModelDescription = "Simple DLNA Renderer for Windows PC",
                DeviceType = "urn:schemas-upnp-org:device:MediaRenderer:1",
                UrlBase = _baseUri,
                Location = new Uri(string.Format("http://{0}:{1}/device.xml", _localIP, _config.Settings.LocalPort)),
                Uuid = _config.Settings.DeviceUuid
            };
            _rootDevice.CustomResponseHeaders.Add(new CustomHttpHeader("X-X_DLNADOC", "DMR-1.50"));

            var service = new SsdpService()
            {
                Uuid = Guid.NewGuid().ToString(),
                ServiceType = "urn:schemas-upnp-org:service:RenderingControl:1",
                //ServiceTypeNamespace = "urn:schemas-upnp-org:service:AVTransport:1",
                ControlUrl = new Uri("control/AVTransport", UriKind.Relative),
                EventSubUrl = new Uri("event/AVTransport", UriKind.Relative),
                ScpdUrl = new Uri("dmr_avts.xml", UriKind.Relative)
            };
            _rootDevice.AddService(service);

            // Now publish by adding them to the publisher.
            _devicePublisher.AddDevice(_rootDevice);
        }

This is an xml file what my http server send on request (but of course I corrected the IP address and port). Practically, I copied (with some changes) this device description from my smart TV ;)

<?xml version="1.0" encoding="UTF-8"?>
<root xmlns="urn:schemas-upnp-org:device-1-0" xmlns:dlna="urn:schemas-dlna-org:device-1-0">
   <specVersion>
      <major>1</major>
      <minor>0</minor>
   </specVersion>
   <device>
      <dlna:X_DLNADOC>DMR-1.50</dlna:X_DLNADOC>
      <deviceType>urn:schemas-upnp-org:device:MediaRenderer:1</deviceType>
      <friendlyName>DLNA renderer</friendlyName>
      <manufacturer>SeNSSoFT</manufacturer>
      <manufacturerURL>http://senssoft.com</manufacturerURL>
      <modelDescription>Simple DLNA Renderer for Windows PC</modelDescription>
      <modelName>DLNA renderer for Windows PC</modelName>
      <modelNumber>001</modelNumber>
      <UDN>uuid:13f3b1b2-1ddc-11b2-969a-000ca7061100</UDN>
      <serviceList>
         <service>
            <serviceType>urn:schemas-upnp-org:service:RenderingControl:1</serviceType>
            <serviceId>urn:upnp-org:serviceId:RenderingControl</serviceId>
            <SCPDURL>/dmr_rcs.xml</SCPDURL>
            <controlURL>control/RenderingControl</controlURL>
            <eventSubURL>event/RenderingControl</eventSubURL>
         </service>
      </serviceList>
      <iconList>
         <icon>
            <mimetype>image/jpeg</mimetype>
            <width>48</width>
            <height>48</height>
            <depth>24</depth>
            <url>/icon/small.jpg</url>
         </icon>
         <icon>
            <mimetype>image/png</mimetype>
            <width>48</width>
            <height>48</height>
            <depth>32</depth>
            <url>/icon/small.png</url>
         </icon>
         <icon>
            <mimetype>image/jpeg</mimetype>
            <width>120</width>
            <height>120</height>
            <depth>24</depth>
            <url>/icon/large.jpg</url>
         </icon>
         <icon>
            <mimetype>image/png</mimetype>
            <width>120</width>
            <height>120</height>
            <depth>32</depth>
            <url>/icon/large.png</url>
         </icon>
      </iconList>
   </device>
</root>

[Yortw]

Thanks for the extra detail. Unfortunately I was only able to spend a few minutes on this last night and haven't conclusively proved much, but here's what I found so far;

1. I can reproduce the issue where the device published by the sample app is not found by UPnP-Pentest-Toolkit
2. The commercial program I work on for my day job uses RSSDP to publish itself as a device, and that was found by UPnP-Pentest-Toolkit.

I suspect but have not proved the difference is the real app actually publishes a correct and matching DDD on the endpoint in the published device, whereas the sample app doesn't as choose an HTTP server framework etc. was beyond the scope of the sample.

When I get time I will see if I can update the sample app to publish the relevant DDD and see if that allows UPnP-Pentest-Toolkit to find it, or if there is another issue.

[Yortw]

By eye, I don't see anything wrong with your code or DDD, but it's hard to be sure. I assume the UDN value in the DDD matches your _config.Settings.DeviceUuid value at runtime.

I assume you can browse to the location of the DDD (runtime value of string.Format("http://{0}:{1}/device.xml", _localIP, _config.Settings.LocalPort)) and see the correct response using a web browser?

[sensboston]

Yes, I'm sending a correct XML response (btw, could you point me to the line in RSSDP code where I can see final XML content?) with the real IP address and port. My standalone renderer is functional now (I don't like built-in Hisense smart TV DLNA renderer, what's why I'm working on my own) but only my UWP app, "Cast to", can see this renderer now :(

The real good news is what you have working RSSDP program; probably we can figure out what's going wrong with my and demo code.

Thanks a lot for spending time for the issue, I'm really appreciate that!

[Yortw]

btw, could you point me to the line in RSSDP code where I can see final XML content?

You can call the ToDescriptionDocument method on an instance of the SsdpRootDevice class to generate the appropriate XML document. That DDD is generated from the properties on the device object being published, so should be correct assuming all properties are filled in correctly.

Note this only generates the XML, nothing ins RSSDP serves it. RSSDP is just about the publishing/search protocol, it doesn't include any kind of HTTP server and therefore can't actually make the DDD available over HTTP itself.

[sensboston]

I know, I know :) I do have my own HTTP server to handle all requests.

By the way, I have a bad news (at least for debugging): UPnP-Pentest-Toolkit is using Microsoft's standard UPnPDeviceFinder COM object from Windows UPnP library so we can't find what's wrong by debugging step by step in source code :(

[Yortw]

Cool, did that answer your question about how to get the xml?

[sensboston]

Yes. This is an xml file from my root device:

<?xml version="1.0" encoding="utf-8"?>
<root xmlns="urn:schemas-upnp-org:device-1-0">
  <specVersion>
    <major>1</major>
    <minor>0</minor>
  </specVersion>
  <URLBase>http://192.168.1.140:5000/</URLBase>
  <device>
    <deviceType>urn:schemas-upnp-org:device:urn:schemas-upnp-org:device:MediaRenderer:1:1</deviceType>
    <friendlyName>Test DLNA renderer</friendlyName>
    <manufacturer>SeNSSoFT</manufacturer>
    <manufacturerURL>http://senssoft.com/</manufacturerURL>
    <modelDescription>Simple DLNA Renderer for Windows PC</modelDescription>
    <modelName>DLNA renderer for Windows PC</modelName>
    <modelNumber>001</modelNumber>
    <UDN>uuid:8ccc9c76-a578-4ddb-bd9f-864d9b898448</UDN>
    <serviceList>
      <service>
        <serviceType>urn:schemas-upnp-org:service:urn:schemas-upnp-org:service:RenderingControl:1:1</serviceType>
        <serviceId>urn:upnp-org:serviceId:00962552-891c-453c-96fb-f3c14804e8f2</serviceId>
        <SCPDURL>dmr_avts.xml</SCPDURL>
        <controlURL>control/AVTransport</controlURL>
        <eventSubURL>event/AVTransport</eventSubURL>
      </service>
    </serviceList>
  </device>
</root>

and this one provided by HTTP request device.xml:

<?xml version="1.0" encoding="utf-8"?>
<root xmlns="urn:schemas-upnp-org:device-1-0" xmlns:dlna="urn:schemas-dlna-org:device-1-0">
  <specVersion>
    <major>1</major>
    <minor>0</minor>
  </specVersion>
  <device>
    <dlna:X_DLNADOC>DMR-1.50</dlna:X_DLNADOC>
    <deviceType>urn:schemas-upnp-org:device:MediaRenderer:1</deviceType>
    <friendlyName>Test DLNA renderer</friendlyName>
    <manufacturer>SeNSSoFT</manufacturer>
    <manufacturerURL>http://senssoft.com</manufacturerURL>
    <modelDescription>Simple DLNA Renderer for Windows PC</modelDescription>
    <modelName>DLNA renderer for Windows PC</modelName>
    <modelNumber>001</modelNumber>
    <UDN>uuid:8ccc9c76-a578-4ddb-bd9f-864d9b898448</UDN>
    <serviceList>
      <service>
        <serviceType>urn:schemas-upnp-org:service:RenderingControl:1</serviceType>
        <serviceId>urn:upnp-org:serviceId:RenderingControl</serviceId>
        <SCPDURL>/dmr_rcs.xml</SCPDURL>
        <controlURL>control/RenderingControl</controlURL>
        <eventSubURL>event/RenderingControl</eventSubURL
      </service>
    </serviceList>
    <iconList>
      <icon>
        <mimetype>image/jpeg</mimetype>
        <width>48</width>
        <height>48</height>
        <depth>24</depth>
        <url>/icon/small.jpg</url>
      </icon>
      <icon>
        <mimetype>image/png</mimetype>
        <width>48</width>
        <height>48</height>
        <depth>32</depth>
        <url>/icon/small.png</url>
      </icon>
      <icon>
        <mimetype>image/jpeg</mimetype>
        <width>120</width>
        <height>120</height>
        <depth>24</depth>
        <url>/icon/large.jpg</url>
      </icon>
      <icon>
        <mimetype>image/png</mimetype>
        <width>120</width>
        <height>120</height>
        <depth>32</depth>
        <url>/icon/large.png</url>
      </icon>
    </iconList>
  </device>
</root>

(both copy-pasted from run-time debugging). Could you take a look what's wrong?

I see, deviceType is different; in the announce xml it's urn:schemas-upnp-org:device:urn:schemas-upnp-org:device:MediaRenderer:1:1 (why 1.1 ?)

Also, icons are missing (probably, I need to add icons too).

[sensboston]

Some small update: it looks like only Hesense Smart TV requested device.xml file; no requests from the standard MS library (i.e. clients) are found at all.

So, looks like the real issue is in the device announcement; Microsoft's code doesn't likes that xml or (maybe) the way how it's announced.

Unfortunately, I need to get my renderer work with the MS standard UPnP stack :( As a temporarily solution, I can offer my renderer to my "Cast to" customers only but it's a weird (it should work with any DLNA client, not with mine only).

So, Troy, you're only one my hope!

[Yortw]

Try setting the SupportPnpRootDevice property to true and the StandardsMode property to Relaxed on the publisher. Some of the MS clients (particularly Windows Explorer) send invalid search requests (missing MX header, odd root device type) and so aren't responded to by default. Setting these items on should make the broadcasts compatible. My real app from work shows in Windows explorer with these properties configured this way (also mentioned in the wiki https://github.com/Yortw/RSSDP/wiki/Frequently-Asked-Questions).

Also your device type is being set incorrectly. The DeviceType property should just be the device type, so in this case MediaRenderer. You need to set the DeviceTypeNamespace property to schemas-upnp-org:device:urn:schemas-upnp-org and the DeviceTypeVersion property to 1 in order to get the correct device type in your xml output. RSSDP will build the full device type and add the urn: prefix itself where relevant.

[Yortw]

Hi,

There is a new branch ('m still working on it when I have time) called Feature/SampleAppToSupport3rdPartyTools.

This branch has no changes to the library itself, but does change the sample app so the sample device shows in the UPnP-Pentest-Toolkit app. While it will now show the device, it doesn't show services. I'm not sure why but I think it might be because the urls (particularly the SCPDUrl) for the service aren't real, the sample isn't intended to provide a real service, just show how to publish & search.

Interestingly about 50% of the device on my network have the same problem in the pentest app (won't show services, just says 'error' with no details). Because the pentest app uses native dll's to do all the real work, it's hard to know what the root cause is.

Really the only thing I had to do to make this work was to ensure the DDD was published via HTTP appropriately. This works for me both on a single machine and across machines on the same network, though in some cases I have had to disable the Windows firewall (just so the HTTP listener will server the DDD, the UDP search messages seem to work regardless).

So as far as I can see the publish should work fine so long as the DDD can be successfully browsed from the machine doing the search, at the end point provided in the device's location property. The services probably work if they are also correctly published & served.

As previously mentioned, your device type is currently incorrect in your DDD, and so is the service type. You need to set the individual properties (namespace, type, version) for device & service, not set DeviceType/ServiceType to the full string. That may also cause some clients to have issues.

[sensboston]

No luck at all, even with a new branch (I just re-target console app to .NET 4.5.1) :(

P.S. Troy, why some of your UPnP devices (I expecting, all these devices or programs are using RSSDP, right?) listed 3 times? I saw (and disabled in RSSDP source code during my previous work with Renderer) that behavior and pretty sure it's definitely not right. Neither UPnP devices nor third-party programs with proper UPnP implementation doesn't announces like this.

[sensboston]

Could you please send me a binary version (but please change it to my fixed local IP address - I do have multiple network cards installed on development PC), like _DevicePublisher = new SsdpDevicePublisher(new SsdpCommunicationsServer(new SocketFactory("192.168.1.140")));

P.S. I've checked MS implementation but in MS case I need to create a separate dll and register it as a COM (to get device published), too much work. Your solution (if I can get it works) much much more useful...

[sensboston]

Troy, I checked changes what you've made for the Feature/SampleAppToSupport3rdPartyTools: this will not work, the issue isn't in http server and xml delivery!

As I already wrote above, after publishing I got requests for device.xml from Hisense Smart TV only (probably, Chinese code!) but not from the UPnP-Pentest-Toolkit, or Microsoft UPnP stack (btw, UPnP-Pentest-Toolkit uses MS library), not from Windows 10 mobile standard DLNA finder, not from Android's BubbleUPnP app (which one also can perfectly see all my UPnP DLNA devices).

So, looks like your registration request has some issues. Probably, we need a WireShark to capture all standard announcements and made by RSSDP, and check what is the difference is. I'm definitely not an expert in UPnP as you, but I'd like to help you, if you tell me how and what do you need!

BTW, I've tried Microsoft sample: https://github.com/Microsoft/Windows-classic-samples/tree/master/Samples/UPnPRegisterDevice and https://github.com/Microsoft/Windows-classic-samples/tree/master/Samples/UPnPDimmerService (you need both); that demo device is registered fine and visible by all UPnP software. I also tried to change xml files to MediaRenderer service and it's visible by all software what I tried. The only reason why I don't like MS way 'cause it's complicated, messy and requires additional steps to get it works (registration of COM object, special permissions or "Run as administrator"). It's a real mess, and I don't like to go this way. Your RSSDP is simple, straightforward and works fine for my purpose. The only issue (but the HUGE one) - my renderer will be visible for my W10M app only but I want to make it working for anything (and planning to publish it here, on GitHub).

[sensboston]

OK, I captured SSDP notify packets from MS demo device (visible), and from my renderer (RSSDP, invisible) by WireShark. Here is the archive with the capture: https://www.dropbox.com/s/3evlhjhtffatkng/pcapng.zip?dl=0

Could you please take a look? I'll also will try to figure out the difference between these packets.

UPDATE: The first two big differences I've found, it's an order of sent packets and EOL (MS uses \r\n, you \n only): Microsoft sending packets by this order: NT: upnp:rootdevice\r\n NT: uuid:5b39368e-f49d-40bb-8a1e-441ae6004c42\r\n NT: urn:microsoft-com:device:DimmerDevice:1\r\n NT: urn:microsoft-com:service:DimmerService:1\r\n and repeat all these packet 3 times

RSSDP sending: NT: upnp:rootdevice\n - 3 times NT: pnp:rootdevice\n - 3 times NT: uuid:8ccc9c76-a578-4ddb-bd9f-864d9b898449\n - 3 times NT: urn:schemas-upnp-org:device:MediaRenderer:1\n - 3 times NT: urn:AVTransport:service:AVTransport:1\n - 3 times

Troy, you know your code much better than me, could you help me with modifications?

UPDATE 2: changing EOL '\n' to '\r\n' didn't helped much...

UPDATE 3: changing packets order - also no results :( Looks like we need take a closer look inside packets data.

[Yortw]

why some of your UPnP devices (I expecting, all these devices or programs are using RSSDP, right?) listed 3 times?

First of all, no, not all of these are using RSSDP. The only device in that screenshot that uses RSSDP is the RSSDP Sample Device. The others are all consumer electronics. The devices shown multiple times all happen to by Sony equipment (a bluray player, a home theatre system, and a TV). I do not know why Sony choose to publish three (or more) different root devices for every physical device instead of one root device with multiple embedded devices, but they do. Note these are not actually duplicates. They look like it because they all have the same friendly name, but each device in the list has a slightly different DDD url and device type, and obviously the DDD for each is different.

I saw (and disabled in RSSDP source code during my previous work with Renderer) that behavior and pretty sure it's definitely not right

Actually it is part of the SSDP spec and any correct and robust implementation should do this. It also has nothing to do with the devices being listed multiple times. I can't find the original document I worked from which I think had the best explanation, but you can see it referenced in;

• Page 15 of http://www.upnp.org/specs/arch/UPnP-arch-DeviceArchitecture-v1.0-20080424.pdf

Due to the unreliable nature of UDP, devices should send each of the above discovery messages more than once, although to avoid network congestion discovery messages should not be sent more than three times

• Section 6.3.1 of https://tools.ietf.org/html/draft-cai-ssdp-v1-03 says;

((DR3 + DR9RS)AM)/TP

The 3 is the number of times the ssdp:discover request will be repeated. The 9 is the number of times the unicast responses to the ssdp:discover requests will be sent out assuming the worst case in which all 3 original requests are received.

Basically the UDP protocol does not guarantee delivery of packets, so to avoid dropped packets causing lost results the SSDP specification says each message should be sent 3 times.

If you look at the code that receives and processes m-search requests you'll see that it "de-dupes" the incoming requests. If multiple search requests from the same source, with the same search target, arrive within a few milliseconds of each other, only the first is processed. This helps reduced the network load as discussed in section 6.3.1 of the document above. I think that was actually advised/suggested by the original specification document I worked from, but I can't find it now.

[Yortw]

Could you please send me a binary version (but please change it to my fixed local IP address

I modified the app to use your IP, you can download the binaries here; https://1drv.ms/u/s!ArC3Ud4fUz-MgfqrPzvwHc1lHschy7w

[Yortw]

Troy, I checked changes what you've made for the Feature/SampleAppToSupport3rdPartyTools: this will not work, the issue isn't in http server and xml delivery!

As I said when I posted that I'd updated the app, I did not change the library. That was not required to solve your original problem (at least on my networks).

Your original problem stated the device from the sample app didn't show in the UPnP Pentest Toolkit. I tested it, found that it didn't but was found by other tools (Intel tools, Windows Explorer, RSSDP itself, and the other SSDP library you listed, by Kakone). The only difference was the sample app didn't actually publish it's DDD over HTTP. I made those changes, and as the screenshot I posted shows, the sample device does now show in the UPnP Pentest Toolkit. So this did help, and did solve your original problem, at least on my home & work networks. You may also have another issue, but your original problem as posted was never going to be solved without this mod.

As I already wrote above, after publishing I got requests for device.xml from Hisense Smart TV only (probably, Chinese code!) but not from the UPnP-Pentest-Toolkit, or Microsoft UPnP stack (btw, UPnP-Pentest-Toolkit uses MS library), not from Windows 10 mobile standard DLNA finder, not from Android's BubbleUPnP app (which one also can perfectly see all my UPnP DLNA devices).

I didn't fully understand what you meant by this, and I'm still at a loss to explain it. UPnP-PenTest_toolkit and other tools I have here all work for with the sample app now the DDD is published. I don't know why it's not working for you. It seems unlikely there is a problem with the packet structure/format, given it works for me with (I assume) the same library code, and multiple tools are all able to decode it.

It could be there is a problem with the content of the notification that is causing an issue, but I can't see that. The most likely culprit in this case would be the device type - did you fix the device type as I previously suggested?

• So, looks like your registration request has some issues. Given I don't know what the problem is I guess this is possible, but the packets I'm seeing all look fine, and again all the third party test tools I have are receiving and able to parse the packets fine. I'm not sure what could be wrong.

Probably, we need a WireShark to capture all standard announcements and made by RSSDP, and check what is the difference is. I'm definitely not an expert in UPnP as you, but I'd like to help you, if you tell me how and what do you need!

If you can send me the wireshark packets captured on your network I can try to take a look. I'm not familiar with wireshark, or low level packet captures, but I'll do what I can.

[sensboston]

Troy, just forget about dups, it's not an issue at all (at least we know how to prevent that behavior). The main issue (I believe, you are interested too 'cause your library is unique, one of a kind) is to make your library be compatible with MS UPnP stack. Fortunately or unfortunately, we are living in the "windows world"; you can't pretend to be "compatible" and "follow the specs" if your announcements are invisible by standard UPnP calls.

So, from my experiments: I changed SSDP message format to be compatible 100% with MS - no luck. Tomorrow (or this Friday 'cause its 4:37 a.m. EST - I have a sleepless night today) I'll try to change an answers and see how it goes.

Another difference what I've found, it's a port number for NOTIFY: you are using 1900 for src and dst Microsoft using some high number (56xxx) for src.

P.S. BTW, unfortunately your built crashes on Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Threading.Tasks, Version=1.0.12.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. I recommend you that extension especially for that kind of work (or any other).

[sensboston]

If you can send me the wireshark packets captured on your network I can try to take a look. I'm not familiar with wireshark, or low level packet captures, but I'll do what I can.

Already did, check my post above :) Or just download that archive: https://www.dropbox.com/s/3evlhjhtffatkng/pcapng.zip?dl=0

[sensboston]

BTW, WireShark is a great tool, you can see, literally, anything happened on your network. I believe, we'll fix that issue very soon (and your library become a super-duper unique tool of all the time for UPnP :D )

P.S. 'cause Microsoft's implementation is too ugly and complicated! ;)

[Yortw]

just forget about dups, it's not an issue at all Agree, but since you asked the question and I knew the answer, I was happy to explain.

you can't pretend to be "compatible" and "follow the specs" if your announcements are invisible by standard UPnP calls.

Also agreed, and I am doing what I can to help. The thing is, I am not seeing any incompatibility. On my network, devices published by RSSDP are found by 4 different pieces of software on my PC's (including the UPnP Pentest toolkit you originally referenced and Windows Explorer), as well as my TVS's/hometheatre/bluray player etc. If there was an obvious problem with the packet format being incompatible I should be seeing it, but I am not. It's a matter of determining what is different at your end. I'm not saying no problem exists, I'm saying I can't see what it is. It's not like it never works.

The line endings thing is interesting. I think there was a mod made in the past to deal with that on different platforms. I think originally it was \r\n but some Linux systems didn't cope with that and Windows didn't care. I can't recall if it was changed to always be \n, or if it's supposed to use the platform specific line ending. I'll have to try and search the history/past issue to remember what we did and why.

[sensboston]

Please check my post(s) above: I already tried different line ending, the same packets send order, and very same (exact content) packets for NOTIFY - no luck. Will try tomorrow (or today) morning/noon modify the answers. Also will try to modify src UDP port number (change from 1900 to 45xxx).

Please check WireShark, it's a really great tool! I'm not a big expert in WireShark or UPnP or network protocols, but this is a not a rocket science but just a very regular network communication. I'm 99.99% sure, the issue is in the very small but annoying thing. We should figure out how to fix it. And, I believe, many people will be happy, not only you and me :)

P.S. I'm trying to help myself, you and all github customers. I believe, you're also interested in that...

[Yortw]

Things you could confirm/try to help.

1. Did you fix the device type? Can you post the new xml for the DDD?

2. If you run the 'Device Sniffer' app from Intel UPnP tools, then publish a device using RSSDP, does that the device sniffer see/understand the packets? From memory device sniffer only cares about the SSDP packets and doesn't care whether the DDD can be retrieved.

3. Can you confirm you've enabled relaxed standards mode on the publisher?

4. From a different PC, can you confirm you can browse to the url specified by the Location property on the published device, and that doing so returns the expected xml without any kind of login required etc.

5. Does the Intel 'Device Spy' utility show the device? That utility requires both the notification packets and the DDD document to work correctly.

[Yortw]

I am working through your posts one at a time.

Feel free to try changing the port, but I don't think it will help. 1900 is required to be the port that notifications and search requests are sent from. Using other ports as the source/listening port is fine, but it's a very common (and recommended) patter to use 1900 there too so you get the notifications as well.

Also the library code I'm using on my home & work networks is using port 1900 everywhere, and tools like UPnP Pentest have no problem seeing the devices in my environments. If the port was the problem I would not expect it to work here (unless there is a firewall issue or something?).

[Yortw]

Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Threading.Tasks,

Do you have .Net 4.5.1 installed? The app runs for me and I believe all the required dll's are in the zip I created.

[Yortw]

Are you able to send me your project or a link to it? Maybe I could build and test your project locally to see if it works for me.

[sensboston]

OK, I'll try to summarize my tries (sorry, no straight answers to your questions but you'll understand why).

1. I tried Microsoft UPnP sample (check links above): it works 100% correctly but MS way is ugly and hard to follow (I already spent time to great C# WPF renderer, and don't like to switch to C++ way)

2. RSSDP is invisible (whatever I tried) on my LAN (btw, I do have ASUS RT-N66U router with the latest firmware) for Microsoft's UPnP stack, and for Android apps.

3. I'm still 99.9% sure, the issue is in message format or something (small) else.

[sensboston]

Are you able to send me your project or a link to it?

Yes, I do. Do you have Skype/IM? My skype id is sens_boston

P.S. I don't like to share unfinished projects for general public (will do for you via IM/Skype)...

[Yortw]

Sorry, I ran out of time last night and I'm about to go out again shortly this morning.

I compared the Wireshark packets you sent me. I don't see any significant/important differences. Things I noted;

• The line endings (already agreed not the problem)
• Header casing - RSSDP uses all upper cases headers, the MS packet had a mixture of all uppercase and mixed-case. The RFC spec for HTTP headers specifically says they should be treated in a case insensitive way, but I have had issues with some poorly behaved Linux devices requiring all uppercase. Since the MS stuff uses a mix itself, I doubt this is an issue.
• The MS packet had two headers in it that aren't in the RSSDP packet. These both look like optional or custom headers and shouldn't cause an issue. RSSDP included the DATE header and the MS packet didn't. I actually thought that was a required header according to the HTTP RFC spec, but either way the fact that RSSDP sends it and MS doesn't should cause them to fail to parse the RSSDP packet.

Otherwise the general structure/formatting of the packets seems the same, and the content varies in expected ways. If there is something odd in the data itself I haven't yet been able to spot what it is. Unfortunately Wireshark is now crashing on startup so I can't open the packet captures to review them again. The symptom I'm getting has been reported lots on the Wireshark support forums/pages, but none of the suggested solutions have worked for me so far. Odd, because it worked last night (I guess rebooting did something).

I tried building the c++ MS sample you linked to above, but I only have community edition of VS for my open source work and that doesn't include the required MFC headers. If you could send me the compiled binaries for the MS sample that does the searching then I could try that and see if it works on my network here. If it doesn't then maybe I can investigate further.

I tried using the extension you linked to, to rebuild the customised sample app. You can download that here; https://1drv.ms/u/s!ArC3Ud4fUz-MgfqrQG5blCQYpdohzdc

I also tested the RSSDP sample app and the UPnP Pentest Toolkit on my parents home network briefly last night and everything worked fine there too (pentest could find the sample rssdp device) , so I still haven't been able to reproduce the issue (since I added the http server to the sample app).

[sensboston]

No worries, take your time! I'm also busy with my daily job, the only time for my fun projects and homebrew apps is a late evening or midnight... ;)

OK, let's go step by step:

• as for WireShark, I'm really surprised - I never experienced crash of this app (I'm not an expert in WireShark but using it for a years). Probably, you have something messy with your hardware or drivers. Also, when you'll have a time, check your PC for troyans or rootkits (I recommend you products from Malwarebytes company, the have a very good free rootkit detector).

• as for MS sample build: here we go https://www.dropbox.com/s/k77iq8ywt4piuzb/Microsoft_UPnP_Sample_Release.zip?dl=0 To get things running, you need to register dll first. Start (as Administrator) command prompt and run command: regsvr32 UPNPSampleDimmerDevice.dll After that you can run RegDevice.exe. As I said before, UPnP devices registered by this way, are visible by any UPnP browser (or, in my case, DLNA renderer browsers including standard UPnP stack for UWP apps - I tried to substitute xml files used by MS sample by mine, from renderer, and it was shown everywhere).

• as for your sample: sorry, it's crashed after device publishing :( Here is screenshot with the stack trace

P.S. Troy, could you authorize me on Skype (I sent you authorization request this night)? I'd like to send you my Renderer project (but don't want to publish it here, at least for now until it's not finished yet)

Also, could you please explain how UPnP announcement works, who keeps these UDP packets alive on the LAN? Probably, router?

[sensboston]

By the way, I still believe that WireShark (or if it's crashes for you, maybe another LAN packet sniffer) and Microsoft sample are keypoints to resolve the issue. 'Cause MS stuff is 100% working and visible, we need to figure out the difference between packets (and whole protocol packet's exchange) on the network level.

[Yortw]

Hi,

I have accepted your connection request on Skype, so if you can send me your project I'll try that.

Do you have the .Net Framework 4.5.1 installed? System.Core is part of the framework, and as that app targets 4.5.1 it is looking for that version. You may need to install or repair that version of the framework if you want to run the sample app I built.

The dimmer sample works fine for me, but I see no relevant difference in it's packets over RSSDP. I was hoping to get binaries for the UPnPGenericUCP to see if running that locally was able to discover RSSDP devices and decipher it's packets.

I don't know how much you know about SSDP/UDP etc, so sorry if some of this seems patronising (also I'm not a network expert, so take this explanation with a grain of salt).

UPnP announcements are UDP packets, containing HTTP protocol content. The packets are not 'kept alive' on the network. The packets are sent to a 'multicast address'. A multicast address is a special address which devices can subscribe to, all packets sent to the multicast address are received by all devices that subscribed, i.e. the address doesn't represent a single device, it represents a possible group of devices. There is a specific address assigned for use by SSDP (one for Ipv4 and another for Ipv6 networks).

So when something sends an SSDP notification every device subscribed to the group should get the message. However, UDP itself is not a 'reliable' protocol. Unlike TCP/IP, UDP does not include checksums/retries and other reliability features. If a packet goes missing, then it goes missing (and this is the reason notifications are sent multiple times in the SSDP spec). Any device not subscribed when a packet is sent won't see it, even if it subscribes shortly after. There is no 'keeping alive'.

The notifications are broadcast periodically. The SSDP spec specifies an algorithm for this, but it's been a while since I read it. From memory it is a randomly generated interval up to 1/2 of the cache interval allowed for the device DDD. I believe another part of the spec says the cache interval should not be less than 1/2 an hour (though frequently that part of the spec is ignored by actual implementations). So if I've remembered and interpreted correctly, the notifications should be sent on random intervals of up to 15 minutes (but often it is much less than that, as devices don't follow the spec well).

Of course that isn't good enough for many situations. Clients start/restart at different times and having to wait up to 15 minutes to get a complete set of devices isn't good. This is why the search function exists.

The search function also sends a UDP packet to the same multicast address used for notifications. Devices see that it's a search request and respond (if appropriate, the search request can be by device type/id etc). The responses are sent directly back to the requesting client, not sent as broadcasts.

The protocol is designed to be used thusly; on start-up perform a search to get a list of current devices. Listen for notifications, and update your list as you receive alive (new device/device still available) and byebye (device has gone offline) notifications. Repeat search should be unnecessary, unless you restart or lose and regain network connectivity etc.

[sensboston]

Do you have the .Net Framework 4.5.1 installed? System.Core is part of the framework, and as that app targets 4.5.1 it is looking for that version. You may need to install or repair that version of the framework if you want to run the sample app I built.

Of course I do:

and it works fine for me. Probably, it's something else (maybe, portable libraries you've used). BTW, this error happened when I pushed "d" (publish device) only, otherwise app runs fine.

[Yortw]

Hi,

So your project works fine for me, with two mods. Here's a screenshot of UPnP Pentest Toolkit showing your test renderer;

The two mods I made are;

1. Undoing your changes to the RSSDP library, specifically in the SendDeviceSearchResponses method in the SsdpDevicePublisherBase class. You added some conditional compilation which takes out two kinds of search responses. One of those responses is the one for the upnp root device requests - this is exactly the request that UPnP Pentest Toolkit sends. Since you've removed it, no response is sent to the pentest tool kit, and it never sees the device. If you change;

        private void SendDeviceSearchResponses(SsdpDevice device, UdpEndPoint endPoint)
        {
// SeNS::
#if false
            bool isRootDevice = (device as SsdpRootDevice) != null;
            if (isRootDevice)
            {
                SendSearchResponse(SsdpConstants.UpnpDeviceTypeRootDevice, device, GetUsn(device.Udn, SsdpConstants.UpnpDeviceTypeRootDevice), endPoint);
                if (IsWindowsExplorerSupportEnabled)
                    SendSearchResponse(SsdpConstants.PnpDeviceTypeRootDevice, device, GetUsn(device.Udn, SsdpConstants.PnpDeviceTypeRootDevice), endPoint);
            }

            SendSearchResponse(device.Udn, device, device.Udn, endPoint);
#endif

            SendSearchResponse(device.FullDeviceType, device, GetUsn(device.Udn, device.FullDeviceType), endPoint);
        }

to

        private void SendDeviceSearchResponses(SsdpDevice device, UdpEndPoint endPoint)
        {
            bool isRootDevice = (device as SsdpRootDevice) != null;
            if (isRootDevice)
            {
                SendSearchResponse(SsdpConstants.UpnpDeviceTypeRootDevice, device, GetUsn(device.Udn, SsdpConstants.UpnpDeviceTypeRootDevice), endPoint);
                if (IsWindowsExplorerSupportEnabled)
                    SendSearchResponse(SsdpConstants.PnpDeviceTypeRootDevice, device, GetUsn(device.Udn, SsdpConstants.PnpDeviceTypeRootDevice), endPoint);
            }

            SendSearchResponse(device.Udn, device, device.Udn, endPoint);

            SendSearchResponse(device.FullDeviceType, device, GetUsn(device.Udn, device.FullDeviceType), endPoint);
        }

Then the pen test toolkit finds it no problem. I recommend removing all your mods/going back to the published library, as that woks fine for me an no mods should be necessary. I'm not sure what other changes you've made and if they might cause issues.

The second mod I made makes your project work with the Intel Device Spy utility. The Device Spy utility also wouldn't find the device, but it has nothing to do with the SSDP packets being sent. Even if you choose the 'Manually Add Device' option (from the file menu) and point it to your DDD URL, it gives an error saying it can't add the device. Debugging the Device Spy source, it turns out it fails to add the device because it can't parse the HTTP response sent from your server. Specifically it states there is a problem with the status line (which is the first line of the response). Note, the parsing is in a compiled library so I couldn't see exactly why it failed, but I noticed your HTTP responses only contain \n for the line endings. If you check the RFC specification for HTTP (http://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html#sec6.1) you'll find it specifically says the line endings must be CRLF. It appears that most web browsers (and probably other HTTP implementations) will cope with either \n or \r\n but the one used by Device Spy enforces the spec strictly. After changing all of your HTTP packets to use \r\n as the line ending then the Device Spy utility finds the renderer no problem, even without using the manual add option. For maximum compatibility I would make sure you use \r\n line endings in your HTTP requests and responses.

[Yortw]

I should note that the two search responses your conditional compilation mod took out are specifically required by the spec in response to a search for root devices. So removing them is definitely incorrect and likely to cause problems. A single search request often results in multiple responses (with slightly different content) for the same device, that's part of SSDP and shouldn't be considered odd or a bug.

[sensboston]

:( It looks like we've got back to the beginning of discussion.

First, 4 search responses from RSSDP produces 4 different devices (take a look to the screenshot):

Probably, it's something wrong with the responses; normal DLNA renderers are not appearing this way.

Second, on my network RSSDP aren't visible by UPnP Pentest Toolkit, Microsoft UPnP stack, Android apps etc.

I'll try to figure out, why https://github.com/kakone/SSDP code (I'm using in my UWP project) thinks it's four devices instead of one (like normal UPnP devices should appear).

[sensboston]

OK, found first bug (probably) in RSSDP code: SSDP (mentioned above) is using "ST: " record in search (file Ssdp.cs, line 75) specific devices (i.e. renderers). RSSDP is IGNORING that search type, and sending all records (see debug output below).

Location:  http://192.168.1.33:2870/dmr.xml   USN:  uuid:13f3b1b2-1ddc-11b2-969c-000ce7060000::urn:schemas-upnp-org:device:MediaRenderer:1
Location:  http://192.168.1.33:2870/dmr.xml   USN:  uuid:13f3b1b2-1ddc-11b2-969c-000ce7060000::urn:schemas-upnp-org:device:MediaRenderer:1
Location:  http://192.168.1.33:2870/dmr.xml   USN:  uuid:13f3b1b2-1ddc-11b2-969c-000ce7060000::urn:schemas-upnp-org:device:MediaRenderer:1
Location: http://192.168.1.140:5000/device.xml   USN: uuid:8ccc9c76-a578-4ddb-bd9f-864d9b898449::upnp:rootdevice
Location: http://192.168.1.140:5000/device.xml   USN: uuid:8ccc9c76-a578-4ddb-bd9f-864d9b898449::pnp:rootdevice
Location: http://192.168.1.140:5000/device.xml   USN: uuid:8ccc9c76-a578-4ddb-bd9f-864d9b898449
Location: http://192.168.1.140:5000/device.xml   USN: uuid:8ccc9c76-a578-4ddb-bd9f-864d9b898449::urn:schemas-upnp-org:device:MediaRenderer:1
Location: http://192.168.1.100:6000/device.xml   USN: uuid:681c1972-c9c3-423c-90e1-73be6b79f081::urn:schemas-upnp-org:device:MediaRenderer:1
Location: http://192.168.1.100:6000/device.xml   USN: uuid:681c1972-c9c3-423c-90e1-73be6b79f081::urn:schemas-upnp-org:device:MediaRenderer:1

And here is SSDP code for search:

        private async Task<IEnumerable<string>> SearchDevicesAsync(IPAddress localAddress, string deviceType)
        {
            var responses = new List<string>();

            var addressType = (localAddress.AddressFamily == AddressFamily.InterNetwork ? AddressType.IPv4 :
                localAddress.IsIPv6LinkLocal ? AddressType.IPv6LinkLocal :
                localAddress.IsIPv6SiteLocal ? AddressType.IPv6SiteLocal : AddressType.Unknown);
            if (addressType != AddressType.Unknown)
            {
                try
                {
                    using (var socket = new Socket(localAddress.AddressFamily, SocketType.Dgram, ProtocolType.Udp))
                    {
                        socket.ExclusiveAddressUse = true;
                        if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
                        {
                            socket.IOControl(SIO_UDP_CONNRESET, new[] { Convert.ToByte(false) }, null);
                        }
                        socket.Bind(new IPEndPoint(localAddress, 0));

                        var multicastEndPoint = new IPEndPoint(IPAddress.Parse(MulticastAddresses[addressType]), UPNP_MULTICAST_PORT);
                        var req = "M-SEARCH * HTTP/1.1\r\n" +
                            $"HOST: {multicastEndPoint}\r\n" +
                            $"ST: {deviceType}\r\n" +
                            "MAN: \"ssdp:discover\"\r\n" +
                            "MX: 3\r\n\r\n";
                        var data = new ArraySegment<byte>(Encoding.UTF8.GetBytes(req));
                        for (int i = 0; i < 3; i++)
                        {
                            await socket.SendToAsync(data, SocketFlags.None, multicastEndPoint);
                        }
                        await ReceiveAsync(socket, new ArraySegment<byte>(new byte[4096]), responses).TimeoutAfter(RECEIVE_TIMEOUT);
                    }
                }
                catch (TimeoutException) { }
                catch (ObjectDisposedException) { }
            }

            return responses;
        }

[sensboston]

OK, here is the fix (to resolve incorrect search responses and multiple device appearing): In SsdpDevicePublisherBase.cs change if (searchTarget.Contains(":service:")) to if (searchTarget.Contains(":service:") || searchTarget.Contains(":device:"))

Update: Good news, this that change RSSDP device (my Renderer) become visible in UPnP Pentest Toolkit!

but sill has some errors by "expanding" services (I'll work on it).

[sensboston]

So, after some experiments, I've got partial success: after changing my socket-based server for handling http requests to HttpListener (I don't like it because extra steps needed to get it work but it's not a big issue), Renderer become visible and services available for UPnP Pentest Toolkit.

Troy, the only one change I want to ask you to merge with master branch (and release new NuGet package - I wanna get rid from all custom sources) - this very simple patch: device_dup.patch.txt

Other changes I've made to RSSDP doesn't affect at all; sorry for this huge thread! BTW, I believe my find will be useful.

My next challenge is making "Renderer" visible by Microsoft's Media.Casting API. I found that TV set announced another UPnP device, will work on that and share my experience here. Maybe, in future, it will be nice add-on to the project's Wiki page.

[Yortw]

Thanks. Working late tonight but with luck I might have time to look tomorrow evening.

[sensboston]

Another very little change request: could you please put services before icons in the root device xml?

[Yortw]

Hi,

Any particular reason for that? The order of XML nodes isn't supposed to matter (and I'm 99% the XML spec even says you shouldn't rely on a parser to enumerate nodes in the order they appear in the document). If you've got a device that relies on a specific order that could be a problem because for all we know there could be someone else out there already using the library with a device that requires a different order.

[sensboston]

Sorry, no real reason (and I know, order doesn't matter). It's just for me: for easier comparsion responses from my TV and RSSDP :)

BTW, I'm sorry - just scratch it! (but not the patch for devices!)

[Yortw]

Sorry, still working on this. Your proposed fix solves your issue but there are other variations from the spec that need to be addressed. Also both your change and mine break at least 11 tests (probably because they are asserting the incorrect behaviour and the tests need fixing). I won't finish tonight.

[sensboston]

I'm really sorry about this! But the fix I've proposed to you, just follows UPnP 1.1 specs, nothing more... (check pages 20, 23). You can't return any device by search request for "MediaRenderer", it's a bug (according to the specs).

By the way, it's definitely not an urgent. Take your time, don't hurry app. Your library is great and very unique, I really love it!

[Yortw]

Hi,

Yes, your change does bring the library into line with the spec when searching for device type. However on re-reading the spec I discovered there are similar issues when searching for ssdp:all or for root devices. If you compare the code to page 21(specifically under the USN section) of the spec you mentioned you'll see the differences. It's these issues I need to work through before republishing.

I'll let you know when I'm done.

Thanks.

[sensboston]

Agree. And one more time: thanks for the good library, and take your time.

[Yortw]

Hi,

I've published a pre-release package, version 3.5.6-beta (https://www.nuget.org/packages/Rssdp/3.5.6-beta).

If you can confirm this works for you then I'll merge the changes and publish an official update.

Thanks.