mDNS resolution not working for iOS devices with app built with Xcode 12.5

[mduchev]

mDNS resolution is always returning 0 results and throwing an UnobservedTaskException silently for iOS devices (w/ iOS 14.5), when the app is built with Xcode 12.5. This happens on both local debugging session and on production released version. The exception message is: Cannot access a disposed object. Object name: 'System.Net.Sockets.UdpClient'.

Everything was/is working fine before the app was built with Xcode 12.5

I'm attaching a sample project that can be run. In the iOS project, there's a method that catches unobserved exceptions and it is logging the inner messages.

Please note that the app must be run at least once, so that the new iOS 14 local network scanning permissions can be granted. Also, if the scan is being run only once, there are no results again but there is no error message. The silent exception is appearing in case we have consequitive scans.

Devices info: Tested on both iPhone X w/ iOS 14.5 & iPhone 12 Max Pro w/ iOS 14.5

Demo project: TestMdns.zip

[cosinekitty]

Without my deeply investigating, #190 is likely to be a fix for this issue. Perhaps you could try that code change and let us know if the problem goes away.

[mduchev]

@cosinekitty This surely looks like it should fix the problem. This doesn't seem to be related to the new Xcode version, but the strange thing is that I started experiencing it as soon as Xcode updated. Nevertheless, I'll come back as soon as I'm able to test the fix, in case a new version isn't released in the meantime.

[SpencerBurgess]

@cosinekitty This surely looks like it should fix the problem. This doesn't seem to be related to the new Xcode version, but the strange thing is that I started experiencing it as soon as Xcode updated. Nevertheless, I'll come back as soon as I'm able to test the fix, in case a new version isn't released in the meantime.

I got a reply on my PR from Claire, there is a little more to the issue than I originally thought, I will also debug a little more to see what we can do to come up with a satisfactory solution. I'm using this in two production environments and it is causing a pain point for our iOS users. I will try and collect a few more stack traces from our logging to see if we can't narrow the issue.

Oh, I am also getting the error with an app built using Xcode 12.4

[mduchev]

Unfortunately, I can also confirm that the PR doesn't fix the issues.

[mduchev]

I've done some additional tests and here's what I found out:

I tried building with both Xamarin.iOS 14.16.0.5 (latest) & 14.14.2.5. a) Using Xcode 12.5 - always 0 results with the exception message b) Using Xcode 12.4 - scan successful These results were happening for both Xamarin.iOS versions, so I can safely conclude that it doesn't have to do anything with the Xamarin.iOS versions. But, like I said, I'm only getting these issues with Xcode 12.5.

IDE: Visual Studio for macOS - version 8.9.8 (build 7) Link behavior: Link Framework SDKs Only Test devices - all were with latest iOS (14.5). Can't speak for the previous versions, but it should happen to them as well, I think.

For now, I can stick with building with Xcode 12.4, but at some point it won't be supported by both Xamarin & macOS updates, so this is not a permanent solution.

Unfortunately, I cannot take a deeper look at the issue/code right now. I'll have to dig later, when I have the time. I just hope that someone will manage to find a solution it in the meantine.

[rcinge]

I believe these documents describe what's causing the problem, and why it suddenly appeared:

How to do UDP broadcast transmission by Wi-Fi communication on iOS14 beta6? https://developer.apple.com/forums/thread/658518?answerId=631476022

From the post: "iOS 14 beta added a feature called local network privacy. [...] Note that while the latter has a focus on the new NWConnectionGroup API, local network privacy applies equally to the legacy BSD Sockets API."

Zeroconf uses the BSD Sockets API, so I believe that's why the problem appeared... here's another Apple document describing how to adapt to the new local network privacy:

How to use multicast networking in your app https://developer.apple.com/news/?id=0oi77447

I am requesting the com.apple.developer.networking.multicast entitlement for an app I am building.

I have already tried to add the appropriate entries to Info.plist without the entitlement and the Zeroconf operation still failed (which was pretty much expected). I will post back if the multicast entitlement plus the Info.plist entries makes things work.

(I have no affiliation with the Zeroconf developers, so whatever they say supercedes anything I say)

[clairernovotny]

@rcinge if that works, would you mind a PR to the readme that documents that requirement for iOS?

[rcinge]

@clairernovotny I will, if it does; still waiting for a response from Apple.

[mduchev]

@rcinge I'd really like for this to be the solution, alas I don't think that this is the fix. These links point to the new changes introduced in iOS 14.0 (the initial release). Everything was working fine untill I had to build with Xcode 12.5, at least for me. Now, if I build my app with Xcode 12.4 and release/deploy it to device with iOS 14.5.1 (latest) everything will work fine (even without the said Entitlements). According to the documentation from one of the links that you've provided, the multicast entitlements is for:

Do you need to support legacy devices? Maintaining compatibility with some legacy devices and software might require the use of custom multicast and broadcast protocols. Since these capabilities give your app complete access to the user’s local network, such access requires the com.apple.developer.networking.multicast restricted entitlement.

At least according to it, it the most common cases, it shouldn't be needed. Another reference from someone else (incl. response from Apple): https://stackoverflow.com/a/65589011/4090219

Response form rep: Thanks for your interest in multicast networking. It sounds like you’re able to use Bonjour / Multicast DNS, which doesn’t require any entitlement, but only requires a change to your Info.plist. If you have further questions about how to use the APIs, please follow up on the developer forums.

[rcinge]

@mduchev @clairernovotny Here's Apple's response to me:

Thank you for your interest in Multicast Networking. Since Bonjour is supported natively on the system, and does not require an entitlement, we’d ask that you adopt one of the system APIs that provides Bonjour functionality.

Best regards, Apple Networking

After thinking about this, this is my guess/story about what is going on:

The reason this issue appeared with Xcode 12.5 is that the new multicast restriction policy is implemented in Apple-shipped library code (the NWConnectionGroup API), and an updated version of that library would ship with the new Xcode. Likewise, the library wrappers for all system calls would also ship with the new release of Xcode.

The core bit: there's some undocumented socket option or system call argument that signals to the macOS kernel that multicast traffic is allowed on this specific socket. The NWConnectionGroup API knows this secret and will enable multicast traffic on a socket if: (1) the desired mDNS protocols and description are coded in Info.plist (2) the protocols specified do not violate Apple's restrictions (3) the device user consents to allow the app to examine network traffic. Finally, as results from the multicast query are returned by the Receive() system call, the NWConnectionGroup API then gets to filter the data it returns to the caller.

The legacy BSD Sockets system call wrappers also know the secret. Before Xcode 12.5, they would enable multicast traffic unconditionally, but starting with Xcode 12.5, they don't enable any multicast traffic.

Final bit: In the macOS kernel, when the network receive system call is executed, if the secret option has been set and the user consent flag is set, all is well. If the secret option is not set, the kernel looks for the multicast entitlement; if the entitlement exists, the receive call proceeds (legacy mode), otherwise it returns the "No route to host" error.

Apple denied my request for the multicast entitlement because it gives away the keys to the kingdom: the BSD Socket API is basically a set of kernel system calls-- it's just too low level for all the policy enforcement. Since I am still developing my app, their logic of "well, use this API instead" is sound.

If the story is true, to make Zeroconf function again on iOS requires code that calls the NWConnectionGroup API.

[zillemarco]

Hi everyone. Any news on this problem? I'm in need of updating and application which uses che bonjour service to discover some specific devices on the network and it is refusing to work. I usually work with my iPhone which has iOS 14.6 and the latest version of Xcode (12.5). With iPhone it is not working but I was able to test an "old" iPad Pro with iOS 14.3 and it is working fine, even using Xcode 12.5.

I added both the description and the bonjour services used by the application inside Info.plist.

Does someone have any new information regarding this problem? This time seems really like Apple killed a lot of applications with a single blow...

---

EDIT I downgraded to Xcode 12.4, as others have, and it actually is working again even with iOS 14.6 (I assume 14.5 too)

[SpencerBurgess]

@mduchev @clairernovotny Here's Apple's response to me:

Thank you for your interest in Multicast Networking. Since Bonjour is supported natively on the system, and does not require an entitlement, we’d ask that you adopt one of the system APIs that provides Bonjour functionality. Best regards, Apple Networking

After thinking about this, this is my guess/story about what is going on:

The reason this issue appeared with Xcode 12.5 is that the new multicast restriction policy is implemented in Apple-shipped library code (the NWConnectionGroup API), and an updated version of that library would ship with the new Xcode. Likewise, the library wrappers for all system calls would also ship with the new release of Xcode.

The core bit: there's some undocumented socket option or system call argument that signals to the macOS kernel that multicast traffic is allowed on this specific socket. The NWConnectionGroup API knows this secret and will enable multicast traffic on a socket if: (1) the desired mDNS protocols and description are coded in Info.plist (2) the protocols specified do not violate Apple's restrictions (3) the device user consents to allow the app to examine network traffic. Finally, as results from the multicast query are returned by the Receive() system call, the NWConnectionGroup API then gets to filter the data it returns to the caller.

The legacy BSD Sockets system call wrappers also know the secret. Before Xcode 12.5, they would enable multicast traffic unconditionally, but starting with Xcode 12.5, they don't enable any multicast traffic.

Final bit: In the macOS kernel, when the network receive system call is executed, if the secret option has been set and the user consent flag is set, all is well. If the secret option is not set, the kernel looks for the multicast entitlement; if the entitlement exists, the receive call proceeds (legacy mode), otherwise it returns the "No route to host" error.

Apple denied my request for the multicast entitlement because it gives away the keys to the kingdom: the BSD Socket API is basically a set of kernel system calls-- it's just too low level for all the policy enforcement. Since I am still developing my app, their logic of "well, use this API instead" is sound.

If the story is true, to make Zeroconf function again on iOS requires code that calls the NWConnectionGroup API.

Testing a few things mentioned here, I can also confirm that Xcode 12.5 is the issue. It seems like we may have to come up with a fix by switching over to use the NWConnectionGroup API, that sounds like a decent long-term solution.

Hi everyone. Any news on this problem? I'm in need of updating and application which uses the bonjour service to discover some specific devices on the network and it is refusing to work. I usually work with my iPhone which has iOS 14.6 and the latest version of Xcode (12.5). With iPhone it is not working but I was able to test an "old" iPad Pro with iOS 14.3 and it is working fine, even using Xcode 12.5.

I added both the description and the bonjour services used by the application inside Info.plist.

Does someone have any new information regarding this problem? This time seems really like Apple killed a lot of applications with a single blow...

EDIT I downgraded to Xcode 12.4, as others have, and it actually is working again even with iOS 14.6 (I assume 14.5 too)

Unless someone has an intuitive quick fix to this issue outside of a downgrade to 12.4, I will go ahead and make a test branch and try to get something running over this next week using the NWConnectionGroup API in replacement of the legacy Socket.

[clairernovotny]

@SpencerBurgess please keep us updated on your test. I also filed an issue on the .NET Runtime repo as ultimately Zeroconf is just using the built-in UdpClient library, so this would seem to affect anything on iOS.

https://github.com/dotnet/runtime/issues/53767

[rcinge]

@SpencerBurgess , @clairernovotny I have built a working Bonjour-based prototype.

This is its location: https://github.com/rcinge/Zeroconf/tree/bonjour_prototype

Currently it supports ResolveAsync() only; I have hardcoded the protocol "_audioplayer-discovery._tcp.local." in the test Xamarin app, simply because that's what I am working on. You should change this to a service that responds on your network. Note the required addition in project ZeroconfTest.Xam.iOS, Info.plist, and note that the NSBonjourServices string(s) should not include the .local:

  <key>NSLocalNetworkUsageDescription</key>
  <string>Looking for local AudioPlayer mDNS/Bonjour service</string>
  <key>NSBonjourServices</key>
  <array>
    <string>_audioplayer-discovery._tcp</string>
  </array>

The protocol used for the query is hardcoded in project ZeroconfTest.Xam, App.cs: ResolveOnClicked().

The scaffold entry point is in project Zeroconf, BonjourScaffold.cs. The integration is in BonjourBrowser.cs.

Note 1:

Apple's API wants the caller to search using ServiceType and Domain. So there's some logic in BonjourBrowser.StartDiscovery() to parse these from the protocol argument of ResolveAsync().

From an extremely rapid skim of the RFC, it looks like ServiceTypes must end in "._tcp." or "._udp." (I am calling these the "delimiters") If there are more "delimiters" or I just plain missed the point, let me know what's required and I'll fix it...

Up to and including the "delimiters" is the service type; what follows the "delimiters" is the domain. If nothing follows those "delimiters," the domain should be assumed to be ".local."

Apple's SearchForServices() documentation encourages developers to leave the domain parameter empty.

Note 2:

If someone knows where the real Xamarin definition of sockaddr, sockaddr_in, and sockaddr_in6 live, then the Sockaddr cruft class can go away.

Note 3:

A lot of the dictionary and Sockaddr stuff was brute force; for instance, all IPAddresses reported by a service are used as part of the key of the discoveredService dictionary and the entire zeroconfHost dictionary. Why? Just because you're paranoid doesn't mean they're not out to get you.

Known bugs/issues:

For multiple protocols, the current code issues a SearchForServices(), but does not pay attention to the SearchStarted/SearchStopped events. It should do this and wait until all SearchStopped events (or errors) are seen?

None of the ResolverListener() is implemented.

Certainly many more.

[neicureuil]

Hi @rcinge I have try your prototype by adding your Zeroconf lib (https://github.com/rcinge/Zeroconf/tree/bonjour_prototype) in my project. Compared to before, I have no more exception and all seems to be fine but I never found any result during my scan, but when i try on Android, I have results.

Have you some news for this issue ? Can you help me ? Thanks

[mduchev]

@neicureuil Have you looked at any UnobservedTaskExceptions? In my project, the exceptions are "silent" and can only be caught if, in the iOS project, you listen for any unobserved exceptions. You can listen to such exceptions if you go to AppDelegate.cs file and add event handler TaskScheduler.UnobservedTaskException += TaskScheduler_UnobservedTaskException; in the FinishedLaunching method.

[neicureuil]

Hi, I just looked at UnobservedTaskExceptions but nothing. Is the code from https://github.com/rcinge/Zeroconf/tree/bonjour_prototype is working for you on IOS ?

Thanks

[mduchev]

I can't say for sure since I haven't tried it with this specific code that you mention. I've only tried it with the Zeroconf NuGet package (https://www.nuget.org/packages/Zeroconf). There you can see the unobserved exception. If you don't see the exception, I think that it is still failing silently, thus resolving in 0 results being returned. You can track the official .NET issue here https://github.com/dotnet/runtime/issues/53767. Unfortunately, I don't see any movement whatsoever there.

[rcinge]

Hi @neicureuil,

Sorry for the delay. I have put together a "wrapper" library ZeroconfSwitch that I hope will help. Rather than pollute/corrupt the Zeroconf project with the workaround, this wrapper library only redirects Zeroconf calls on Xamarin.iOS (iOS 14.5 and later). All other calls on other platforms are forwarded directly to Zeroconf.

The previous code in the fork of Zeroconf was very much a proof-of-concept-- this wrapper is somewhat tested and more usable.

You should be able to compile the Android and iOS test apps in your environment and run them and see some kind of results... especially since the provided Info.plist includes the _apple-mobdev2._tcp service... your iOS device should see itself.

Hope this helps, and is not too late for your project.

[clairernovotny]

@rcinge I haven't had a chance to look at what you needed to do yet, but we can add an iOS specific target that does what's needed. I'd happily take a PR that adds that.

[rcinge]

@clairernovotny I've submitted PR #212 . The CI build did fail in Azure; in my IDE, everything compiled and tested fine.

I changed the Zeroconf project SDK to MSBuild.Sdk.Extras, and added a global.json file specifying version 3.0.23. I'm guessing that's the issue... it's a fundamental change. I will investigate...

[rcinge]

@clairernovotny In the raw log of the failed build, I see the error message:

2021-08-11T18:58:05.9645544Z   1:25>C:\Users\VssAdministrator\.nuget\packages\msbuild.sdk.extras\3.0.23\Build\Workarounds.targets(27,5): error : If you are building projects that require targets from full MSBuild or MSBuildFrameworkToolsPath, you need to use desktop msbuild ('msbuild.exe') instead of 'dotnet build' or 'dotnet msbuild' [D:\a\1\s\Zeroconf\Zeroconf.csproj]
2021-08-11T18:58:05.9646553Z        Done executing task "Error" -- FAILED.
2021-08-11T18:58:05.9647130Z   1:25>Done building target "_SdkWarnWhenUsingMSBuildCore" in project "Zeroconf.csproj" -- FAILED.
2021-08-11T18:58:05.9647879Z   1:25>Done Building Project "D:\a\1\s\Zeroconf\Zeroconf.csproj" (Build target(s)) -- FAILED.
2021-08-11T18:58:05.9648453Z   1:19>Done executing task "MSBuild" -- FAILED.
2021-08-11T18:58:05.9649007Z   1:19>Done building target "DispatchToInnerBuilds" in project "Zeroconf.csproj" -- FAILED.
2021-08-11T18:58:05.9649645Z   1:19>Done Building Project "D:\a\1\s\Zeroconf\Zeroconf.csproj" (pack target(s)) -- FAILED.

It's obviously the change of SDK in Zeroconf/Zeroconf.csproj that is the cause of the failure.

I don't see any way to influence this setting using the azure-pipelines.yml file; is this a setting in the Azure account that needs changing? Unfortunately I don't know the CI environment well enough to troubleshoot this further.

[clairernovotny]

@rcinge In the PR, you can update the azure-pipelines.yml:

https://github.com/novotnyllc/Zeroconf/blob/main/azure-pipelines.yml#L36-L41

Should be something like

- task: VSBuild@1
  inputs:    
    solution: .\Zeroconf\Zeroconf.csproj
    configuration: $(BuildConfiguration)
    msbuildArgs: '/restore /m /t:Pack /p:PackageOutputPath=$(Build.ArtifactStagingDirectory)\Packages'

That'll make it use msbuild.exe for building instead of dotnet build.

[rcinge]

@clairernovotny That fixed the build. Thank you...

[neicureuil]

Hi, Sorry for the late reply. I have used the code https://github.com/rcinge/Zeroconf/tree/bonjour_prototype instead of the classic Zeroconf lib and Its seems to works.

But from one day to the next, Its doesn't find my service on IOS. So I saw that you update the ZeroConf lib. So I update my project to reuse the lastest version of the lib, but now I got this error :

But now I got the error : System.Net.Sockets.SocketException (0x80004005): No route to host What I can do to fix that ?

I have also test with Android and no problems.

Regards

[rcinge]

Hi @neicureuil,

So I update my project to reuse the lastest version of the lib, but now I got this error : But now I got the error : System.Net.Sockets.SocketException (0x80004005): No route to host What I can do to fix that ?

I have seen the "No route to host" when using previous versions of Zeroconf. Maybe your project is unintentially using a previous version that does not include the Xamarin iOS workaround? The "No route to host" would be the Zeroconf library trying to use the socket interface and getting blocked by iOS 14.5+.

Another possibility: the NSBonjourServices and/or NSLocalNetworkUsageDescription keys and values are missing or incorrect in Info.plist... this would be the iOS 14.5+ blocking unconfigured domain browsing.

You can temporarily add something like the following to your app:

IReadOnlyList<string> domains = ZeroconfResolver.GetiOSInfoPlistServices(selectedDomain);

The "domains" list should contain the same information from the NSBonjourServices section of Info.plist. If the list contains your service, the Info.plist is not the problem.

I recommend restarting your Visual Studio, then verify you're referencing the correct version of Zeroconf, then make sure your Xamarin.iOS application's Info.plist contains the required keys and values-- then rebuild the project.

If none of the above fix the issue, will you provide a stack trace?

But from one day to the next, Its doesn't find my service on IOS.

Try adjusting the scanTime parameter to ZeroconfResolver.ResolveAsync. The default value is 2 seconds; see if increasing the value makes a difference.