search

Ableton / LinkKit

iOS SDK for Ableton Link, a new technology that synchronizes musical beat, tempo, and phase across multiple applications running on one or more devices.
http://ableton.github.io/linkkit
Other
147 stars 10 forks source link

iOS 14 Unexpected Behavior #56

Closed olympianoiseco closed 2 years ago

olympianoiseco commented 3 years ago

I am testing on iOS14 and am getting some unexpected behavior with Link, where it fails to connect to peers on the local network.

Update : I've been able to reproduce the issue with LinkHut.

To test: Open Ableton on my computer (10.1.17, macOS 10.15.6) and enable Link. Open my app on iPad (iOS 13.2, latest LinkKit version) and enable Link, Open my app on iPhone (iOS 14, latest LinkKit version) and enable Link. The system requests permissions for local network usage, and I accept)

Expected result : Link works on all three devices.

Actual Result: Link works between Ableton and the iPad (iOS 13). The iPhone (on iOS14) does not find the other link enabled apps.

This is possibly related to the restrictions on local network access with iOS14. On iOS14, local network access must be confirmed by the user. As expected, this dialog pops-up when enabling Link for the first time. However, granting access does not fix the issue. In addition, if you deny access, the Link View Controller will show the enabled state regardless of permissions.

Additional breadcrumbs : The App Store build of my app (built with a previous iOS SDK, and older LinkKit) works as expected.

If I open up another one of my Link enabled apps on my device, my app is able to connect to all of the Links in the network. As soon as I quit the other Link app on the device, it disconnects.

I'm able to re-create the issue using the LinkHut demo project. LinkHut will not connect to other Link enabled apps in the network unless the older Link enabled app is also open. LinkHut works correctly on my iPad with iOS13.

Any thoughts or things to try would be greatly appreciated.

Thanks, Ben

fgo-ableton commented 3 years ago

Hey Ben,

Thanks for reporting this. Here is what I tried to reproduce this:

When giving permission to access the local network, all three devices find each other and everything works as expected.

I have no idea what is causing the issue for you. Have you tried to completely uninstall the app that is not working? Maybe the OS remembers that the app does not have networking permissions for some reason, and just reinstalling via Xcode does not reset that. That's unfortunately the only I can think of at the moment.

Best, Florian

olympianoiseco commented 3 years ago

Thanks Florian, I've tried a couple of other things (updating Xcode, restarting the device, re-installing the app) and am still running into the issue. I will report back when I figure out what's going on. Hope you are well! Ben

fgo-ableton commented 3 years ago

This is weird! Let me know if there is something I should test.

uwehollatz commented 3 years ago

I am running into exact same problem as @olympianoiseco describes when preparing my app for an iOS14 update. I am a bit lost on how to fix this.

@fgo-ableton : The problem is reproducible with LinkHut by setting the deployment target to any value greater than iOS 11.4, so there is a strong relationship to the issue #57

frogg commented 3 years ago

@fgo-ableton : The problem is reproducible with LinkHut by setting the deployment target to any value greater than iOS 11.4, so there is a strong relationship to the issue #57

I'd even go as far and say both #56 and #57 have the same root cause and could be fixed by the same fix. We need to know which networking privacy rules(?) changed with iOS 11.4 then.

fgo-ableton commented 3 years ago

Ok. I can reproduce this now. Link does not use Bonjour. It uses multicast UDP messages on port 20808. I found https://developer.apple.com/forums/thread/662082 and https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_developer_networking_multicast It seems there are indeed new requirements in iOS 14. And as @uwehollatz pointed out, when compiling against the iOS 11.4 SDK, Link still work fine - even when running on iOS 14.

frogg commented 3 years ago

Ok. I can reproduce this now. Link does not use Bonjour. It uses multicast UDP messages on port 20808. I found https://developer.apple.com/forums/thread/662082 and https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_developer_networking_multicast It seems there are indeed new requirements in iOS 14. And as @uwehollatz pointed out, when compiling against the iOS 11.4 SDK, Link still work fine - even when running on iOS 14.

Oh wow, that sounds like you need to request a special entitlement from Apple in order to make LinkKit work then?

This entitlement requires permission from Apple before you can use it in your app. Request permission from the Multicast Networking Entitlement Request page.

designerfuzzi commented 3 years ago

No special requirements from Apple needed, just User Consent to agree that the local network is used from the App that asks for consent

-

Am 12.10.2020 um 11:30 schrieb Frederik Riedel notifications@github.com:

Ok. I can reproduce this now. Link does not use Bonjour. It uses multicast UDP messages on port 20808. I found https://developer.apple.com/forums/thread/662082 and https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_developer_networking_multicast It seems there are indeed new requirements in iOS 14. And as @uwehollatz pointed out, when compiling against the iOS 11.4 SDK, Link still work fine - even when running on iOS 14.

Oh wow, that sounds like you need to request a special entitlement from Apple in order to make LinkKit work then?

This entitlement requires permission from Apple before you can use it in your app. Request permission from the Multicast Networking Entitlement Request page.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

uwehollatz commented 3 years ago

I cannot set the target of my app to anything below iOS13 since I am relying on this version. So basically every app which uses Ableton Link on iOS and needs the target set to iOS12 and higher will stop working ... ok, I have filled out the form to request the multicast network entitlement form Apple. I will let you know about my outcomes then.

uwehollatz commented 3 years ago

No special requirements from Apple needed, just User Consent to agree that the local network is used from the App that asks for consent - …

no sorry @designerfuzzi, only the consent is not working.

Reinissance commented 3 years ago

Can confirm: the nslocalnetworkusagedescription is needed, as well as a bonjourservice entry: at least my app using audiobus needed it for this to work, but for link I didn’t put no entry to this array and it works as expected for me. Could be it’s needed there because before I made it for audiobus the user wasn’t asked for permission to grant the usage?

uwehollatz commented 3 years ago

OK - here is my update: I have requested the com.apple.developer.networking.multicast entitlement for my app at Apple yesterday. Today morning I got approval, Apple is fast :) Then I was following the steps (New Process) as described here https://developer.apple.com/forums/thread/663271 And ... Its working now! So I can confirm: When you target iOS12 and up, you have to include NSLocalNetworkUsageDescription in your Info.plist and you need to request the Multicast Network Entitlement for your AppID and apply it to your app as described.

fgo-ableton commented 3 years ago

Just adding NSLocalNetworkUsageDescription and a Bonjour entry for the app does not seem to work. Which is in accordance with what Apple sais. This unfortunately means all apps with Link support will have to request the entitlement. Also one is not able to test with the LinkHut example app on a device anymore. 😞

fgo-ableton commented 3 years ago

Can confirm: the nslocalnetworkusagedescription is needed, as well as a bonjourservice entry: at least my app using audiobus needed it for this to work, but for link I didn’t put no entry to this array and it works as expected for me. Could be it’s needed there because before I made it for audiobus the user wasn’t asked for permission to grant the usage?

Maybe this works because Audiobus is actually using Bonjour, and that also enables other network traffic?

Reinissance commented 3 years ago

That’s my guess. The app also uses osc protocol so there is an entry for it on the bonjour service too, so these two seem to open the network connectivity.

designerfuzzi commented 3 years ago

OSC Port (TCP/UDP) Messaging can be driven without Bonjour entries also but the DNS record can be implemented for convenience so Devices can discover other hosts in a local network. While OSC could be driven as Multicast Network its standard not to do so and instead to use normal IP targeting to specific hosts which is why any OSC app would start with introduction of a list of known devices and IP’s to communicate with.

Multicast can keep this process away from users when the protocol is constructed that way and the other Peers are not targeted specifically.

Which leads to the assumption that fgo is right. Use of Link would need the Apple Multicast Entitlement.

Then more wondering why we are the first to discover this issue.

Another thing is, would it work when you implement the old Link bundle in an App that has deployment target set to iOS14?

uwehollatz commented 3 years ago

Another thing is, would it work when you implement the old Link bundle in an App that has deployment target set to iOS14?

No. Once you target iOS12 and up and run on iOS14 you will need the entitlement.

fgo-ableton commented 3 years ago

Another thing is, would it work when you implement the old Link bundle in an App that has deployment target set to iOS14?

No. Once you target iOS12 and up and run on iOS14 you will need the entitlement.

Using Xcode 11 and targeting iOS 13 might also allow running on iOS 14. But I don't have the devices here to verify that.

olympianoiseco commented 3 years ago

@fgo-ableton I requested the entitlement from Apple and got this response, which I don't know how to respond to :

Thank you for your interest in Multicast Networking. Can you please provide more details on which multicast or broadcast protocol (IP address and port) you need to use? What messages are exchanged between devices?

@uwehollatz, what did you tell them to get approved?

uwehollatz commented 3 years ago

This is what I have filled into the form to get approval:

Describe your app:

Explain why your app needs to send multicast or broadcast traffic, or browse for all Bonjour service types

olympianoiseco commented 3 years ago

Thanks @uwehollatz. That's more or less what I said in my application, but I didn't include a link to the website. Hopefully adding the link to the Ableton Link website will be enough to get it through.

fgo-ableton commented 3 years ago

@olympianoiseco The IP and port Link uses are 224.76.78.75:20808. I'll add some documentation once we have successfully gone through the entitlement process.

fgo-ableton commented 3 years ago

Our request for the entitlement has been approved. We also referenced https://www.ableton.com/en/link/ I added some information about iOS 14 compatibility to the documentation.

fgo-ableton commented 3 years ago

@olympianoiseco Are there any news on the entitlement process from your side?

frogg commented 3 years ago

@olympianoiseco Are there any news on the entitlement process from your side?

I requested an entitlement in a similar manner, and mine just got approved this morning.

olympianoiseco commented 3 years ago

Mine just came through last night as well! Had to bug them, but it's all good now. Haven't implemented the entitlement yet but I don't expect that to be an issue. Thanks!