Ableton / LinkKit

iOS SDK for Ableton Link, a new technology that synchronizes musical beat, tempo, and phase across multiple applications running on one or more devices.
http://ableton.github.io/linkkit
Other
147 stars 10 forks source link

are Callbacks from Link guaranteed to execute in specific order? #64

Closed designerfuzzi closed 2 years ago

designerfuzzi commented 2 years ago

making use of Link in a macOS app. If SyncStartStop is enabled, the according callback fires as expected.

in this state when i start my clock/ aka request a start the callback calls back also. Not unexpected. But i can't make use of it because I try to allow my app also to stop when the peers session stops. Meaning the immediately fired callback stops my right before called start request.

Considering i may do something wrong in general.

Or

my SyncStartStop callback should actually fire before my apps local clock mechanism is starting, meaning

if SyncStartStop=enabled

  • i would not start my local clock and wait for the callback (introducing some microseconds latency by design, not perfect) from where i would start the local clock mechanism. And if SyncStartStop=disabled
  • i would start my local clock directly without waiting for the callback, as it will never come if disabled.

my actual question is, is the sorting of the SyncStartStop callback guaranteed to fire as direct answer of a requested session start? And side question, is my written intention on how to correct?

Thanks for your thoughts in advance.

fgo-ableton commented 2 years ago

Hey, I'm not sure I'm able to follow.

When start stop sync is enabled the callback should be fired every time the state changes. The callback contains a bool that says if you should be playing or not: https://github.com/Ableton/LinkKit/blob/938f6e330879826fa0c090efb2a8a0024f94966a/LinkKit/ABLLink.h#L100

Depending on how your application is set up it might be easiest to check for changes of isPlaying at the same place the tempo changes are handled. Here's how we do it in the example app: https://github.com/Ableton/LinkKit/blob/938f6e330879826fa0c090efb2a8a0024f94966a/examples/LinkHut/LinkHut/AudioEngine.m#L188 In the example app we only change the UI state when we get a callback. Also, if choose to start/stop the engine when you receive a callback you should eventually align to a sessionState so you app is in time with other peers.

designerfuzzi commented 2 years ago

lol, github bug on top.. thats fun! erased my entire answer. so again..

Actually not coding for iOS the moment, the problem occurs with Link for macOS, so no ABLLink.h or .o at all in use the moment. Despite i make use of LinkKit in iOS which handles only one single project at app life cycle. My MacOS App handles multiple projects with multiple tempo that a user can change at any time. So i needed objc delegates that can be changed and get triggered by the callbacks and to keep sorted Link is managed from a singleton without metronome render.

All works fine, just Synced start stop was troublesome.

to address my issues I changed my code to more clear naming scheme.. what is refered to as "isPlaying" is possibly just a local parameter, not any kind of state. (i say 'possibly' because I compare functionality from the macOS version and LinkKit for iOS) They are just not the same same. Some methods available in LinkKit are missing in the macOS version and wise versa.

Because "isPlaying" is very confusing as it tells nothing what is playing.. i introduced some "localInvokedTransport" property i set when my client app requests to start its clock locally. The same property is set to false no matter who stops the clock. That way i can distinguish if sync occured from remote or from locally.

also introduced the missing engineData.requestStart which does not exist as getter in ableton::linkaudio::AudioEngine for macOS.

With all above i know who requested the start or stop, my client or a remote (i.e.Ableton).

resulting in some surprises.. requesting start and stop from my app works perfectly synced. requesting start and stop from Ableton works nice, my client app starts and stops according to links remote state change. but starts/stops non-sycned immediately.. no matter how I ask for timeAtBeat..

leading me to the following problem, the docs for SessionState say.. "When observing a change of start/stop state, audio playback of a peer should be started or stopped the same way it would have happened if the user had requested that change at the according time locally. " but doing results in immediate action instead of synced. There is also no forced change of phase or beat happening, clearly visible/audible on a control app that runs in the same net.

to find the next time to invoke clock or relate my clock to i do.. where beat is 0.0 usually and quantum is 4.0..

ABLLinkSessionStateRef session = state.link.captureAppSessionState();
return state.link.clock().microsToTicks(session.timeAtBeat(beat, quantum));

assuming for a remote/peer invoked sync start/stop thats not right.

my earlier question was because (as it is still) when i open Ableton and start a startstopsynced link session, the callback is not invoked, but when i hit stop it is.. pointing me in some trouble in the pull mechanism or Ableton does just expose its timeline position after the first invocation. Which made me think the callback on startStopSync is not guaranteed to happen for a new peer. Also on multiple iOS devices with multiple apps testet, SyncToStartStop does not work for most even if the viewController suggests it is implemented while some work properly. Slowly getting a clue why this is, but i cant change that.. assuming other coders get also confused what "Sync" meant in SyncToStartStop or i just don't get what the feature should be other than a remote start/stop invocation.

being just 5 minutes away from schönhauser.. i would even invest in coffee to demonstrate and get this solved.

designerfuzzi commented 2 years ago

fun fact: when i ask for timeAtBeat with beat=8.0 and quantum=4.0 2 to force getting a more future timing to start, Ableton starts synced and my app as well as expected. But still in some rare cases i get negative time values.. `state.link.clock().microsToTicks(session.timeAtBeat(8.0, quantum2));`

designerfuzzi commented 2 years ago

seems i found a solution to my major troubles.

-(uint64_t)timeForNextAlignedBeatOfRemoteInvokation {
    ABLLinkSessionStateRef session = state.link.captureAppSessionState();
    _quantum = state.audioPlatform.mEngine.quantum();
    session.requestBeatAtTime(4.0, state.link.clock().micros(), _quantum);
    return state.link.clock().microsToTicks(session.timeAtBeat(4.0, _quantum));
}

as my client should start anyway in "future" and needs to have no negative time results that could cause my clock to start at 0/aka immediately - i have enough time to update AppSessionState and quantum and requestBeatAtTime which as much i testet gives future proof timings when invoked from remotely occurring starts or stops.

That answers also my side-effect question if SyncToStartStop guarantees that the callback is acting when remotely invoked and seems to solve the additional Issue i had that when Ableton is starting up and first time started a link'ed SyncToStartStop session that the first expected callback was not acting at all.

Sumup.. i was not expecting from the docs that requestBeatAtTime is needed for remote sync start/stop despite it was written it was specially designed for that. Hint: maybe an additional comment in source could point at that more specific so others don't do same mistake over and over again.

fgo-ableton commented 2 years ago

Happy to hear you made progress. I have to admit I can't quite follow your thoughts.

From the documentation: "Start/stop state changes only follow user actions. This means applications will not adapt to, or automatically change the start/stop state of a Link session when they are joining". Your app shouldn't start when it joins a running session. It should follow the start/stop commands that happen while it is in a session. So what you are seeing is expected behavior. "A start/stop state represents the user’s intent to start or stop transport at a given time." It will not automatically align your app. You will have to call 'requestBeatAtTime` to align your app. Link internally does not know about the quantum the app is using, so it can't do the alignment for you.

Regarding the framework. LinkKit.zip does contain LinkKit.xcframework which works for iOS and Catalyst. If you are not using UIKit might just want to use the plain C++ version of Link: https://github.com/ableton/link

designerfuzzi commented 2 years ago

yes i already used the plain C++ version, years ago developed a Quartz Composer Plugin that hooked into Ableton Link driving real time visuals with it.. can still be downloaded in github ^^. i think made it shortly after I was at MidiHack 2015 or so.. or was it earlier.. anyway long time ago.. :) Then used LinkKit heavily for iOS ever since and always had little troubles to wrap my head around of the tiny differences between the iOS and plain C++ version and the naming of methods..

Now it works perfectly with the plain C++ and an adopted Objective-C Class for macOS almost identical to the one LinkKit is offering. Just that is written as Singleton and introduces a Protocol that allows Delegates to be set on any ObjcClass or SwiftClass and calls their protocol methods/functions according for each callback Link offers after checking for protocol compliance. This made it very easy to handle a project that can have multiple sequencing (data) sessions open in parallel.

It works so well now i can even offer users to choose what to accept as remotely invoked SyncToStartStop types, on start or on stop or both and even can be changed while it is running. Your help was the last kick i needed, so thanks for that.

And i still have some plans changing tiny stuff, in example to make my Custom framework working on old G4 macs driving a local virtual midi host similar to Link2Midi, so Ableton 4.0 or Reason 3.0 and super old sound cards dont have to go in the trash can..

this is how it looks like the moment.. As you can see thats half of an Octatrack Sequencer meant to edit Projects and extend Midi features that can't be done with the real machine, like sliding between Midi values or exports the whole Octratrack Project as Ableton 10 or 11 Set or including all CC, progChange, Tempos as Scenes.. being a big step closer, thanks again.

Bildschirmfoto 2021-10-08 um 12 42 18