ofxAudioUnitFilePlayer pause / resume, priming, seeking

[admsyn]

Hey all, this PR covers a few bugs and feature requests :)

@sebleedelisle There's a simple pause() function on ofxAudioUnitFilePlayer now which takes care of all that fiddly timey wimey stuff. It works like most media players, so calling play() then pause() then play() again will resume where playback stopped at pause(). Calling stop() will reset the playhead back to the beginning.

@jasonlevine I've added another function called playAtSampleTime() which lets you start playback at an arbitrary point in the file (e.g. an argument of 44100 will start one second in).

@antimodular I've added automatic file priming to ofxAudioUnitFilePlayer. This should address the issue where the file player sometimes doesn't start. By default, play() will call prime(), but you can do it manually as well. Let me know if you ever see the issue again, though.

Closes #8 and closes #5

[jasonlevine]

Hey @admsyn ! Thanks for the changes. From my initial testing, it seems like the problem of ofxFilePlayers not starting has been solved. However, I am experiencing two new problems. The first is that my program falls into what seems to be an infinite loop when it reaches the prime() function here: OFXAU_RETURN(AudioUnitSetProperty(*_unit, kAudioUnitProperty_ScheduledFilePrime, kAudioUnitScope_Global, 0, &framesToPrime, sizeof(framesToPrime)), "priming file player");

So it never makes it to: _primed = true;

After quitting and restarting about 7 times, it finally started playing all seven fileplayers but AAAH! they're all totally out of sync. After quitting and restarting a couple more times, they fell into sync, but eventually the infinite loop problem came back. Not sure what kind of useful information I can give you, but I can tell you that I did these tests with an existing project and a brand new project, so the problem is not unique to my existing project that was made with the earlier version of ofxAudioUnit.

[admsyn]

It's weird that it's returning out of the kAudioUnitPropertyScheduledFilePrime call, from what I understand that's not really a call that should fail. Is it printing out "error <some number> while priming file player"?

To help me narrow it down, what's the sequence of calls you're doing on the file players to get them into that state?

Also to make them start in sync you should do something like:

#include <mach/mach_time.h>
uint64_t now = mach_absolute_time();
player1.play(now);
player2.play(now);

or prime them all ahead of time before calling each of the play() functions. If that's not working for you then let me know.

[jasonlevine]

Ok. Calling prime() before play() seems to have solved the timing issue, and it hasn't fallen into the infinite loop since. However, when calling prime() before each play() like this:

// stems is a vector<ofxAudioUnitFilePlayer *>
  for ( int i = 0; i < NUMTRACKS; i++ ) {
        stems[i]->prime();
        stems[i]->play();
    }

I consistently had dropped tracks.

Now I've switched to

    for ( int i = 0; i < NUMTRACKS; i++ ) {
        stems[i]->prime();
    }

    for ( int i = 0; i < NUMTRACKS; i++ ) {
        stems[i]->play();
    }

And all tracks are playing and in sync.

As for the infinite loop, there is no error and no crash. Just a rainbow beach ball and 100% of the CPU being used. If I paused the execution, Xcode always lands on that property setting call in prime(). Like I said it seems to be fixed from calling prime() before play(), but I can't be 100% sure.

[admsyn]

Yeah that behaviour makes sense. When prime() is called, it waits until the file player is actually primed before returning (i.e. it waits until enough data has been read off disk). If you have a few file players and you're calling prime and play alternately as you were there, it'd make sense that there'd be enough buildup of time to de-sync the earlier fileplayers from the later ones. Maybe I should document the priming behaviour better :)

The infinite loop thing is definitely a bug I'll look at, there shouldn't be any way to call the public file player API that allows that to happen.

[jasonlevine]

testing update: I'm getting dropped tracks left and right. This could just be human-pattern matching but it feels like it goes through phases of dropping tracks or not dropping tracks. If I just rebuild code enough times without making any changes, I get out of the "dropping tracks" phase. Then, maybe a half hour or so later, I fall back into it. I also notice that the first track is often the dropped track, but then on the next build 3-5 random tracks will be dropped.

[admsyn]

I've noticed it kind of going through phases like that in the past, and it feels like it has something to do with the low-level OSX audio system getting stressed out about misbehaving Audio Unit-based apps. At least, it seems like it starts happening after I've had to force quit an app I'm working on a few times due to buggy code. If that's the case, the force-quits you had to do after the infinite loop prime() stuff might have set it off.

That said, the prime() code was supposed to fix this sort of thing.

If you restart, do things start behaving better again?

[jasonlevine]

No. Tracks still dropping after restart. It feels like the three phases are:

1. "no tracks dropped" which lasts through several consecutive rebuilds
2. "track 1 dropped" which lasts through several consecutive rebuilds
3. "random number of random tracks dropped" which sporadically appears during phase 2.

[jasonlevine]

Ok. This might be meaningful. So since I restarted, I haven't been able to get out of phase 2(track 1 not playing). So I remember that previous to the addition of priming to ofxAudioUnit, I thought I had fixed the tracks not starting problem by unrolling this for loop:

    for ( int i = 0; i < NUMTRACKS; i++ ) {
        stems[i]->play();
    }

to this:

    stems[0]->play();
    stems[1]->play();
    stems[2]->play();
    stems[3]->play();
    stems[4]->play();
    stems[5]->play();
    stems[6]->play();

It worked in the moment, but later on the tracks began being dropped again. So I tried it in my current situation and the first time a run the code all the tracks play and are in sync... but halfway through the song! I quit and rebuild and now it starts at the beginning, all tracks playing. Then I switch back to my original for loop, and now it works perfectly. Both times I unrolled my for loop, something woke up core audio.

[admsyn]

all the tracks play and are in sync... but halfway through the song!

Now that's pretty interesting. Maybe (since the apps aren't being quit like normal) there's some buildup of dead resources like file handles and threads that coreaudiod is mismanaging. The fact that stuff happens differently while just re-using the same code would point the finger at something coreaudiod related.

[jasonlevine]

OK.... are you ready for this testing update?

So all morning track 3 won't play. I rebuild and rebuild, unroll and reroll the for loop, and finally restart the machine. Track 3 still won't play. So I unroll the for loop again, and this time all tracks start but some of them are out of sync. I rebuild again and for the rest of the day (about 5-6 hours of nonstop coding) there are NO dropped tracks. Then I notice a lazy error on my part: I left the for loop AND the unrolled version BOTH uncommented. Like so:

  for ( int i = 0; i < NUMTRACKS; i++ ) {
        stems[i]->play();
    }

    stems[0]->play();
    stems[1]->play();
    stems[2]->play();
    stems[3]->play();
    stems[4]->play();
    stems[5]->play();
    stems[6]->play();

So I comment out the unrolled loop, and BAM! track three doesn't start!!! I rebuild three more times and track 3 never starts. Then I uncomment the unrolled loop and all tracks play. I can't believe it. I try commenting out the for loop. Now only 4 of the 7 tracks start! uncomment the for loop and every track plays again. I tried all 3 combinations over and over again with consistent results.

WTF is going on? Is it something about not enough time to prime? so the second time I call play() the dropped track has a chance to start? Maybe some sort of delay is needed between prime and play? And why is the same track consistently dropped when I used the unrolled for loop, while a random number of random tracks are dropped when I use the for loop? I thought the compiler would generate the same object code for both, but obviously this is not the case. I'm so confused...

ps I got my hands on the Learning Core Audio book again.....

[admsyn]

I'm so confused...

Same here :) I'm going to guess it's a combination of ofxAudioUnit doing something not quite right, and the AUFilePlayer / coreaudiod having some sort of weird edge case or bug being triggered. I'm going to keep plugging away at it, but the real frustrating thing about it is that there's not really any way to tell when it's actually fixed :/

Next time you're running into issues, try a sudo killall coreaudiod (core audio will restart after a few seconds). Let me know if that "fixes" it.

Also PS: there's a recorder (ofxAudioUnitRecorder) now. It can record m4a's if you stick it between audio units (like a tap).