iOS: ofSoundStream access to AudioUnit

[cerupcat]

On iOS, we need to a way to get the AudioUnit. For example, Audiobus (almost required for most iOS audio apps now) needs a reference to the AudioUnit created. Is there a simple way to make it public or add a getter to access the audioUnit?

I had a preview pull request that was integrated, but I guess wasn't added back after the latest update: https://github.com/openframeworks/openFrameworks/pull/1923#issuecomment-19528599

[admsyn]

Hey @cerupcat. The design of the current iOS sound stream has the RemoteIO as an implementation detail of the SoundStream. IMO, adding a getter or static reference to it would lead to some pretty hard to debug behaviour.

You might be better off creating a different version of the sound stream, something like a SoundAudioBusStream, so that you have full control over the RemoteIO and can configure it the way AudioBus needs it set up. The AudioBus documentation makes it seem like a pretty easy way to do that would be to have the SoundAudioBusStream basically be a copy of the SoundOuputStream, but with a ABAudiobusAudioUnitWrapper property that's attached to the RemoteIO once it's booted up (i.e. at the end of - start).

If you wanted to go for the getter approach, it could sit nicely in the ofxiOSSoundStream, as it has access to the active SoundInputStream / SoundOutputStream (which expose the RemoteIO as a public property).

[cerupcat]

This is the Audiobus init function:

self.audiobusAudioUnitWrapper = [[ABAudiobusAudioUnitWrapper alloc] initWithAudiobusController:self.audiobusController audioUnit:self.audioController.audioUnit output:output input:input];

It requires passing in the audioUnit created (which is the problem since OF isn't exposing it). Previously I made it static based on a reputable example online and it's in the previous version of my app without any issues. We don't need it to be static as long as it's possible to get access to it. I'm not sure I followed what you meant by creating a SoundAudioBusStream.

You can see my previous method of making it static here: https://github.com/cerupcat/openFrameworks/commit/fe534e2fb8418a23fd5cd9416a31d26d7b5db33b

[admsyn]

Right, what I'm suggesting is that you add the Audiobus init functionality to either:

• The implementation of the SoundStream that you're using (i.e. SoundOutputStream.mm)
• Elsewhere, and access the RemoteIO by adding a getter to the ofxiOSSoundStream. The implementation of the getter would look something like this:

AudioUnit ofxiOSSoundStream::getOutputRemoteIO() {
    return [(SoundOutputStream *)soundOutputStream audioUnit];
}

Note that you'll have to call it after ofSoundStreamSetup, since it'll just be NULL before that. You'll also have to #include <AudioToolbox/AudioToolbox.h> in the header that refers to AudioUnit.

Adding it to the SoundStream impl would give you the most control, which might be necessary depending on the subtleties of Audiobus integration. Adding the getter to ofxiOSSoundStream might work as a quick-hack solution though.

The reason I'm saying we shouldn't just add a static reference or getter to openFrameworks is because it'd break encapsulation pretty bad, and would (IMO) almost certainly lead to frustrating bugs and weird behaviour as the RemoteIO is changed / initialized / dealloced / configured / etc by the SoundOutputStream. Example:

Q:why is my callback not happening? I've been looking at stackoverflow for days and everything seems right.. A:because SoundOutputStream is re-configuring the RemoteIO behind your back.

Q:why is my app crashing when I try to use my RemoteIO pointer after the app gets interrupted? A:because the SoundOutputStream created a new pointer to it and you got left with a dangling pointer.

etc.

There's probably a pretty decent opening for an addon that provides a slightly different SoundStream that's automatically set up for Audiobus (since we can't just add an Audiobus SDK dependency to oF). Adding inter-app audio support is on my to-do list, though, once that functionality is public.. :)

Pinging @julapy since he's more familiar with the intent of the iOS sound stream class design than I.

[cerupcat]

Thanks!

Ideally I'm looking for a solution that doesn't require hacking core each time there's a new OF update. I'm not sure this is a Audiobus only issue as I imagine there's other instances where we might want to access the audio unit. Wouldn't having a getter return the AudioUnit (instead of a static variable) keep encapsulation?

While I could create a static method (like previously) or a getter, I'd rather have something implemented in core so that I don't need to hack it or update an addon when OF changes. I've used the static approach previously with an app that has over 300k downloads without any ill effects.

There's also been a significant amount of changes in 0.8.0 in reference to audio, so I'm not exactly sure where I'd make such changes.

[astwist]

Hi @cerupcat

Sorry I'm a newb but please could you show me what you changed to get audiobus working in the later OF?

I had it working fine in 7.2 using the same code as yours, but now I get this error:

Thanks for any guidance, dead in the water here! Alex

[cerupcat]

Attached is some code showing how to integrate Audiobus. I made my own ABiOSSoundStream that you use instead of ofSoundStream. IT makes the AudioUnit accessible which it's not in the OF one.

On Tue, Jun 17, 2014 at 4:02 AM, alkex1982 notifications@github.com wrote:

Hi Cerupcat,

Sorry I'm a newb but please could you show me what you changed to get audiobus working in the later OF?

I had it working fine in 7.2 using the same code as yours, but now I get this error: [image: screen shot 2014-06-17 at 1 00 47 pm] https://cloud.githubusercontent.com/assets/7049799/3299231/b2ccfd70-f60e-11e3-9fcb-f2f1ae21b0bc.png

Thanks for any guidance, dead in the water here! Alex

— Reply to this email directly or view it on GitHub https://github.com/openframeworks/openFrameworks/issues/2544#issuecomment-46293279 .

Seth Sandler

• Blog: sethsandler.com
• Twitter: www.twitter.com/sethsandler
• Linkedin: http://www.linkedin.com/in/sethsandler
• Skype: sethsandler

[bilderbuchi]

@cerupcat attachments don't show in github comments ;-)

[astwist]

@cerupcat Hi Seth, thanks for your response, I went back to of_v0072 and works ok using the old example you emailed me a while back, even in xcode 5.11 compiling against ios7. This is fine for me, thanks for your help! Cheers Alex

[Miles-Tone]

@cerupcat hi seth, do you have an example project supporting audio bus 2, using of 008? I'm having difficulties with audio bus forcing a change in block size. is this something that can be adjusted for outside of pd? also are you still using ofxpd? thanks, miles

[astwist]

Hey Miles,

Seth kindly shared his code demonstrating how he sets up audiobus2 in OF

I'm away from pc right now but I'll forward you the files when I get home.

Cheers Alex Sent from my iPhone

On 21 août 2014, at 16:14, Miles-Tone notifications@github.com wrote:

@cerupcat hi seth, do you have an example project supporting audio bus 2, using of 008? I'm having difficulties with audio bus forcing a change in block size. is this something that can be adjusted for outside of pd? also are you still using ofxpd? thanks, miles

— Reply to this email directly or view it on GitHub.

[Miles-Tone]

That's fantastic, thank you so much. What audio engine are you using? Thanks, Miles

On 21 Aug 2014, at 18:09, alkex1982 notifications@github.com wrote:

Hey Miles,

Seth kindly shared his code demonstrating how he sets up audiobus2 in OF

I'm away from pc right now but I'll forward you the files when I get home.

Cheers Alex Sent from my iPhone

On 21 août 2014, at 16:14, Miles-Tone notifications@github.com wrote:

@cerupcat hi seth, do you have an example project supporting audio bus 2, using of 008? I'm having difficulties with audio bus forcing a change in block size. is this something that can be adjusted for outside of pd? also are you still using ofxpd? thanks, miles

— Reply to this email directly or view it on GitHub. — Reply to this email directly or view it on GitHub.

[astwist]

Hi Miles,

Here is the project Seth sent me. and below in bold is a copy/paste of the instructions he included in the email. I haven't gotten around to integrating audiobus2 in my apps yet (it's a project for early september after the summer vacay madness), but I have successfully used audiobus1 with the ofSoundStream, please let me know what you will be using, perhaps we can share our results :)

Good luck!

Cheers Alex

I'm not 100% sure what OF version this is using. I think it's 008.

What's different from the previous version is that I made my own ABiOSSoundStream which is basically a duplicate of OF's but it makes the audioUnit public (since OF's isn't public) and is required by Audiobus.

So instead of setting up audio output using ofSoundStreamSetup, you'll need to create an instance of ABiOSSoundStream. I'm including my SoundManager.cpp so you can see how I do it.

On Thu, Aug 21, 2014 at 4:14 PM, Miles-Tone notifications@github.com wrote:

@cerupcat https://github.com/cerupcat hi seth, do you have an example project supporting audio bus 2, using of 008? I'm having difficulties with audio bus forcing a change in block size. is this something that can be adjusted for outside of pd? also are you still using ofxpd? thanks, miles

— Reply to this email directly or view it on GitHub https://github.com/openframeworks/openFrameworks/issues/2544#issuecomment-52925377 .

[bilderbuchi]

@alkex1982 the github comment system doesn't handle attachments. probably best to take this to the OF forum

[astwist]

Hey Miles,

can you share an email address to send the attachment to? or email me alkex1982 at gmail for the file.

Cheers

On Thu, Aug 21, 2014 at 10:13 PM, Miles-Tone notifications@github.com wrote:

That's fantastic, thank you so much. What audio engine are you using? Thanks, Miles

On 21 Aug 2014, at 18:09, alkex1982 notifications@github.com wrote:

Hey Miles,

Seth kindly shared his code demonstrating how he sets up audiobus2 in OF

I'm away from pc right now but I'll forward you the files when I get home.

Cheers Alex Sent from my iPhone

On 21 août 2014, at 16:14, Miles-Tone notifications@github.com wrote:

@cerupcat hi seth, do you have an example project supporting audio bus 2, using of 008? I'm having difficulties with audio bus forcing a change in block size. is this something that can be adjusted for outside of pd? also are you still using ofxpd? thanks, miles

— Reply to this email directly or view it on GitHub. — Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/openframeworks/openFrameworks/issues/2544#issuecomment-52975654 .

[Miles-Tone]

hmmmmmmm I'm having some difficulties dealing with ARC in ABiOSSoundStream, lots of requests to use _bridge etc. Im going to keep on pushing with it. I've only just recently changed to using arc, so I'm still a bit new to it

[admsyn]

@Miles-Tone you can selectively compile certain files with ARC, instead of having to do it for the whole project. See this stackoverflow post (there's a corresponding -fobjc-arc if you want to do the opposite).

[Miles-Tone]

thanks adam, ill give that a go now

[Miles-Tone]

bingo, thats great. Thanks Adam.

There doesn't seem to be audioUnit in ABiOSSoundStream. Appears as an issue here:

ABiOSSoundStream * stream = (ABiOSSoundStream*)stream->getSoundOutStream(); [self.audiobus initWithApiKey:AUDIOBUS_KEY audioUnit:stream.audioUnit];

audioUnit doesn't exist in this ABiOSSoundStream. Do i have the right file, or am i being a little dense?