syedhali / EZAudio

An iOS and macOS audio visualization framework built upon Core Audio useful for anyone doing real-time, low-latency audio processing and visualizations.
Other
4.93k stars 821 forks source link

Devices with no Inputs will crash EZAudio and Solution is to extend the Framework to handle Aggregated Devices. #389

Open designerfuzzi opened 2 years ago

designerfuzzi commented 2 years ago

As of 2021 aggregated Devices exist or how to call them.. Combined Devices which can be setup to only use Inputs or Outputs just the same as some devices will just have no inputs or outputs. When i try to instantiate EZAudioDevice or enumerating it will crash in EZAudioDevice.m

+ (NSInteger)channelCountForScope:(AudioObjectPropertyScope)scope forDeviceID:(AudioDeviceID)deviceID {

    AudioObjectPropertyAddress address;
    address.mScope = scope;
    address.mElement = kAudioObjectPropertyElementMaster;
    address.mSelector = kAudioDevicePropertyStreamConfiguration;

    // - - - - catch configuration - - - - -
    AudioBufferList streamConfiguration;
    UInt32 propSize = sizeof(streamConfiguration);
    EZAudioUtilities_checkResult(AudioObjectGetPropertyData(deviceID,
                                                 &address,
                                                 0,
                                                 NULL,
                                                 &propSize,
                                                 &streamConfiguration),
                                        "Failed to get frame size");

    // - - - - use read property - - - - -
    NSInteger channelCount = 0;
    for (NSInteger i = 0; i < streamConfiguration.mNumberBuffers; i++)
    {       
        channelCount += streamConfiguration.mBuffers[i].mNumberChannels;
        //FIXME: ^ crash! channelCount is ridiculous high number here and mNumberChannels is empty
    }

    return channelCount;

}

msg: [AudioHAL_Client] HALC_ProxyIOContext.cpp:1984:GetPropertyData: HALC_ProxyIOContext::_GetPropertyData: bad property data size for kAudioDevicePropertyStreamConfiguration

[AudioHAL_Client] HALC_ShellObject.cpp:401:GetPropertyData: HALC_ShellObject::GetPropertyData: call to the proxy failed, Error: 561211770 (!siz)

[AudioHAL_Client] HALPlugIn.cpp:292:ObjectGetPropertyData: HALPlugIn::ObjectGetPropertyData: got an error from the plug-in routine, Error: 561211770 (!siz)

`Error: Failed to get frame size ('!siz')`

`

i guess this is because EZAudio is fairly an old framework that didnt know about Aggregated Devices.

but why, we could improve it. after googling a found some hints how to do it in this gist

maybe someone wants to join solving this issue

ps: here a screenshot of my audio devices..

devices_

device.name Built-in Microphone -- device.UID AppleHDAEngineInput:1B,0,1,0:1 device.name Built-in Output -- device.UID AppleHDAEngineOutput:1B,0,1,1:0
device.name QU-16 Audio -- device.UID AppleUSBAudioEngine:Allen&Heath Ltd:QU-16:14130000:2,3
device.name BlackHole 16ch -- device.UID BlackHole16ch_UID
device.name Combi -- device.UID ~:AMS2_Aggregate:0

as you can read, it is also possible to create an Aggregate Device with no inputs and outputs at all.

when i change back the system setup to a normal audio device and erase the aggregated device my App does not crash and just works flawless also showing properly the channels.. it is not a Blackhole issue, as this works also perfectly with EZAudio.

designerfuzzi commented 2 years ago

ok.. didn't make a Fork to push commits.. to bizzy with my project.. Coded the following solution that enables EZAudio to handle Aggregated Devices.. Sticking with the way it is done in EZAudio in EZAudioDevice.h implement this declaration

/**
Enumerates a specific Aggregate Device.
   - OSX only
@param deviceID aggregate AudioDeviceID to check for its SubDeviceList
@param block When enumerating this block executes repeatedly for each EZAudioDevice found. It contains two arguments - first, the EZAudioDevice found, then a pointer to a stop BOOL to allow breaking out of the enumeration)
*/
+ (void)enumerateSubDevice:(AudioDeviceID)deviceID UsingBlock:(void(^)(EZAudioDevice *subdevice, BOOL *stop))block;

and in EZAudioDevice.m extend the @interface EZAudioDevice () for macOS with the following property

@property (nonatomic, assign, readwrite) BOOL isAggregateDevice;

then exchange the @implementation that is supposed to handle macOS with the following parts, Everything will work as before but this time it takes aggregated devices into the device lookup and does'nt crash your app. Aggregate Devices appear then with their given name and sorting you did set up in Audio-Midi-Setup.app

#elif TARGET_OS_MAC

+ (void)enumerateDevicesUsingBlock:(void(^)(EZAudioDevice *device, BOOL *stop))block
{
    if (!block)
    {
        return;
    }

    // get the present system devices
    AudioObjectPropertyAddress address = EZAudioDevice_addressForPropertySelector(kAudioHardwarePropertyDevices);
    UInt32 devicesDataSize;
    [EZAudioUtilities checkResult:AudioObjectGetPropertyDataSize(kAudioObjectSystemObject,
                                                                &address,
                                                                0,
                                                                NULL,
                                                                &devicesDataSize)
                                                                operation:"Failed to get data size"];

    // enumerate devices
    NSInteger count = devicesDataSize / sizeof(AudioDeviceID);
    AudioDeviceID *deviceIDs = (AudioDeviceID *)malloc(devicesDataSize);

    // fill in the devices
    [EZAudioUtilities checkResult:AudioObjectGetPropertyData(kAudioObjectSystemObject,
                                                            &address,
                                                            0,
                                                            NULL,
                                                            &devicesDataSize,
                                                            deviceIDs)
                                                            operation:"Failed to get device IDs for available devices on OSX"];

    BOOL stop = NO;
    for (UInt32 i = 0; i < count; i++)
    {
        AudioDeviceID deviceID = deviceIDs[i];

        EZAudioDevice *device = [EZAudioDevice new];
        device.deviceID = deviceID;
        device.manufacturer = EZAudioDevice_manufacturerForDeviceID(deviceID);
        device.name = EZAudioDevice_namePropertyForDeviceID(deviceID);
        device.UID = EZAudioDevice_UIDPropertyForDeviceID(deviceID);

        //
        // AggregateDevices could crash EZAudio. AggregateDevices can have 0 inputs and/or 0 outputs
        // and will not respond to kAudioObjectPropertyScopeInput or kAudioObjectPropertyScopeOutput as is
        // instead we lookup kAudioAggregateDevicePropertyActiveSubDeviceList and climb into those devices to find in & outputs
        bool isAggregate = EZAudioDevice_isAggregateDevice(deviceID);
        device.isAggregateDevice = isAggregate;

        if (!isAggregate)
        {
            device.inputChannelCount =  EZAudioDevice_channelCountForScope(kAudioObjectPropertyScopeInput , deviceID);
            device.outputChannelCount = EZAudioDevice_channelCountForScope(kAudioObjectPropertyScopeOutput, deviceID);
        }
        else
        {
            __block NSInteger inputs = 0;
            __block NSInteger outputs = 0;
            [self enumerateSubDevice:deviceID UsingBlock:^(EZAudioDevice *subdevice, BOOL *stop)
            {
                //printf("subdevice.name=%s has %lu InputChannels and %lu OutputChannels\n",subdevice.name.UTF8String, subdevice.inputChannelCount, subdevice.outputChannelCount);
                if (subdevice.inputChannelCount > 0)
                {
                    inputs +=subdevice.inputChannelCount;
                }
                if (subdevice.outputChannelCount > 0) {
                    outputs +=subdevice.outputChannelCount;
                }
            }];
            device.inputChannelCount  = inputs;
            device.outputChannelCount = outputs;
        }

        block(device, &stop);
        if (stop)
        {
            break;
        }

    }

    free(deviceIDs);
}

//------------------------------------------------------------------------------

static AudioDevicePropertyID EZAudioDevice_getDeviceTransportType(AudioDeviceID deviceID) {
    AudioDevicePropertyID deviceTransportType = 0;
    UInt32 propSize = sizeof(AudioDevicePropertyID);
    AudioObjectPropertyAddress propAddress = EZAudioDevice_addressForPropertySelector(kAudioDevicePropertyTransportType);
    [EZAudioUtilities checkResult:AudioObjectGetPropertyData(deviceID,
                                                            &propAddress,
                                                            0,
                                                            nil,
                                                            &propSize,
                                                            &deviceTransportType)
                                                            operation:"EZAudioDevice failed getDeviceTransportType"];
    return deviceTransportType;
}

//------------------------------------------------------------------------------

static bool EZAudioDevice_isAggregateDevice(AudioDeviceID deviceID) {
    AudioDevicePropertyID deviceType = EZAudioDevice_getDeviceTransportType(deviceID);
    return deviceType == kAudioDeviceTransportTypeAggregate;
}

//------------------------------------------------------------------------------

+ (void)enumerateSubDevice:(AudioDeviceID)deviceID UsingBlock:(void(^)(EZAudioDevice *subdevice, BOOL *stop))block
{
    if (!block)
    {
        return;
    }
    NSString *aggregatedDeviceName = EZAudioDevice_namePropertyForDeviceID(deviceID);
    UInt32 devicesDataSize = 0;
    AudioObjectPropertyAddress propAddress = EZAudioDevice_addressForPropertySelector(kAudioAggregateDevicePropertyActiveSubDeviceList);
    UInt32 subcount = EZAudioDevice_getNumberOfSubDevices(deviceID, propAddress, &devicesDataSize);
    AudioDeviceID *subdeviceIDs = (AudioDeviceID *)malloc(devicesDataSize);
    [EZAudioUtilities checkResult:AudioObjectGetPropertyData(deviceID,
                                                            &propAddress,
                                                            0,  
                                                            NULL,
                                                            &devicesDataSize,
                                                            &subdeviceIDs[0])
                                                            operation:"EZAudioDevice failed getAggregateDeviceSubDeviceList"];
    BOOL stop = NO;
    for (int a = 0; a < subcount; a++)
    {
        AudioDeviceID subdeviceID = subdeviceIDs[a];
        EZAudioDevice *device = [[EZAudioDevice alloc] init];
        device.deviceID = subdeviceID;
        device.manufacturer = EZAudioDevice_manufacturerForDeviceID(subdeviceID);
        device.name = [NSString stringWithFormat:@"%@ (%@)",aggregatedDeviceName, EZAudioDevice_namePropertyForDeviceID(subdeviceID)];
        device.UID = EZAudioDevice_UIDPropertyForDeviceID(subdeviceID);
        device.inputChannelCount =  EZAudioDevice_channelCountForScope(kAudioObjectPropertyScopeInput , subdeviceID);
        device.outputChannelCount = EZAudioDevice_channelCountForScope(kAudioObjectPropertyScopeOutput, subdeviceID);
        // kAudioSubDeviceInputChannelsKey
        device.isAggregateDevice = NO; // subdevices are not aggregateDevices, they are aggregated!

        block(device, &stop);
        if (stop)
        {
            break;
        }
    }

    free(subdeviceIDs);
}

//------------------------------------------------------------------------------

static UInt32 EZAudioDevice_getNumberOfSubDevices(AudioDeviceID deviceID, AudioObjectPropertyAddress propAddress, UInt32 *devicesDataSize)
{
    UInt32 propSize = 0;
    [EZAudioUtilities checkResult:AudioObjectGetPropertyDataSize(deviceID,
                                                                &propAddress,
                                                                0,
                                                                nil,
                                                                &propSize)
                                                               operation:"EZAudioDevice failed to getNumberOfSubDevices"];
    *devicesDataSize = propSize;
    return propSize / (UInt32)sizeof(AudioDeviceID);
}

//------------------------------------------------------------------------------

+ (NSArray *)devices
{
    __block NSMutableArray *devices = [NSMutableArray array];
    [self enumerateDevicesUsingBlock:^(EZAudioDevice *device, BOOL *stop)
    {
        [devices addObject:device];
    }];
    return devices;
}

//------------------------------------------------------------------------------

EZAudioDevice *EZAudioDevice_deviceWithPropertySelector(AudioObjectPropertySelector propertySelector)
{
    AudioDeviceID deviceID;
    UInt32 propSize = sizeof(AudioDeviceID);
    AudioObjectPropertyAddress address = EZAudioDevice_addressForPropertySelector(propertySelector);
    [EZAudioUtilities checkResult:AudioObjectGetPropertyData(kAudioObjectSystemObject,
                                                            &address,
                                                            0,
                                                            NULL,
                                                            &propSize,
                                                            &deviceID)
                                                            operation:"Failed to get device device on OSX"];
    EZAudioDevice *device = [[EZAudioDevice alloc] init];
    device.deviceID = deviceID;
    device.manufacturer = EZAudioDevice_manufacturerForDeviceID(deviceID);
    device.name = EZAudioDevice_namePropertyForDeviceID(deviceID);
    device.UID = EZAudioDevice_UIDPropertyForDeviceID(deviceID);
    bool isAggregate = EZAudioDevice_isAggregateDevice(deviceID);
    device.isAggregateDevice = isAggregate;
    if (!isAggregate) {
        device.inputChannelCount = EZAudioDevice_channelCountForScope(kAudioObjectPropertyScopeInput, deviceID);
        device.outputChannelCount = EZAudioDevice_channelCountForScope(kAudioObjectPropertyScopeOutput, deviceID);
    } else {
        __block NSInteger inputs = 0;
        __block NSInteger outputs = 0;
        [EZAudioDevice enumerateSubDevice:deviceID UsingBlock:^(EZAudioDevice *subdevice, BOOL *stop)
        {
            if (subdevice.inputChannelCount > 0)
            {
                inputs +=subdevice.inputChannelCount;
            }
            if (subdevice.outputChannelCount > 0) {
                outputs +=subdevice.outputChannelCount;
            }
        }];
        device.inputChannelCount  = inputs;
        device.outputChannelCount = outputs;
    }

    return device;
}

//------------------------------------------------------------------------------

+ (EZAudioDevice *)currentInputDevice
{
    return EZAudioDevice_deviceWithPropertySelector(kAudioHardwarePropertyDefaultInputDevice);
}

//------------------------------------------------------------------------------

+ (EZAudioDevice *)currentOutputDevice
{
    return EZAudioDevice_deviceWithPropertySelector(kAudioHardwarePropertyDefaultOutputDevice);
}

//------------------------------------------------------------------------------

+ (NSArray *)inputDevices
{
    __block NSMutableArray *devices = [NSMutableArray array];
    [self enumerateDevicesUsingBlock:^(EZAudioDevice *device, BOOL *stop)
    {
        if (device.inputChannelCount > 0)
        {
            [devices addObject:device];
        }
    }];
    return devices;
}

//------------------------------------------------------------------------------

+ (NSArray *)outputDevices
{
    __block NSMutableArray *devices = [NSMutableArray array];
    [self enumerateDevicesUsingBlock:^(EZAudioDevice *device, BOOL *stop)
    {
        if (device.outputChannelCount > 0)
        {
            [devices addObject:device];
        }
    }];
    return devices;
}

//------------------------------------------------------------------------------
#pragma mark - Utility
//------------------------------------------------------------------------------

AudioObjectPropertyAddress EZAudioDevice_addressForPropertySelector(AudioObjectPropertySelector selector)
{
    AudioObjectPropertyAddress address;
    address.mScope = kAudioObjectPropertyScopeGlobal;
    address.mElement = kAudioObjectPropertyElementMaster;
    address.mSelector = selector;
    return address;
}
//------------------------------------------------------------------------------

NSString *EZAudioDevice_stringPropertyForSelector(AudioObjectPropertySelector selector, AudioDeviceID deviceID)
{
    AudioObjectPropertyAddress address = EZAudioDevice_addressForPropertySelector(selector);
    CFStringRef string;
    UInt32 propSize = sizeof(CFStringRef);
    [EZAudioUtilities checkResult:AudioObjectGetPropertyData(deviceID,
                                                            &address,
                                                            0,
                                                            NULL,
                                                            &propSize,
                                                            &string) 
                                                            operation:[NSString stringWithFormat:@"Failed to get device property (%u)",(unsigned int)selector].UTF8String];
    return (__bridge_transfer NSString *)string;
}

//------------------------------------------------------------------------------

NSInteger EZAudioDevice_channelCountForScope(AudioObjectPropertyScope scope, AudioDeviceID deviceID)
{
    AudioObjectPropertyAddress address;
    address.mScope = scope;
    address.mElement = kAudioObjectPropertyElementMaster;
    address.mSelector = kAudioDevicePropertyStreamConfiguration;

    // - - - - - - - - - - - - -
    AudioBufferList streamConfiguration;
    UInt32 propSize = sizeof(streamConfiguration);
    [EZAudioUtilities checkResult:AudioObjectGetPropertyData(deviceID,
                                                            &address,
                                                            0,
                                                            NULL,
                                                            &propSize,
                                                            &streamConfiguration) 
                                                            operation:"Failed to get frame size"];

    NSInteger channelCount = 0;
    for (NSInteger i = 0; i < streamConfiguration.mNumberBuffers; i++)
    {
        channelCount += streamConfiguration.mBuffers[i].mNumberChannels;
    }

    return channelCount;
}

//------------------------------------------------------------------------------

NSString *EZAudioDevice_manufacturerForDeviceID(AudioDeviceID deviceID)
{
    return EZAudioDevice_stringPropertyForSelector(kAudioDevicePropertyDeviceManufacturerCFString, deviceID);
}

//------------------------------------------------------------------------------

NSString *EZAudioDevice_namePropertyForDeviceID(AudioDeviceID deviceID)
{
    return EZAudioDevice_stringPropertyForSelector(kAudioDevicePropertyDeviceNameCFString, deviceID);
}

//------------------------------------------------------------------------------

NSString *EZAudioDevice_UIDPropertyForDeviceID(AudioDeviceID deviceID)
{
    return EZAudioDevice_stringPropertyForSelector(kAudioDevicePropertyDeviceUID, deviceID);
}

//------------------------------------------------------------------------------

- (NSString *)description
{
    return [NSString stringWithFormat:@"%@ { deviceID: %i, manufacturer: %@, name: %@, UID: %@, inputChannelCount: %ld, outputChannelCount: %ld }",
            [super description],
            self.deviceID,
            self.manufacturer,
            self.name,
            self.UID,
            self.inputChannelCount,
            self.outputChannelCount];
}

//------------------------------------------------------------------------------

- (BOOL)isEqual:(id)object
{
    if ([object isKindOfClass:self.class])
    {
        EZAudioDevice *device = (EZAudioDevice *)object;
        return [self.UID isEqualToString:device.UID];
    }
    else
    {
        return [super isEqual:object];
    }
}

//------------------------------------------------------------------------------

#endif

cheers.. also Changed Objc calls to C calls where possible without changing the API to reduce obj_msg sending