AudioStreamMicrophone is broken on iOS hardware

[jhud]

Godot version: v3.1.1.stable.official

OS/device including version: iOS 13.1

Issue description: The audio input (Microphone) on iOS hardware does not work. It works on the simulator.

There are three separate issues causing this:

1. no microphone permission string in info.plist in XCode export by default (already reported in issue #30307)
2. the incorrect AVAudioSession.AudioSessionCategory is set at startup in app_delegate.mm. It works when changed to AVAudioSessionCategoryPlayAndRecord, ie: [[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayAndRecord error:nil]; Maybe this should be configurable from the Godot export settings, or inferred from the sound bus usage?

3. a "-50" error from AudioUnitRender in audio_driver_coreaudio.cpp whenever microphone input is enabled. This is because bufferList.mBuffers[0].mDataByteSize is too small, since there is a mismatch between the capture buffer format and the audio device.

Steps to reproduce:

1. Download minimal reproduction project below
2. Disable mute on the bus panel in Godot to ensure audio input and output occurs simultaneously.
3. Export to iOS build
4. Install .ipa on iOS hardware, and the app will fail to start or record.
5. Open XCode and install on simulator - observe that audio recording works properly.

Minimal reproduction project: This project will trigger it on iOS hardware (but works fine on the simulator). You may need to unmute audio on the "Record" bus, so that audio is input and output simultaneously: https://github.com/godotengine/godot-demo-projects/tree/master/audio/mic_record

[samgreen]

I think this is related to #34338 and #30308

Great work on the investigation and based on a first glance I would agree with your points

[bruvzg]

no microphone permission string in info.plist in XCode

Fixed in master, but is not backported to 3.1.2.

Maybe this should be configurable from the Godot export settings, or inferred from the sound bus usage?

We already have Enable audio input project setting, it should be fine to infer session category from it:

    if (GLOBAL_GET("audio/enable_audio_input")) {
        [[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayAndRecord error:nil];
    } else {
        [[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryAmbient error:nil];
    }

since there is a mismatch between the capture buffer format and the audio device.

What's input audio format on iOS devices?

On macOS Godot's audio recording works with the external microphone connected via 3.5 mm jack (24-bit int, 44.1 kHz), but not with virtual Background Music inputs (32-bit float, 48.0 kHz).

[taeyoonwf]

I guess nobody fixed this problem yet, so I made a pull request because I need to resolve this as soon as possible.

[zarmath]

II have the same problem. I have a program that records audio with the microphone and then plays it back. It works perfectly on all operating systems except when I export it to iOS

[taeyoonwf]

@zarmath You can test my pull request here if you want to. You can modify a few files and follow this instruction. Then the .a files for the iPhone export template will come out. And then, you can download the latest official iPhone export template, replace .a files in it, and copy the zip file to /Users/username/Library/Application Support/Godot/templates/the_version/. That's all.

[erikandershed]

Can’t record on iOS @taeyoonwf I have the same problem and I do rum 3.3.4. Can you please make a zip from you pull. I can't do it. Big Big thanks!

Microphone on macOS @bruvzg You need to make the sample rate on you microphone and godot to match. Check you midi setting to see the sample rate for the microphone and then add the same in godot project. And the it works.

[taeyoonwf]

@erikandershed My suggestion is don't use godot in order to use iPhone microphone recording. I realized the recording quality was also poor after I fixed the bug. I fixed a lot to make godot able to achieve the same quality compared to basic iOS microphone recording examples. You can check my repository here. It's quite a mess, and it includes other things not related to the iOS recording. Plus, you will have to rebuild godot. The manuel of it was not clear, so I had to struggle so much. I don't recommend it for your mental health.

[erikandershed]

@taeyoonwf I am designen a game where the sound quality is not importent. Is there a way for me to get a zip of the done build Godot. I have no idea how to do this from your repository. Please

[taeyoonwf]

@erikandershed Even I have never made a build of version 3.3.4. As far as I know, the export template and the tool must have the same version, so the only way to make a successful iOS export is that you fork and fix Godot source by yourself. I'm so sorry to say this, but it's true. Besides, the recording quality was not the only issue. The recorded wave source will have random high pitch noises quite often. I gave up fixing the issue by myself and made a new recording method instead. Good luck.

[erikandershed]

@taeyoonwf aa ok. Big thanks to you for taking the time to answer to my question.

[audiosdroid]

We developed an iOS Audio Godot plugin named AudiosPlugin that solves this issue. Please see our GitHub repo: https://github.com/audiosdev/AudiosPlugin-Godot-iOS-Audio-Plugin/

[audiosdev]

I developed a Godot iOS plugin that solves the Godot audio recording issue and playback issue in iOS devices. Please see the GitHub repo:

https://github.com/audiosdev/AudiosPlugin-Godot-iOS-Audio-Plugin/

Thanks!

[akien-mga]

Is this still reproducible in 4.0 RC 3 or later?

[Ughuuu]

I can reproduce this in 4.2 on iOS.

https://forum.godotengine.org/t/getting-error-audiooutputunitstart-failed-when-exporting-to-ios-and-using-microphone/36857