[BUG]: Not getting audio stream raw data in iOS

[dhalloop7]

Platform - iOS - 14.3 Severity - Crash and Issue

I am trying to get audio stream raw data for live stream audio over the network. For Android, it’s working perfectly. but for iOS, it’s not returning any audio stream and it’s data.

Please look in my code and required all logs.

Future<FlutterSoundRecorder> openRecorderAudioSession(
    {AudioFocus focus = AudioFocus.requestFocusAndKeepOthers,
    SessionCategory category = SessionCategory.playAndRecord,
    SessionMode mode = SessionMode.modeDefault,
    int audioFlags = outputToSpeaker,
    AudioDevice device = AudioDevice.speaker}) async {
  PermissionStatus permissionStatus = await Permission.microphone.status;
  print("OPEN RECORDER SESSION");
  if (permissionStatus.isGranted) {
    return await _soundRecorder?.openAudioSession(
        mode: mode,
        audioFlags: audioFlags,
        category: category,
        device: device,
        focus: focus);
  } else if (permissionStatus.isDenied || permissionStatus.isUndetermined) {
    await Permission.microphone.request();
    return null;
  } else {
    await openAppSettings();
    return null;
  }
}

Future<void> startRecorder(
    {Codec codec = Codec.defaultCodec,
    String toFile = null,
    StreamSink<Food> toStream = null,
    int sampleRate = 44100,
    int numChannels = 1,
    int bitRate = 16000,
    AudioSource audioSource = AudioSource.defaultSource}) async {
  print("STARTING RECORDER");
  return await _soundRecorder?.startRecorder(
      toStream: toStream,
      sampleRate: sampleRate,
      bitRate: bitRate,
      numChannels: numChannels,
      codec: codec,
      toFile: toFile,
      audioSource: audioSource);
}

void startRecord() async {
    _audioStreamName = DateTime.now().toUtc().millisecondsSinceEpoch.toString();

    FlutterSoundRecorder status =
        await MySoundRecorderPlayer.instance.openRecorderAudioSession();

    var _recordingDataController = StreamController<Food>();

    _micBytesSubscription = _recordingDataController.stream.listen((buffer) {
      print("GOT BUFFER $buffer");
      if (buffer is FoodData) {
        print("I AM HERE------- ${buffer.data.length}");
        if (buffer == null) {
          buffer.data = Uint8List(0);
        }
        if (buffer != null && buffer.data.isNotEmpty)
          recordedData.addAll(buffer.data);
        sendAudioMessage(buffer.data);
      }
    });

    if (status != null) {
      await MySoundRecorderPlayer.instance
          .startRecorder(
              numChannels: 1,
              sampleRate: 44100,
              codec: Codec.pcm16,
              toStream: _recordingDataController.sink);
      isRecording = MySoundRecorderPlayer.instance.soundRecorder.isRecording;
    }
  }

flutter pub deps | grep flutter_sound
|-- flutter_sound 7.5.3+1
|   |-- flutter_sound_platform_interface 7.5.3+1
|   |-- flutter_sound_web 7.5.3+1
|   |   |-- flutter_sound_platform_interface...

Logs from XCODE Before Play audio from URL using audioplayers

2021-01-27 18:29:34.758715+0530 Runner[12921:1650581] flutter: ADDED STATUS:::: AmerAudioStreamStatus.RECORDING 2021-01-27 18:29:34.785665+0530 Runner[12921:1650581] flutter: OPEN RECORDER SESSION 2021-01-27 18:29:34.787897+0530 Runner[12921:1650343] IOS:--> setAudioFocus 2021-01-27 18:29:34.788688+0530 Runner[12921:1650343] IOS:--> initializeFlautoPlayer 2021-01-27 18:29:34.971545+0530 Runner[12921:1650343] IOS:<-- setAudioFocus 2021-01-27 18:29:34.973374+0530 Runner[12921:1650581] flutter: STARTING RECORDER 2021-01-27 18:29:35.649576+0530 Runner[12921:1650575] convertToBuffer: error (null) 2021-01-27 18:29:35.649764+0530 Runner[12921:1650575] convertToBuffer: error (null) 2021-01-27 18:29:36.041564+0530 Runner[12921:1650575] convertToBuffer: error (null) 2021-01-27 18:29:36.041866+0530 Runner[12921:1650575] convertToBuffer: error (null) 2021-01-27 18:29:36.436381+0530 Runner[12921:1650575] convertToBuffer: error (null) 2021-01-27 18:29:36.436874+0530 Runner[12921:1650575] convertToBuffer: error (null) 2021-01-27 18:29:36.830745+0530 Runner[12921:1650575] convertToBuffer: error (null) 2021-01-27 18:29:36.831037+0530 Runner[12921:1650575] convertToBuffer: error (null) 2021-01-27 18:29:37.249061+0530 Runner[12921:1650575] convertToBuffer: error (null) 2021-01-27 18:29:37.249551+0530 Runner[12921:1650575] convertToBuffer: error (null) 2021-01-27 18:29:37.643821+0530 Runner[12921:1650575] convertToBuffer: error (null) 2021-01-27 18:29:37.644307+0530 Runner[12921:1650575] convertToBuffer: error (null) 2021-01-27 18:29:38.037730+0530 Runner[12921:1650575] convertToBuffer: error (null) 2021-01-27 18:29:38.038047+0530 Runner[12921:1650575] convertToBuffer: error (null) 2021-01-27 18:29:38.432738+0530 Runner[12921:1650575] convertToBuffer: error (null) 2021-01-27 18:29:38.433369+0530 Runner[12921:1650575] convertToBuffer: error (null) 2021-01-27 18:29:38.850708+0530 Runner[12921:1650575] convertToBuffer: error (null)

Logs from XCODE after Play audio from URL using audioplayers And Crashing

2021-01-27 18:18:29.264193+0530 Runner[12812:1645697] flutter: ADDED STATUS:::: AmerAudioStreamStatus.RECORDING 2021-01-27 18:18:29.267173+0530 Runner[12812:1645697] flutter: OPEN RECORDER SESSION 2021-01-27 18:18:29.267708+0530 Runner[12812:1645697] flutter: STARTING RECORDER 2021-01-27 18:18:29.297139+0530 Runner[12812:1645440] [aurioc] AURemoteIO.cpp:1095:Initialize: failed: -10851 (enable 1, outf< 2 ch, 0 Hz, Float32, non-inter> inf< 2 ch, 0 Hz, Float32, non-inter>) 2021-01-27 18:18:29.297688+0530 Runner[12812:1645440] [avae] AVAEInternal.h:76 required condition is false: [AVAEGraphNode.mm:823:CreateRecordingTap: (IsFormatSampleRateAndChannelCountValid(format))] 2021-01-27 18:18:29.302495+0530 Runner[12812:1645440] Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: IsFormatSampleRateAndChannelCountValid(format)' First throw call stack: (0x1a4f379d8 0x1b92a0b54 0x1a4e4650c 0x1b4e84984 0x1b4ee2d0c 0x1b4ecab3c 0x1b4f47de8 0x1b4f291e4 0x1102d8094 0x1102d860c 0x1102d776c 0x1012e71f8 0x1012e7bb8 0x1042ea224 0x103db1f9c 0x104089d88 0x103dbb42c 0x103dbd3a0 0x1a4eb83e0 0x1a4eb7fe4 0x1a4eb74c4 0x1a4eb1850 0x1a4eb0ba0 0x1bbbee598 0x1a77a03d8 0x1a77a5958 0x100e75370 0x1a4b8f568) libc++abi.dylib: terminating with uncaught exception of type NSException *** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: IsFormatSampleRateAndChannelCountValid(format)' terminating with uncaught exception of type NSException

[Larpoux]

Could you try with : const int tSampleRate = 44000; ?

I had several problems that I do not really understand with several sample rates on iOS. This is something that must be investigated more.

Please let us know if better ...

[dhalloop7]

It worked with 44000 sample rate for iOS and the recording quality is also good but the crash is still there.

I got a log again for the same. Am I doing something wrong?

Here is the code for stop the recorder as well.

  Future<void> stopRecorder() async {
    return await _soundRecorder?.stopRecorder();
  }

Future<void> closeRecorderAudioSession() async {
     return await _soundRecorder.closeAudioSession();
  }

void stopRecorder() async {
    if (_micBytesSubscription != null) {
      await _micBytesSubscription.cancel();
      _micBytesSubscription = null;
    }

    if (isRecording) {
      await MySoundRecorderPlayer.instance.stopRecorder();
      // await MySoundRecorderPlayer.instance.closeRecorderAudioSession();
      sendAudioMessage(Uint8List(0));
    }
    isRecording = MySoundRecorderPlayer.instance.soundRecorder.isRecording;
  }

[Larpoux]

I do not understand :

      sendAudioMessage(Uint8List(0));

and

       buffer.data = Uint8List(0);

[dhalloop7]

sendAudioMessage(Uint8List(0)); It's my internal function to manage that stream is over.

buffer.data = Uint8List(0); it's for if my recording buffer / raw data is null so I can manage for stream status. As we are managing sound stream recording and storing files on the server from the raw bytes and play the live stream on another device.

[Larpoux]

I am not sure that Flutter Sound will be happy if you feed him with an empty array. But I read your code very quickly and maybe I did not understood.

[dhalloop7]

After giving empty array / Uint8List(0) it's for our internal use. I'll not let Flutter sound unhappy with it.

But I am not sure why it's crashing? When I am trying to record again after play the URL with audioplayers plugin

[Larpoux]

Your crash is because iOS does not like the bytes that we give it to be translated to Float32 (iOS works with PCM-Float32, Apple never does like others).

I am busy, now, and I cannot help you very much. Perhaps you can run your App under XCode, and post the XCode logs.

Sorry not to help you more.

[dhalloop7]

Below logs are only from XCODE. There is nothing related to an empty array. That empty array is for my internal use only which is not related to any platform-specific.

Logs from XCODE after Play audio from URL using audioplayers And Crashing

2021-01-27 18:18:29.264193+0530 Runner[12812:1645697] flutter: ADDED STATUS:::: AmerAudioStreamStatus.RECORDING 2021-01-27 18:18:29.267173+0530 Runner[12812:1645697] flutter: OPEN RECORDER SESSION 2021-01-27 18:18:29.267708+0530 Runner[12812:1645697] flutter: STARTING RECORDER 2021-01-27 18:18:29.297139+0530 Runner[12812:1645440] [aurioc] AURemoteIO.cpp:1095:Initialize: failed: -10851 (enable 1, outf< 2 ch, 0 Hz, Float32, non-inter> inf< 2 ch, 0 Hz, Float32, non-inter>) 2021-01-27 18:18:29.297688+0530 Runner[12812:1645440] [avae] AVAEInternal.h:76 required condition is false: [AVAEGraphNode.mm:823:CreateRecordingTap: (IsFormatSampleRateAndChannelCountValid(format))] 2021-01-27 18:18:29.302495+0530 Runner[12812:1645440] Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: IsFormatSampleRateAndChannelCountValid(format)' First throw call stack: (0x1a4f379d8 0x1b92a0b54 0x1a4e4650c 0x1b4e84984 0x1b4ee2d0c 0x1b4ecab3c 0x1b4f47de8 0x1b4f291e4 0x1102d8094 0x1102d860c 0x1102d776c 0x1012e71f8 0x1012e7bb8 0x1042ea224 0x103db1f9c 0x104089d88 0x103dbb42c 0x103dbd3a0 0x1a4eb83e0 0x1a4eb7fe4 0x1a4eb74c4 0x1a4eb1850 0x1a4eb0ba0 0x1bbbee598 0x1a77a03d8 0x1a77a5958 0x100e75370 0x1a4b8f568) libc++abi.dylib: terminating with uncaught exception of type NSException *** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: IsFormatSampleRateAndChannelCountValid(format)' terminating with uncaught exception of type NSException

[Larpoux]

Do you use audioplayers to play your data ? Be aware that Flutter Sound is actually only:

• monophony (not stereo)
• INT16

[Larpoux]

(enable 1, outf< 2 ch, 0 Hz, Float32, non-inter> inf< 2 ch, 0 Hz, Float32, non-inter>)

Flutter Sound is mono and int16

[Larpoux]

And little Endian

[dhalloop7]

I am playing those raw bytes using Flutter Sound only. audioplayers is only for play audio from the URL

[dudizimber]

@dhalloop7 Did you manage to solve it?

[Larpoux]

For your info: I am currently working on a new project : Taudio. This project is everything about the W3C Web Audio API recommandation.

It is supposed to replace one day all the Flutter Sound verbs related to Audio Streams. I have great expectations for this plugin. But this is a major work and unfortunately I am a little bit slow in my development. Be patient : I am pretty sure that it will be a terrific library 😄

[dudizimber]

@Larpoux That sounds amazing! (Pun not intended) But is there a way to solve this bug for now on iOS?

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Leave a comment or this will be closed in 7 days.