search

Canardoux / flutter_sound

Flutter plugin for sound. Audio recorder and player.
Mozilla Public License 2.0
877 stars 573 forks source link

[BUG]: Recording does not work when using non-Airpod Pro Bluetooth Headphones with iOS 15.4.x device #885

Closed ericadu closed 2 years ago

ericadu commented 2 years ago

Flutter Sound Version :

Severity

Platforms you faced the error

Describe the bug A clear and concise description of what the bug is.

Recording to stream does not capture any audio when using bluetooth audio devices that are not Airpod Pros. Tested using real iOS devices 15.4.x.

Tested using the following bluetooth headphones:

To Reproduce Steps to reproduce the behavior:

  1. Pair bluetooth headphones (that are not Airpod Pros) to iOS device
  2. Try recording to stream

Observation: No FoodData is being captured.

Error Domain=NSCocoaErrorDomain Code=3840 "Unable to parse empty data." UserInfo={NSDebugDescription=Unable to parse empty data.}

Logs!!!!

(This is very important. Most of the time we cannot do anything if we do not have information on your bug). To activate the logs, you must instantiate your modules with the Log Level set to Level.debug :

FlutterSoundPlayer myPlayer = FlutterSoundPlayer(logLevel: Level.debug);
FlutterSoundRecorder myRecorder = FlutterSoundRecorder(logLevel: Level.debug);

See this

Some potentially helpful code snippets.

I created a helper service to manage audio sessions and to print out the current routes. initSession gets called in my initState component of my recorder widget

import 'package:audio_session/audio_session.dart';

class AudioSessionService {
  Future<void> initSession() async {
    final session = await AudioSession.instance;
    await session.configure(AudioSessionConfiguration(
      avAudioSessionCategory: AVAudioSessionCategory.playAndRecord,
      avAudioSessionCategoryOptions:
          AVAudioSessionCategoryOptions.allowBluetooth |
              AVAudioSessionCategoryOptions.allowBluetoothA2dp |
              AVAudioSessionCategoryOptions.defaultToSpeaker,
      avAudioSessionMode: AVAudioSessionMode.spokenAudio,
      avAudioSessionRouteSharingPolicy:
          AVAudioSessionRouteSharingPolicy.defaultPolicy,
      avAudioSessionSetActiveOptions: AVAudioSessionSetActiveOptions.none,
      androidAudioAttributes: const AndroidAudioAttributes(
        contentType: AndroidAudioContentType.speech,
        flags: AndroidAudioFlags.none,
        usage: AndroidAudioUsage.voiceCommunication,
      ),
      androidAudioFocusGainType: AndroidAudioFocusGainType.gain,
      androidWillPauseWhenDucked: true,
    ));
    print('setup');
    AVAudioSession().availableInputs.then((res) {
      res.forEach((element) {
        print(element.portName);
        print(element.portType);
        print(element.channels);
        print(element.dataSources);
        print(element.selectedDataSource);
        if (element.selectedDataSource != null) {
          print(element.selectedDataSource!.name);
        }
      });
    });

    AVAudioSession().currentRoute.then((res) {
      res.inputs.forEach((element) async {
        print('inputs');
        print(element.portName);
        print(element.portType);
      });
      res.outputs.forEach((element) {
        print('outputs');
        print(element.portName);
        print(element.portType);
      });
    });
  }
}

Recorder Widget


// ** Full code redacted for clarity, only including relevant parts
  @override
  void initState() {
    recordingDataController.stream.listen((FoodData data) {
      print("DATA");
    });
    _initAudio();
    super.initState();
  }

  void _initAudio() async {
    if (_mRecorder != null) {
      await _mRecorder!.openRecorder();
      await getIt<AudioSessionService>().initSession();
      if (mounted) {
        setState(() {
          _mRecorderIsInited = true;
        });
      }
    }
  }

  Future<void> _record() async {
    print('recording');
    AVAudioSession().currentRoute.then((res) {
      res.inputs.forEach((element) async {
        print('inputs');
        print(element.portName);
        print(element.portType);
      });
      res.outputs.forEach((element) {
        print('outputs');
        print(element.portName);
        print(element.portType);
      });
    });
}

LOGS OUTPUT

flutter: setup
flutter: PLAYING
flutter: iPhone Microphone
flutter: AVAudioSessionPort.builtInMic
flutter: []
flutter: [Instance of 'AVAudioSessionDataSourceDescription', Instance of 'AVAudioSessionDataSourceDescription', Instance of 'AVAudioSessionDataSourceDescription']
flutter: Instance of 'AVAudioSessionDataSourceDescription'
flutter: Front
flutter: Airpods ED
flutter: AVAudioSessionPort.bluetoothHfp
flutter: []
flutter: []
flutter: null
flutter: inputs
flutter: iPhone Microphone
flutter: AVAudioSessionPort.builtInMic
flutter: outputs
flutter: Airpods ED
flutter: AVAudioSessionPort.bluetoothA2dp
flutter: recording
flutter: ┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
flutter: │ #0   FlutterSoundRecorder.startRecorder (package:flutter_sound/public/flutter_sound_recorder.dart:590:13)
flutter: │ #1   _RecordButtonBarRealtimeState._record (package:app/screens/chat_detail/record/record_button_realtime.dart:293:12)
flutter: ├┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
flutter: │ 🐛 FS:---> startRecorder
flutter: └───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
flutter: ┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
flutter: │ #0   FlutterSoundRecorder._startRecorder (package:flutter_sound/public/flutter_sound_recorder.dart:614:13)
flutter: │ #1   FlutterSoundRecorder.startRecorder.<anonymous closure> (package:flutter_sound/public/flutter_sound_recorder.dart:592:13)
flutter: ├┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
flutter: │ 🐛 FS:---> _startRecorder.
flutter: └───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
flutter: inputs
flutter: Airpods ED
flutter: AVAudioSessionPort.bluetoothHfp
flutter: outputs
flutter: Airpods ED
flutter: AVAudioSessionPort.bluetoothHfp
flutter: ┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
flutter: │ #0   FlutterSoundRecorder.startRecorderCompleted (package:flutter_sound/public/flutter_sound_recorder.dart:234:13)
flutter: │ #1   MethodChannelFlutterSoundRecorder.channelMethodCallHandler (package:flutter_sound_platform_interface/method_channel_flutter_sound_recorder.dart:74:22)
flutter: ├┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
flutter: │ 🐛 ---> startRecorderCompleted: true
flutter: └───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
flutter: ┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
flutter: │ #0   FlutterSoundRecorder.startRecorderCompleted (package:flutter_sound/public/flutter_sound_recorder.dart:243:13)
flutter: │ #1   MethodChannelFlutterSoundRecorder.channelMethodCallHandler (package:flutter_sound_platform_interface/method_channel_flutter_sound_recorder.dart:74:22)
flutter: ├┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
flutter: │ 🐛 <--- startRecorderCompleted: true
flutter: └───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
flutter: ┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
flutter: │ #0   FlutterSoundRecorder._startRecorder (package:flutter_sound/public/flutter_sound_recorder.dart:689:13)
flutter: │ #1   <asynchronous suspension>
flutter: ├┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
flutter: │ 🐛 FS:<--- _startRecorder.
flutter: └───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
flutter: ┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
flutter: │ #0   FlutterSoundRecorder.startRecorder (package:flutter_sound/public/flutter_sound_recorder.dart:602:13)
flutter: │ #1   <asynchronous suspension>
flutter: ├┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
flutter: │ 🐛 FS:<--- startRecorder
flutter: └───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[tcp] tcp_input [C206.1:3] flags=[R] seq=2968491827, ack=0, win=0 state=LAST_ACK rcv_nxt=2968491827, snd_una=1392928498
[tcp] tcp_input [C206.1:3] flags=[R] seq=2968491827, ack=0, win=0 state=CLOSED rcv_nxt=2968491827, snd_una=1392928498
[tcp] tcp_input [C206.1:3] flags=[R] seq=2968491827, ack=0, win=0 state=CLOSED rcv_nxt=2968491827, snd_una=1392928498
[tcp] tcp_input [C206.1:3] flags=[R] seq=2968491827, ack=0, win=0 state=CLOSED rcv_nxt=2968491827, snd_una=1392928498

and then my backend service throws an error because I'm sending it empty bytes.

ericadu commented 2 years ago

I'm curious if it has to do with the fact that the protocol is AVAudioSessionPort.bluetoothHfp?

ericadu commented 2 years ago

Also unsure if actually an https://github.com/ryanheise/audio_session/issues issue.

ericadu commented 2 years ago

Update: I tried swapping flutter_sound for mic_stream in my app, and can confirm that it works with mic_stream. Current hypothesis is that it is a flutter_sound issue.

fallenpanda1 commented 2 years ago

Update on this issue (which I'm working with @ericadu on):

  1. Seems like this affects all 16Khz recording devices. I believe this generally means lower end mics, or bluetooth mics when poor signal quality is detected.
  2. Here's a PR with a fix: https://github.com/Canardoux/flutter_sound_core/pull/5
Larpoux commented 2 years ago

Allen, I am impressed by your bug fix. I am going to merge your Pull Request to the master branch. Thank you for your time. This is really terrific job.

ericadu commented 2 years ago

Thank you so much @fallenpanda1 and @Larpoux! Very much appreciate it 🙏🏼🙏🏼🙏🏼🙏🏼🙏🏼

Larpoux commented 2 years ago

@ericadu , @fallenpanda1

Allen : your Pull Request is now integrated inside Flutter Sound release 9.2.12 I am impressed that you was able to debug FlutterSoundCore without any help from me. If you had told me before, I would had help you to setup a development environment. Well done, Allen

Erica : can you try Flutter Sound release 9.2.12 and tell us if your issue is fixed ?

fallenpanda1 commented 2 years ago

If you had told me before, I would had help you to setup a development environment.

Ooh nice, thanks for the offer! I'm primarily an iOS engineer by trade, so it wasn't too hard for me to get the code up and running, haha. I'm more clueless around Flutter code though.