Silence problem when I try to use with video streaming

[omerrciftcii]

Hello thank you so much for this package, very useful. I am trying add realtime transcription to video call. If I use this package during the video call, I get silence. Have you any advise for this issue?

[felixjunghans]

Hi, thanks for liking the package. I'm not sure I understand your problem yet. Can't you hear the person you are talking to or can the person you are talking to not hear you anymore? And how exactly do you pass the sound to the Google Speech Api. Since this package has no influence on the playback of sound, but only processes transferred audio content, I cannot imagine that the problem is caused by the google_speech. But maybe you can describe your problem in more detail or show with a code example how you pass the sound to google_speech. Then I will definitely try to help you.

I would also like to know if the transcription still works even if you do not hear any sounds.

[omerrciftcii]

I am using this package for microphone stream sound_stream.

For example I am on the video call. (I am using agora_rtc engine for video call), I have button in the video screen to turn on the live captions. When I click the button, I get this error about 10 seconds later: [ERROR:flutter/lib/ui/ui_dart_state.cc(166)] Unhandled Exception: gRPC Error (11, Audio Timeout Error: Long duration elapsed without audio. Audio should be sent close to real time.)

I am calling this function when I press the button: ` void streamingRecognize() async { await _recorder.start();

setState(() {
  recognizing = true;
});
final serviceAccount = ServiceAccount.fromString(
    '${(await rootBundle.loadString('assets/speakappbuddy-a3ff0cd894ff.json'))}');
final speechToText = SpeechToText.viaServiceAccount(serviceAccount);
final config = _getConfig();

final responseStream = speechToText.streamingRecognize(
    StreamingRecognitionConfig(config: config, interimResults: true),
    _recorder.audioStream);

responseStream.listen((data) {
  setState(() {
    text =
        data.results.map((e) => e.alternatives.first.transcript).join('\n');
    recognizeFinished = true;
  });
}, onDone: () {
  setState(() {
    recognizing = false;
  });
});

}` here is the _getConfig()

RecognitionConfig _getConfig() => RecognitionConfig( encoding: AudioEncoding.LINEAR16, model: RecognitionModel.basic, enableAutomaticPunctuation: true, sampleRateHertz: 16000, languageCode: 'en-US');

But if I use speech_to_text without video calling, it is working fine. Maybe the problem about I am using same microphone input for speech_to_text_api and video call sdk. Both of them try to use microphone at the same time.

Thank you for your interest about my issue

[felixjunghans]

Okay, now I understand your problem. First of all, it is not a problem that this package causes. The error you get back says that no audio content was sent to the Google Speech Api. For me this error always came up when no microphone input was detected. I share your suspicion that sound_stream can't pick up data from the microphone because the microphone is already used in the video call. I don't have a quick solution for this problem myself, but I like to keep the issue open in case someone else has an idea.

[omerrciftcii]

Thank you, if I figure out this problem, I will share here

[vhghorecha]

@omerrciftcii Did you manage to solve the issue? I am facing same problem.

[omerrciftcii]

@vhghorecha Actually it is about android restrictions, there is no problem with android 10. They allow to microphone share in android 10

[vhghorecha]

@omerrciftcii I am getting this issue in iOS only not in Android and in real device only

[omerrciftcii]

@vhghorecha unfortunately I don't know about IOS, how did you solve this problem for earlier versions?

[felixjunghans]

@vhghorecha @omerrciftcii Is there any progress on the problem or already a solution? Can the issue be closed?

[vhghorecha]

@felixjunghans No solution. It is not working parallel with video recording.

[omerrciftcii]

@felixjunghans I am closing the issue, the only solution is to use android 10 or higher. Thanks