anarchuser / mic_stream

Flutter plugin to get an infinite audio stream from the microphone
https://pub.dev/packages/mic_stream
GNU General Public License v3.0
100 stars 68 forks source link

Unable to read stream settings (sample rate, bit depth...) before listening on 0.7.0-dev #69

Closed DurandA closed 11 months ago

DurandA commented 1 year ago

When trying to upgrade my project to 0.7.0-dev, I noticed than reading microphone stream settings (await MicStream.sampleRate, await MicStream.bitDepth, ...) do not complete until the stream is listened to.

For use-cases requiring stream settings in advance, is it possible to get settings without listening to the microphone and dropping samples until the mic is actually used?

anarchuser commented 1 year ago

Thank you for bringing up this suggestion. First of all, which platform are you using? The reason for using Futures as parameters is the Fact that iOS does not guarantee to use your configured parameters. On the other hand, the other supported platforms will always return the parameters you give in.

For the aforementioned reason, allowing the retrieval of stream settings on iOS without the stream actually starting would be detrimental - there could be no guarantee that the values don't change from before to after stream start.

On another note, you can get the default parameters and you know the stream settings you specify yourself, and prior to recording these won't change. Is there a specific reason you need to read these parameters before recording?

DurandA commented 1 year ago

Hi @anarchuser and thank you for your awesome package.

I am using mic_stream for both iOS and Android.

The current issue I have is that I delegate the stream processing to another component (taking stream parameters as input) which subscribe to the stream. I.e. something like this:

Future<bool> _startListening() async {
    _stream = MicStream.microphone(
        audioSource: AudioSource.DEFAULT,
        sampleRate: 16000,
        channelConfig: ChannelConfig.CHANNEL_IN_MONO,
        audioFormat: AUDIO_FORMAT);

    samplesPerSecond = (await MicStream.sampleRate)!.toInt();

    manager = AudioCodesManager(
      audioStream: _stream,
      sampleRateHz: samplesPerSecond,
    );

    setState(() {
      _isRecording = true;
    });
}

I also noticed that the Completers are recreated on subscribe: https://github.com/anarchuser/mic_stream/blob/528338d903c30110dc8170b86e62efdccded3731/lib/mic_stream.dart#L51-L61 https://github.com/anarchuser/mic_stream/blob/528338d903c30110dc8170b86e62efdccded3731/lib/mic_stream.dart#L151-L153

This makes the code error prone as if the Completer are inititialized before subscribing to the MicStream, it will await for the wrong Completer.

anarchuser commented 1 year ago

Thank you for the detailed response. First of all, yes, I will take another look at the initialisation of the getter Futures / Completers.

I'm not sure how easily I can improve the getter situation in general. The easiest way might be to read the first value myself and initialise the params thereafter. If you have a concrete idea, feel free to make suggestions.

DurandA commented 1 year ago

I spent a bit more time thinking about it and I am still unsure how it should be done.

If I may suggest some slight changes, I would try to avoid asyncExpand() if possible because it discards the multi-subscriber nature of receiveBroadcastStream.

https://github.com/anarchuser/mic_stream/blob/528338d903c30110dc8170b86e62efdccded3731/lib/mic_stream.dart#L108-L108

With the current implementation, I am not sure if stream deactivation would happen automatically if the resulting single-subscriber stream is converted to a broadcast stream and then the stream listener count drops to 0.

Also, the library subscribe to the mic stream and cancel the subscription as a way to make calls on first stream event: https://github.com/anarchuser/mic_stream/blob/528338d903c30110dc8170b86e62efdccded3731/lib/mic_stream.dart#L155-L157

Instead, I would prefer doing this as a side effect (e.g. using doOnData or similar).

anarchuser commented 1 year ago

sorry, took me some time to get back to this. Regarding the microphone permission handling, I tried different ways but this is the only way to keep the microphone() method synchronous. I think your point is of no concern, though - if permission is denied, asyncExpand will yield a single error from the throw and then terminate the whole closure. Regardless of whether the stream has listeners, _setupMicStream() won't ever be called unless a new call to microphone() succeeds in requesting permissions.

Your second point means adding a new dependency, but it does seem like it'd be worth it. I'll give it a deeper look.

anarchuser commented 1 year ago

I also noticed that the Completers are recreated on subscribe:

I addressed this now in 7cd6aa8e36620e3790d1d4ca1408e3e2251db3c3 by setting the old Completers to the future of the new ones. I'm not sure whether this is appropriate though; do you think the old Completers should receive a StateError instead?

anarchuser commented 1 year ago

I tried replacing the listener Code with the rxdart library (i.e., doOnListen), but to no success. I couldn't get it to work whatsoever. Feel free to try it out yourself if you want; otherwise it'll stay as-is.

As for your initial request - I've thought about it, and maybe there is a way to first set the initial parameters, and then later the actually used ones. However, I still don't think this is a good idea. It would hide the discrepancy between given and used parameters in a very opaque way. Instead, I can offer you to add getters for your last used parameters if you wish. You can then still assemble an amalgamation that gives your given parameters until the used parameters return.

anarchuser commented 11 months ago

This should be fixed now with version 0.7.1. For now, I changed it to a more synchronous approach again so there are no discrepancies between configured and actual values. An appropriate asynchronous approach would require some restructuring in the native parts of the project which may or may not become part of a future update.

If issues persist please reopen. And please feel free to give feedback.