docs: Document that iOS app gives a warning to the user about microphone permission -- but only in debug mode and not in production

[filiph]

Description

An app that calls SoLoud.init() (or maybe just imports flutter_soloud?) will give a warning to the user about microphone permission.

NOTE: I need to look at this more thoroughly but I wanted to file the issue so that I don't forget, or in case it's an easy fix.

Steps To Reproduce

1. Create an app
2. Import flutter_soloud
3. (possibly not needed?) Call Soloud.instance.init() at app start
4. Run on macOS

Expected Behavior

App runs immediately

Screenshots

TODO - this doesn't happen on subsequent runs of the app, so I can't easily reproduce right now

Additional Context

This, to me, is a major issue. Most game devs don't need microphone input, and yet their games will ask for microphone permissions on startup. This looks shady ("why does this game need to hear my mic input?"). I'd go as far as to say that, if this cannot be worked around, let's just drop the Capture functionality altogether (e.g. move it to a fork of flutter_soloud) — it's a very niche functionality and it's not worth the churn of developers who understandably avoid an audio engine that makes it look like they're spying on their players.

[alnitak]

This is weird! There are no calls from within SoLoud to SoLoudCapture that could trigger the use of a microphone. Also, AFAIK, the permission should be explicitly written in the Info.plist and asked in the app code. I also looked at macos/flutter_soloud.podspec and there are no references to the microphone.

We could try to remove all the deprecated capture references in soloud.dart but I highly doubt that this is the problem.

[filiph]

Yeah. I wonder if it's just something on the SoLoud C++ side "touching" an API, or even just declaring that it will be using some API, and macOS (Xcode?) detecting that. I need to experiment more.

[alnitak]

Since the capture feature is all made using only miniaudio, and the permission dialog should be popped with pkg:permission_handler, I really think this could be an OS "problem" that thinks that some audio IO is requested. Don't know much about mc OS, but I found something that I hope may help here

[filiph]

Thanks for looking into it! I think the salient parts are:

[when] Core Audio opens full duplex devices for both input and output

(I do have an external sound card that does both input and output in one box. I think this is quite rare, fortunately.)

And:

the dialog can be avoided by setting com.apple.security.device.audio-input to false in entitlements file. I'll try that.

I'll look into this — setting audio-input to false explicitly.

[LeeMatthewHiggins]

I just hit this issue. I have refactored my audio engine replacing the audioplayers package (it has many issues looping and playing multiple sounds on Android). I have a game on the store, I'm about to do a release and I have an issue because now the game requires a string to justify why it uses a mic. I cannot justify that, and don't want the prompt it will invoke. Another related issue is I'm not able to set the audio context mode. The game previously used mix with others on iOS so you can listen to music while playing the game, however I've not been able to set that, even using another package (it get overridden by so loud). This is currently a blocker for me, how can I help?

[filiph]

This looks like a high priority for the package for sure.

For the audio context mode, can you please create a separate issue, @LeeMatthewHiggins?

As for the mic permissions — can you confirm that this mic permission is really triggered in production? I was under the impression it won't.

In parallel, I have a question for you, @alnitak. If we stripped out all the code that accesses the mic (which I thing is worth it to unlock the most common use case for games), would that help the issue? Or is SoLoud (C++) just always going to require the mic permission?

[alnitak]

@LeeMatthewHiggins if I understand well does this happen on iOS only? AFAIK the mic permission is asked when in the info.plist there is <key>NSMicrophoneUsageDescription</key>. Are you able to see when exactly in your code it asks for mic permission? Maybe when you call SoLoud.init()?

If we stripped out all the code that accesses the mic (which I thing is worth it to unlock the most common use case for games), would that help the issue? Or is SoLoud (C++) just always going to require the mic permission?

@filiph are you able to reproduce this issue on iOS? If so we can try to get rid of capture things in the plugin and try, but TBH I don't think that will resolve this issue (it's worth a try anyway). SoLoud C++ itself doesn't make use of the mic. The capture logic is separated from SoLoud. The capture is performed only using miniaudio and the initialization is performed when explicitly calling SoLoudCapture.initialize().

[LeeMatthewHiggins]

@LeeMatthewHiggins if I understand well does this happen on iOS only? AFAIK the mic permission is asked when in the info.plist there is <key>NSMicrophoneUsageDescription</key>. Are you able to see when exactly in your code it asks for mic permission? Maybe when you call SoLoud.init()?

If we stripped out all the code that accesses the mic (which I thing is worth it to unlock the most common use case for games), would that help the issue? Or is SoLoud (C++) just always going to require the mic permission?

@filiph are you able to reproduce this issue on iOS? If so we can try to get rid of capture things in the plugin and try, but TBH I don't think that will resolve this issue (it's worth a try anyway). SoLoud C++ itself doesn't make use of the mic. The capture logic is separated from SoLoud. The capture is performed only using miniaudio and the initialization is performed when explicitly calling SoLoudCapture.initialize().

So specifically in my case, I only noticed when uploading a build to the app store. I get an email saying there is an issue with your app upload. see attached.

The problem is I have no justification for using the mic. (the previous version using audioplayers did not have this issue).

Important: Come to think of it, I don´t actually think I had the in app prompt for mic (I never use the mic features in the game). I have checked and the app in my ios permissions settings has no entry for mic permissions, So this might not be an issue for the prompt, but I still can´t set the audio mode to the one I want. I think that might be releated, as the audio mode will determine what permissions you need. i.e playAndRecord will require mic.

[filiph]

Ok, from the email, it seems that the App Store does some static analysis on the app's code (or binary?), and it finds the reference to APIs.

@alnitak, is it feasible to make a version of SoLoud that doesn't have that ability? I don't think there's anything we can do on the Dart side that will have an effect (as the reference to the mic API will still be in the C++ library). But maybe we can build SoLoud (C++) in a way that doesn't have this capability? Like with a compilation flag?

[alnitak]

@filiph , @LeeMatthewHiggins I created dev/noCapture branch without the capture feature. All the cpp and Dart sources for the capture have been removed. Can you please try with that branch? To try, just replace flutter_soloud: ^x.y.z with

flutter_soloud:
  git:
    url: git://github.com/alnitak/flutter_soloud.git
    ref: dev/noCapture

in the dependencies: of pubspec.yaml.

While writing this, I saw the above comment to appear :). So is it possible to check for this issue without performing an upload to the store?

[LeeMatthewHiggins]

@filiph , @LeeMatthewHiggins I created dev/noCapture branch without the capture feature. All the cpp and Dart sources for the capture have been removed. Can you please try with that branch? To try, just replace flutter_soloud: ^x.y.z with

flutter_soloud:
  git:
    url: git://github.com/alnitak/flutter_soloud.git
    ref: dev/noCapture

in the dependencies: of pubspec.yaml.

While writing this, I saw the above comment to appear :). So is it possible to check for this issue without performing an upload to the store?

Trying now, will upload to the store and test. Needed to alter the pubspec code to: flutter_soloud: git: ref: noCapture url: https://github.com/alnitak/flutter_soloud.git

[alnitak]

Thank you @LeeMatthewHiggins

@alnitak, is it feasible to make a version of SoLoud that doesn't have that ability? I don't think there's anything we can do on the Dart side that will have an effect (as the reference to the mic API will still be in the C++ library). But maybe we can build SoLoud (C++) in a way that doesn't have this capability? Like with a compilation flag?

@filiph I don't see any compiler flag related to capture with the mic. The only things that come to my mind, are the frameworks used in the flutter_soloud.podspec in this line: s.ios.framework = ['AudioToolbox', 'AVFAudio'] Maybe the store has seen those frameworks and just presupposes that the app uses the mic(?).

[LeeMatthewHiggins]

No Luck I´m affraid. I want to check its nothing else doing it, my app is quite complex. I use rive.app framework and they recently added audio and i checked and they use mini audio. I will confirm its not them.

[filiph]

Thanks for exploring this, @LeeMatthewHiggins! Would you be able to stub out the audio with a no-op class, and check whether removing flutter_soloud altogether makes a difference? That would make it crystal clear where the problem is.

I don't see any compiler flag related to capture with the mic. The only things that come to my mind, are the frameworks used in the flutter_soloud.podspec in this line: s.ios.framework = ['AudioToolbox', 'AVFAudio']

@alnitak Do you know if we need both for playing? @LeeMatthewHiggins, can you try directly removing one or the other from your build?

[LeeMatthewHiggins]

Yes I have an abstraction layer that lets me swap our audioplayers, solound and no-op I will get back to you ASAP. Rive app added audio at around the same time so want to exclude that from the mix too.

[alnitak]

Do you know if we need both for playing?

@filiph I don't remember why I used both, I am googling but I can't find info for now. But I saw that "AVFAudio" is for both playback and recording

[LeeMatthewHiggins]

OK. please don´t kill me.

It looks like it was actually audio_session package. I added the audio session package so that I could set the mix with others audio session mode, as I can´t see a way to do this in soloud. However this did not work as soloud seems to override it. BUT if I remove audio_session I don't get the warning from the app store upload!

SOOOOO the only issue I have, is I´m not able to set the audio session mode so that I can have mix with others (The usecase of using music player while playing my game).

Right now I use the no capture branch and don´t get the message from the app store. (I will try the master branch next)

[alnitak]

@LeeMatthewHiggins no worries! Thanks for your findings! Please let us know if, with the main branch, this issue doesn't come up. Just to figure out it's all ok.

[LeeMatthewHiggins]

OK confirmed. I have uploaded a build using flutter_soloud: ^2.0.1 and I do not have issue with the app store upload. And I do not get the mic permission prompt.

So my only remaining issue is how to set the audio session mode so it can mix with music players. Sorry if I created a wild goose chase.

[alnitak]

@LeeMatthewHiggins great! Thank you.

For the audio session mode, can you please open a new issue telling us your findings also when using audio_session?

@filiph do you still need this issue open? Feel free to close if you think it's worth it.

[LeeMatthewHiggins]

@LeeMatthewHiggins great! Thank you.

For the audio session mode, can you please open a new issue telling us your findings also when using audio_session?

@filiph do you still need this issue open? Feel free to close if you think it's worth it.

Done here https://github.com/alnitak/flutter_soloud/issues/86

[filiph]

I'm glad we figured it out!

@alnitak, I'm going to keep this issue open but rename the issue to: "Document that iOS app gives a warning to the user about microphone permission -- but only in debug mode and not in production".