tauri-apps / tauri

Build smaller, faster, and more secure desktop and mobile applications with a web frontend.
https://tauri.app
Apache License 2.0
85.15k stars 2.57k forks source link

[bug] Access the microphone fron Rust on macos #9928

Closed jafioti closed 5 months ago

jafioti commented 5 months ago

Describe the bug

I'm trying to record audio from rust, using cpal on macos. It seems to work correctly when the app is in dev mode, or built but not signed. Once the app is signed, even with the following Info.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>NSMicrophoneUsageDescription</key>
    <string>Request microphone access for voice input</string>
</dict>
</plist>

the permission popup does not appear and the microphone does not record any audio.

Is it possible to access the microphone from rust, or does it need to be done on the JS side?

Reproduction

Create a new app with an Info.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>NSMicrophoneUsageDescription</key>
    <string>Request microphone access for voice input</string>
</dict>
</plist>

and use cpal to start recording:

let device = cpal::default_host()
    .devices()
    .unwrap()
    .find(|d| d.name().unwrap() == "MacBook Pro Microphone")
    .unwrap();
let config = device.default_input_config().unwrap();
let sample_rate = config.sample_rate().0;
let channels = config.channels();

let clip = Arc::new(Mutex::new(Some(vec![])));
let clip_2 = clip.clone();

let stream = device
    .build_input_stream(
        &config.into(),
        move |data, _| write_input_data(data, &clip_2),
        move |err| eprintln!("an error occurred on stream: {err}"),
        None,
    )
    .unwrap();

stream.play().unwrap();

Expected behavior

This should request access to the microphone, show the recording icon in the rop right and start recording audio from the microphone, but when the app is built and signed none of that happens.

Full tauri info output

[✔] Environment
    - OS: Mac OS 14.5.0 X64
    ✔ Xcode Command Line Tools: installed
    ✔ rustc: 1.78.0 (9b00956e5 2024-04-29)
    ✔ cargo: 1.78.0 (54d8815d0 2024-03-26)
    ✔ rustup: 1.27.1 (54dd3d00f 2024-04-24)
    ✔ Rust toolchain: stable-aarch64-apple-darwin (environment override by RUSTUP_TOOLCHAIN)
    - node: 22.2.0
    - npm: 10.8.0

[-] Packages
    - tauri [RUST]: 2.0.0-beta.20
    - tauri-build [RUST]: 2.0.0-beta.16
    - wry [RUST]: 0.40.0
    - tao [RUST]: 0.28.0
    - tauri-cli [RUST]: 2.0.0-beta.16
    - @tauri-apps/api : not installed!
    - @tauri-apps/cli [NPM]: 2.0.0-beta.16

[-] App
    - build-type: bundle
    - CSP: default-src blob: data: filesystem: ws: wss: http: https: tauri: 'unsafe-eval' 'unsafe-inline' 'self' img-src: 'self'; connect-src ipc: http://ipc.localhost
    - frontendDist: ../dist
    - devUrl: http://localhost:1420/
    - framework: SolidJS
    - bundler: Vite

Stack trace

No response

Additional context

No response

jafioti commented 5 months ago

Ok I worked out the issue. The solution was found here: https://github.com/tauri-apps/tauri/issues/8314#issuecomment-1829864033

You basically need a seperate entitlements file with the following:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>com.apple.security.device.microphone</key>
    <true/>
    <key>com.apple.security.device.audio-input</key>
    <true/>
  </dict>
</plist>

and need to reference that file in your tauri conf

louis030195 commented 3 months ago

@jafioti thanks for this

by any chance do you know how to use audio screen capture? https://github.com/RustAudio/cpal/pull/894

like which permissions i need? i can correctly record mac microphone but get denied display audio recording

tried bunch of things

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>NSMicrophoneUsageDescription</key>
    <string>This app requires microphone access to record audio.</string>
    <key>NSScreenCaptureUsageDescription</key>
    <string>This app requires screen capture access to record the screen.</string>
    <!-- <key>NSSystemExtensionUsageDescription</key>
    <string>This app requires system extension access to capture audio output.</string> -->
</dict>
</plist>
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <!-- <key>com.apple.security.device.audio-input</key>
    <true/> -->
    <!-- <key>com.apple.security.temporary-exception.audio-unit-host</key>
    <true/> -->
    <!-- <key>com.apple.security.temporary-exception.mach-lookup.global-name</key>
    <array>
        <string>com.apple.audio.coreaudiod</string>
        <string>com.apple.audio.audiohald</string>
    </array> -->
    <!-- <key>com.apple.security.device.camera</key>
    <true/>
    <key>com.apple.security.device.microphone</key>
    <true/>
    <key>com.apple.security.files.user-selected.read-write</key>
    <true/>
    <key>com.apple.security.screen-capture</key>
    <true/>
    <key>com.apple.developer.persistent-content-capture</key>
    <true/>
    <key>com.apple.developer.sustained-execution</key>
    <true/> -->
    <!-- <key>com.apple.security.temporary-exception.sbpl</key>
    <string>(allow mach-lookup (global-name-regex #"^com.apple.coreservices.launchservicesd$"))</string>

    <key>com.apple.security.cs.allow-jit</key>
    <true/>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <key>com.apple.security.cs.allow-dyld-environment-variables</key>
    <true/>
    <key>com.apple.security.cs.disable-library-validation</key>
    <true/>
    <key>com.apple.security.automation.apple-events</key> -->
</dict>
</plist>
thewh1teagle commented 3 months ago

I had the same issue with mic permission. fixed in https://github.com/thewh1teagle/vibe/commit/f30d6d61d0bcae7cb63f8b75b3adea2e4695ff71 Works for both mic and screen audio recording.