search

google / oboe

Oboe is a C++ library that makes it easy to build high-performance audio apps on Android.
Apache License 2.0
3.64k stars 554 forks source link

Redmi Note 8 Ginkgo (and other Redmi devices) - microphone not recording when an application is already recording #1427

Open MarkoGill opened 2 years ago

MarkoGill commented 2 years ago

Android version(s): 11

Android device(s): Redmi Note 8 gingko. Some customer reports from other Xiaomi devices (I can dig deeper if needed).

Oboe version: 1.6.1

App name used for testing: Oboe Tester, and pre-installed Recorder app are used for testing. The issue reproduces in Smule.

Short description An application can spin up an Oboe input audio stream, and be unable to record. This requires that some other application first begins a recording.

Steps to reproduce

  1. Begin a recording in the Recorder app
  2. Background the Recorder app
  3. Open Oboe Tester
  4. Select Record and Play
  5. Press Record
  6. Press Stop
  7. Press Play
  8. optionally go back to your recorder app and stop its recording

Expected behavior

  1. Audio recording is paused in the Recorder app when Oboe Tester starts recording
  2. Oboe Tester is able to gain control of the audio session

This expected behavior is observed on the Pixel 3. Android 11.

Actual behavior

  1. The Recorder app continues recording when Oboe Tester is in the Recording phase.
  2. Oboe Tester does not record any audio

Device

ro.product.brand = xiaomi ro.product.manufacturer = Xiaomi ro.product.model = Redmi Note 8 ro.product.device = ginkgo ro.product.cpu.abi = arm64-v8a ro.build.description = ginkgo-user 11 RKQ1.201004.002 V12.0.2.0.RCOMIXM release-keys ro.hardware = qcom ro.hardware.chipname = ro.arch = | grep aaudio =

Any additional context

Redmi Example https://soundcloud.com/mark-gill-623185005/redmi-multiple-recordings-bug/s-APwxKq20QeF?si=c8f46b7f1d294dbda3ac762c1b157150

Pixel Example https://soundcloud.com/mark-gill-623185005/pixel-3-multiple-recordings-example/s-w4Uuwv8FrMe?si=c8f46b7f1d294dbda3ac762c1b157150

In these examples I narrate what I am doing on the phone during the process to illustrate the problem. The recordings are made with the Recorder app.

Notice the pixel example includes a drop out in recording. This is when Oboe Tester is recording.

This issue does not seem to be AAudio or OpenSL specific. I can provide more examples if necessary.

philburk commented 2 years ago

Thanks for the detailed report.

When you are recording in OboeTester, are the frame counts advancing and you record silence? Or are the frame counts frozen?

What happens if you disable "MMAP" checkbox in OboeTester before you press RECORD?

MarkoGill commented 2 years ago

Hi Phil,

frame counts the fields written, read, and cb will advance while recording. Silence is recorded in Oboe Tester (and also Smule).

MMAP There's a few things to note here.

On this device, MMAP is unticked and unselectable on a fresh launch of Oboe Tester. After the first recording, MMAP becomes selectable. If I select it, then press record, the app becomes unresponsive for 5-10 seconds and then starts recording. It has all the same symptoms whether MMAP is enabled or disabled.

During this process of testing the Redmi stopped working in Oboe Tester altogether, even when only recording from a single application. I had closed the Recorder app (by swiping it from the open applications tray) while it was recording. It seemed to still maintain control of the audio session. I relaunched Recorder from the main Android screen and it was still recording, but before this relaunch it was not visible from the open applications tray.

Let me know if more information is needed.

philburk commented 2 years ago

On this device, MMAP is unticked and unselectable on a fresh launch of Oboe Tester. After the first recording, MMAP becomes selectable.

Oh! That sounds like an Oboe bug. The MMAP button should not be enabled. I was able to reproduce this. Thanks. It is a separate issue from the recording interference.

So the Redmi Note 8 does not support MMAP. That is helpful to know.

MarkoGill commented 2 years ago

The same recording process applied to the Samsung S20 has a different result. There is an extreme distortion when returning to the Recorder app. Apologies if you'd prefer this to be in a separate ticket.

VOLUME WARNING: be careful with your ears on this one https://soundcloud.com/mark-gill-623185005/samsung-s20-2-apps-recording/s-ecbBPCN6m4k?si=3404c4cac7074d83b2396f715a50dff9

ro.product.brand = samsung ro.product.manufacturer = samsung ro.product.model = SM-G981W ro.product.device = x1q ro.product.cpu.abi = arm64-v8a ro.build.description = x1qcsx-user 11 RP1A.200720.012 G981WVLU2DUH1 release-keys ro.hardware = qcom ro.hardware.chipname = SM8250 ro.arch = | grep aaudio = [aaudio.hw_burst_min_usec]: [2000]

EDIT: This behavior does not happen if I select OpenSL in Oboe Tester

philburk commented 2 years ago

The same recording process applied to the Samsung S20 has a different result.

This sounds like a separate issue. If so please file a separate bug report so they don't get confused.

chrismanchester commented 2 years ago

@philburk, we had been puzzled by the high volume of reports related to these devices, not assuming that recording in our app while simultaneously recording with another was a common use case. We recently discovered an explanation: this issue is present whenever the Google Voice Assistant is configured to respond to voice prompts. I had a device that reproduced this unreliably, but upgrading to the most recent MIUI (12.5.2) made it completely consistent. The logs consist of repeated instances of:  

2022-02-22 10:26:41.544 733-16460/? D/audio_hw_primary: in_standby: enter: stream (0xe9bfa780) usecase(23: audio-record)
2022-02-22 10:26:41.548 733-16460/? D/audio_hw_primary: start_input_stream: enter: stream(0xe9bfa780)usecase(23: audio-record)
2022-02-22 10:26:41.548 733-16460/? E/audio_hw_primary: start_input_stream: use case assigned already in use, stream(0xe9bfa780)usecase(23: audio-record)
2022-02-22 10:26:41.598 733-16460/? D/audio_hw_primary: start_input_stream: exit: status(-22)

The only reliable workaround in OboeTester is to enable the VoiceCommunication input preset, which is less than ideal for our purposes. This suggested to me AAudioStreamBuilder_setPrivacySensitive() might also be effective, but unfortunately it was not.

We haven't been able to reproduce this on any other devices, so it seems like a regression specific to Xiaomi.

philburk commented 2 years ago

This has been reported to Qualcomm as b/222301335 | Redmi Note 8 Ginkgo - Voice Assistant interferes with microphone recording

philburk commented 2 years ago

Qualcomm was not able to reproduce this. It does seem specific to Xiaomi. We will contact Xiaomi.

New bug visible to Xiaomi at b/226408733

barantes commented 2 years ago

For the record, I see the same issue described by @chrismanchester (https://github.com/google/oboe/issues/1427#issuecomment-1048320467) in our app running on Xiaomi Redmi Note 10 Pro - Android 12 - MIUI 13. Disabling the Google Assistant app or revoking its Recording permission fixes the problem.

robertwu1 commented 1 year ago

Maybe this is related to MMAP being disabled on Redmi Note 8/10.

The AAudio non-MMAP path may be missing coverage on other phones and perhaps may happen other phones that don't support MMAP

flamme commented 1 year ago

I tested on Pixel device with disabling MMAP. And it still works. When the OboeTester was recording and the recorder app was at background, the recorder app only got silent data. That is expected.

I remembered there used to be a bug about fast path capture cannot be muted correctly fixed in Android 11. It may due to your Redmi device doesn't contain the fix. Do you have any device that is in Android 12 that can rerun the test?

chrismanchester commented 1 year ago

@flamme Unfortunately this was my only Xiaomi device and I've since lost access to it, but https://github.com/google/oboe/issues/1427#issuecomment-1120493043 suggests Xiaomi on Android 12 is impacted. My Oppo device running Android 12 doesn't reproduce the issue.

flamme commented 1 year ago

Thanks for pointing that out, @chrismanchester. The comment you mentioned does indicate that is an issue specific on Xiaomi device. As Phil mentioned, we have reported this to Xiaomi and Qcom.

flamme commented 1 year ago

Thanks Robert for giving me the redmi device to test!

I was able to repo on the Redmi device even with disabling Google assistant. The problem was caused by Redmi device failed to start input stream.

The audio framework will silent the background recorder when there is a foreground app starting record successfully. But if the foreground app fails to start recording, audio framework will still let the original recording app keep capturing audio.

When Recorder app was recording background, the Redmi audio HAL failed to start input stream when OboeTester tried to start recording. In that case, due to OboeTester was not active recording, audio framework would still let Recorder keep recording. See the code here. See Redmi error as below. This is the same root as mentioned by Phil at b/226408733

08-11 17:28:32.721   776  5547 E audio_hw_primary: start_input_stream: use case assigned already in use, stream(0xf00fa500)usecase(23: audio-record)
philburk commented 7 months ago

We pinged Xiaomi and are awaiting a response.