search

google / oboe

Oboe is a C++ library that makes it easy to build high-performance audio apps on Android.
Apache License 2.0
3.71k stars 566 forks source link

Mmap input silent on Samsung Exynos devices when Atmos and Hey Google enabled #1526

Closed chrismanchester closed 2 years ago

chrismanchester commented 2 years ago

Android version(s): 12 Android device(s): SM-G985F, SM-G975F, SM-G996N Oboe version: 1.6.2 App name used for testing: OboeTester

Short description One or both inbuilt microphones will produce silence with mmap enabled when Dolby Atmos and Hey Google are enabled

Steps to reproduce

Expected behavior

Actual behavior

Device

SM-G985F, SM-G975F, SM-G996N

for p in \
    ro.product.brand ro.product.manufacturer ro.product.model \
    ro.product.device ro.product.cpu.abi ro.build.description \
    ro.hardware ro.hardware.chipname ro.arch "| grep aaudio";
    do echo "$p = $(adb shell getprop $p)"; done

ro.product.brand = samsung ro.product.manufacturer = samsung ro.product.model = SM-G975F ro.product.device = beyond2 ro.product.cpu.abi = arm64-v8a ro.build.description = beyond2ltexx-user 12 SP1A.210812.016 G975FXXUEHVC6 release-keys ro.hardware = exynos9820 ro.hardware.chipname = exynos9820 ro.arch = | grep aaudio = [aaudio.hw_burst_min_usec]: [2000]

Any additional context

Disabling Atmos will not fix the input until the phone is booted, but turning off Hey Google will consistently fix the input. Somewhat inconsistently booting the device will fix the input, but enabling Atmos and toggling Hey Google will consistently put the device in a state where Hey Google being enabled will silence mmap input until the device is booted again. The issue only impacts one of two internal mics on the SM-G975F and SM-G966N, while both internal mics are impacted on the SM-G985F. Data Paths test outputs: OboeTester-DataPaths-samsung-SM-G975F-20220406-102044.txt OboeTester-DataPaths-samsung-SM-G996N-20220406-101433.txt OboeTester-DataPaths-samsung-SM-G985F-20220405-164441.txt

philburk commented 2 years ago

Thank you for this! The connection between Atmos, Hotword, and MMAP is an important observation. It may explain why it was hard to reproduce in the lab. I will try to reproduce this myself. But I will also pass it on to Samsung immediately.

I assume the DataPaths tests were run with Atmos ON and Hey Google ON.

Would it also be possible to get Data Path test results for:

Atmos | Hey Google
  OFF | ON
  OFF | OFF
philburk commented 2 years ago

I filed internally b/228360745 | AAudio MMAP input silent on Samsung Exynos devices when Atmos and Hey Google enabled

chrismanchester commented 2 years ago

Thank you for this! The connection between Atmos, Hotword, and MMAP is an important observation. It may explain why it was hard to reproduce in the lab. I will try to reproduce this myself. But I will also pass it on to Samsung immediately.

@philburk Thank you. I imagine this is a pretty interesting one.

I assume the DataPaths tests were run with Atmos ON and Hey Google ON.

Yes, all of those logs were taken with Atmos ON/Hey Google ON.

Would it also be possible to get Data Path test results for:

Atmos | Hey Google
  OFF | ON
  OFF | OFF

Atmos OFF/Hey Google OFF: OboeTester-DataPaths-samsung-SM-G975F-20220407-091029-atmos-off-heygoogle-off.txt

Atmos OFF/Hey Google ON: OboeTester-DataPaths-samsung-SM-G975F-20220407-092256-atmos-off-heygoogle-on.txt

Interestingly Hey Google alone will result in some dead inputs, but fewer than when both are enabled.

robertwu1 commented 2 years ago

Interestingly Hey Google alone will result in some dead inputs, but fewer than when both are enabled.

Can you also collect Atmos On/Hey Google off to complete the full 2x2 matrix?

Thanks!

chrismanchester commented 2 years ago

Interestingly Hey Google alone will result in some dead inputs, but fewer than when both are enabled.

Can you also collect Atmos On/Hey Google off to complete the full 2x2 matrix?

Thanks!

Of course. Here is the result for Atmos ON/Hey Google OFF: OboeTester-DataPaths-samsung-SM-G975F-20220407-120826-atmos-on-heygoogle-off.txt

robertwu1 commented 2 years ago

I did some analysis on the data paths tests. 

Atmos | Hey Google | Input Preset Failures | Input Device Failures | Output Devices Failures
   ON |         ON |                     8 |                    11 |                       9
   ON |        OFF |                     5 |                    10 |                       9
  OFF |         ON |                     5 |                    10 |                       8
  OFF |        OFF |                     5 |                    10 |                       8

As Chris alluded, most of the issues occur when both Hey Google and Atmos are used, there are more failures than normal. However, even the baseline has 6 only passed tests. It has 23 failed tests and 5 skipped tests.

Below are the tests that differ between the runs:

  Atmos | Hey Google | Test 1 | Test 7 | Test 9 | Test 11 | Test 34
     ON |         ON |   FAIL |   FAIL |   FAIL |    FAIL |    FAIL
     ON |        OFF |   PASS |   PASS |   PASS |    PASS |    FAIL
    OFF |         ON |   PASS |   PASS |   PASS |    PASS |    PASS
    OFF |        OFF |   PASS |   PASS |   PASS |    PASS |    PASS

Tests 1, 7, 9 are all Input Preset Tests running MMAP. This implies that all MMAP tests fail when both Atmos and Hey Google are enabled. They pass otherwise. Seems like no data is coming in.

// FAILED Test 1
========================== #1
Requested:
  INP-M, ID =  0, SR =     0, Perf = LL, SH, ch = 0/1, inPre = Generic
  OUT-L, ID =  0, SR =     0, Perf = LL, SH, ch = 0/1
Actual:
  INP-M, ID = 11, SR = 48000, Perf = LL, SH, ch = 0/1, inPre = Generic
      burst=96, size=1440, cap=1536
  OUT-L, ID =  3, SR = 48000, Perf = LL, SH, ch = 0/1
      burst=192, size=768, cap=1536
Result:
  maxMag = 0.00000, jitter = 0.13090, xruns = 11/0, FAIL !!!!, mag, jitter

// PASSED Test 1
========================== #1
Requested:
  INP-M, ID =  0, SR =     0, Perf = LL, SH, ch = 0/1, inPre = Generic
  OUT-L, ID =  0, SR =     0, Perf = LL, SH, ch = 0/1
Actual:
  INP-M, ID = 11, SR = 48000, Perf = LL, SH, ch = 0/1, inPre = Generic
      burst=96, size=1440, cap=1536
  OUT-L, ID =  3, SR = 48000, Perf = LL, SH, ch = 0/1
      burst=192, size=768, cap=1536
Finished early after 1204 msec.
Result:
  maxMag = 0.10460, jitter = 0.05671, xruns = 36/0, PASS

Test 11 is an Input Devices Test with the built in microphone. The following are two logs, one failing and one passing. Like above, it seems like there is just no data.

// FAILED
========================== #11
Requested:
  INP-M, ID = 11, SR =     0, Perf = LL, SH, ch = 0/1, inPre = VoiceRec
  OUT-L, ID =  0, SR =     0, Perf = LL, SH, ch = 0/2
Actual:
  INP-M, ID = 11, SR = 48000, Perf = LL, SH, ch = 0/1, inPre = VoiceRec
      burst=96, size=1440, cap=1536
  OUT-L, ID =  3, SR = 48000, Perf = LL, SH, ch = 0/2
      burst=192, size=768, cap=1536
Result:
  maxMag = 0.00000, jitter = 0.13090, xruns = 0/0, FAIL !!!!, mag, jitter

// PASSED
========================== #11
Requested:
  INP-M, ID = 11, SR =     0, Perf = LL, SH, ch = 0/1, inPre = VoiceRec
  OUT-L, ID =  0, SR =     0, Perf = LL, SH, ch = 0/2
Actual:
  INP-M, ID = 11, SR = 48000, Perf = LL, SH, ch = 0/1, inPre = VoiceRec
      burst=96, size=1440, cap=1536
  OUT-L, ID =  3, SR = 48000, Perf = LL, SH, ch = 0/2
      burst=192, size=768, cap=1536
Finished early after 1103 msec.
Result:
  maxMag = 0.19178, jitter = 0.05539, xruns = 0/0, PASS

Test 34 is Output Devices Test without MMAP. The following are two logs, one failing and one passing. This one seems like potentially the volume is too low, since maxMag is non-zero.

// FAILED 
========================== #34
Requested:
  INP-L, ID =  0, SR =     0, Perf = LL, SH, ch = 0/2, inPre = VoiceRec
  OUT-L, ID =  3, SR =     0, Perf = LL, SH, ch = 1/2
Actual:
  INP-L, ID = 11, SR = 48000, Perf = LL, SH, ch = 0/2, inPre = VoiceRec
      burst=192, size=4096, cap=4096
  OUT-L, ID =  3, SR = 48000, Perf = LL, SH, ch = 1/2
      burst=192, size=768, cap=1536
Result:
  maxMag = 0.01208, jitter = 1.24228, xruns = 0/0, FAIL !!!!, jitter

// PASSED
========================== #34
Requested:
  INP-L, ID =  0, SR =     0, Perf = LL, SH, ch = 0/2, inPre = VoiceRec
  OUT-L, ID =  3, SR =     0, Perf = LL, SH, ch = 1/2
Actual:
  INP-L, ID = 11, SR = 48000, Perf = LL, SH, ch = 0/2, inPre = VoiceRec
      burst=192, size=4096, cap=4096
  OUT-L, ID =  3, SR = 48000, Perf = LL, SH, ch = 1/2
      burst=192, size=768, cap=1536
Finished early after 1302 msec.
Result:
  maxMag = 1.14704, jitter = 0.06142, xruns = 0/0, PASS

@philburk it's quite worrying how most of the configurations are broken on Exynos and that Atmos + Hey Google breaks all but one of the remaining cases.

chrismanchester commented 2 years ago

@robertwu1 @philburk Thank you very much for taking a look at this issue. Have we received word from Samsung on the issue? It would be very helpful to us to understand the scope of the problem if they know (specific chipsets, os versions), or even an estimated timeline to ship a fix.

philburk commented 2 years ago

I pinged Samsung on May 4 and May 10. Waiting for a response.

philburk commented 2 years ago

Samsung says they are investigating this internally.

chrismanchester commented 2 years ago

Thank you Phil for the update. In case it's helpful to the investigation, while we initially reproduced this only on Exynos devices, we found a Qualcomm device through our monitoring, the SM-A528B, that reproduces a silence issue under the same conditions.

philburk commented 2 years ago

Samsung responded:

We have reproduced aaudio mmap input silence when "Hey google" enabled and we are checking this issue with S.LSI. We Will update it soon if there is any progress.

BTW, could you share additional information of SM-A528B such as binary version ?

Maybe enter:

for p in \
    ro.product.brand ro.product.manufacturer ro.product.model \
    ro.product.device ro.product.cpu.abi ro.build.description \
    ro.hardware ro.hardware.chipname ro.arch "| grep aaudio";
    do echo "$p = $(adb shell getprop $p)"; done
chrismanchester commented 2 years ago

Thanks for the update! Here is the info from the A52:

ro.product.brand = samsung
ro.product.manufacturer = samsung
ro.product.model = SM-A528B
ro.product.device = a52sxq
ro.product.cpu.abi = arm64-v8a
ro.build.description = a52sxqxx-user 12 SP1A.210812.016 A528BXXU1CVE2 release-keys
ro.hardware = qcom
ro.hardware.chipname = SM7325
ro.arch = 
| grep aaudio = [aaudio.hw_burst_min_usec]: [2000]
[aaudio.mmap_exclusive_policy]: [2]
[aaudio.mmap_policy]: [2]
philburk commented 2 years ago

Samsung is reporting that the SM-A528B has been fixed and the fix will be included in the release soon.

They also found the cause of the silence on Samsung Exynos devices and have a patch. That fix will appear in Android 13 after sufficient testing.

I am closing this because there is nothing for us to do except wait for the fixes to roll out.

@chrismanchester - Thanks so much for reporting this!