search

google / oboe

Oboe is a C++ library that makes it easy to build high-performance audio apps on Android.
Apache License 2.0
3.69k stars 563 forks source link

[question] Device running on OpenSL ES seems to ignore Oboe default buffer size #1163

Closed Mapiarz closed 3 years ago

Mapiarz commented 3 years ago

Hi

I'm using FluidSynth on Android with Oboe driver (https://github.com/FluidSynth/fluidsynth/blob/master/src/drivers/fluid_oboe.cpp).

My test device is Samsung Galaxy S6 running Android 7.0.0 - so no support for AAudio, uses OpenSL ES. Before I initialize the FS driver (which opens the audio stream, see referenced code above), I set oboe::DefaultStreamValues::FramesPerBurst to various values (just for testing). No matter what value I set, it seems to be ignored and the device opens an audio stream with the optimal frames per burst, which for this device is 240. I also set the performance mode to be 'LowLatency'.

Here's the revelant adb logcat log:

01-14 15:48:05.695  9618  9633 I synth   : start top
01-14 15:48:05.695  9618  9633 I synth   : oboe defaults set to: SampleRate = 48000; FramesPerBurst = 64
01-14 15:48:05.697  9618  9633 I libOpenSLES: Emulating old channel mask behavior (ignoring positional mask 0x3, using default mask 0x3 based on channel count of 2)
01-14 15:48:05.704  3095  3321 I AudioFlinger: add dynamic flag, can move to deep thread, session 857
01-14 15:48:05.704  3095  3200 I APM_AudioPolicyManager: stopOutput() output 21, stream 3, session 849
01-14 15:48:05.704  9618  9633 D AudioTrack: Client defaulted notificationFrames to 240 for frameCount 480
01-14 15:48:05.705  9618  9633 I AudioTrack: This process already got info. FadeIn[0] FadeOut[0] FadeInRing[0]
01-14 15:48:05.705  3095  3321 W AudioPolicyIntefaceImpl: Skipped to add effects on session 857
01-14 15:48:05.705  3095  3321 I APM_AudioPolicyManager: startOutput() output 21, stream 3, session 857
01-14 15:48:05.724  3095  3307 D audio_hw_primary: out_set_parameters: enter: kvpairs: routing=2
01-14 15:48:05.724  3095  3307 I audio_hw_primary: out_set_parameters adev->out_device:2
01-14 15:48:05.724  3095  3307 V audio_hw_primary: select_devices output_scenario:0 input_scenario:-1 out_snd_device 0x2 in_snd_device:0x0
01-14 15:48:05.724  3095  3307 I audio_hw_primary: select_devices - do not update mixer_path
01-14 15:48:05.724  3095  3307 D AudioFlinger: setCurDevice() 0x2
01-14 15:48:05.724  9618  9633 I synth   : start bottom

And audio_flinger dumpsys:

Library vr360audio
  VR360Audio / Samsung
    UUID: e6388202-e7a4-4c72-b68a-332eeeba269b
    TYPE: 07f548ab-d8bb-443d-ae20-c30beb717e03
    apiVersion: 00020000
    flags: 00000030
Library playbackrecorder
  Playback Recorder / The Android Open Source Project
    UUID: 517a26c0-fd7c-11e5-a837-0800200c9a66
    TYPE: 48da0d50-fd7c-11e5-a837-0800200c9a66
    apiVersion: 00020000
    flags: 000000B0
Library loudness_enhancer
  Loudness Enhancer / The Android Open Source Project
    UUID: fa415329-2034-4bea-b5dc-5b381c8d1e2c
    TYPE: fe3199be-aed0-413f-87bb-11260eb63cf1
    apiVersion: 00020000
    flags: 00000008
Library soundalive_sec
  SoundAlivePlus / Samsung
    UUID: cf65eb39-ce2f-48a8-a903-ceb818c06745
    TYPE: c4da1d1f-7cdf-42e2-ba60-efc7eb3508a3
    apiVersion: 00020000
    flags: 00000240
Library soundalive
  SoundAlive_EQ / Samsung
    UUID: 0c117b70-f97f-11e0-be50-0002a5d5c51b
    TYPE: 0bed4300-ddd6-11db-8f34-0002a5d5c51b
    apiVersion: 00020000
    flags: 00000210
  SoundAlive_Virtualizer / Samsung
    UUID: c747f6a0-418a-11e1-a621-0002a5d5c51b
    TYPE: 37cc2c00-dddd-11db-8577-0002a5d5c51b
    apiVersion: 00020000
    flags: 00000210
  SoundAlive_BassBoost / Samsung
    UUID: a926a540-418a-11e1-b2f1-0002a5d5c51b
    TYPE: 0634f220-ddd4-11db-a0fc-0002a5d5c51b
    apiVersion: 00020000
    flags: 00000210
Library myspace
  MySpace / Samsung
    UUID: 3462a6e0-655a-11e4-8b67-0002a5d5c51b
    TYPE: b6c0ace0-655a-11e4-9801-0002a5d5c51b
    apiVersion: 00020000
    flags: 00000210
Library mysound
  MySound / Samsung
    UUID: 263a88e0-50b1-11e2-bcfd-0800200c9a66
    TYPE: d2bc05e0-50b0-11e2-bcfd-0800200c9a66
    apiVersion: 00020000
    flags: 00000200
Library pre_processing
  Noise Suppression / NXP Software Ltd.
    UUID: df0afc20-93ce-11e0-98de-0002a5d5c51b
    TYPE: 58b4b260-8e06-11e0-aa8e-0002a5d5c51b
    apiVersion: 00020000
    flags: 00000203
  Acoustic Echo Canceler / NXP Software Ltd.
    UUID: d6dbf400-93ce-11e0-bcd7-0002a5d5c51b
    TYPE: 7b491460-8d4d-11e0-bd61-0002a5d5c51b
    apiVersion: 00020000
    flags: 00000203
  Automatic Gain Control / NXP Software Ltd.
    UUID: 03b75f00-93ce-11e0-9fb8-0002a5d5c51b
    TYPE: 0a8abfe0-654c-11e0-ba26-0002a5d5c51b
    apiVersion: 00020000
    flags: 00000203
Library output_session_processing
  Far-End Noise Suppression / NXP Software Ltd.
    UUID: cbcc5980-476d-11e1-82ee-0002a5d5c51b
    TYPE: f88eb3c0-476b-11e1-9375-0002a5d5c51b
    apiVersion: 00020000
    flags: 00000248
  Loudness Maximizer for Voice / NXP Software Ltd.
    UUID: 989d9460-413d-11e1-8b0d-0002a5d5c51b
    TYPE: 4c0aa460-413a-11e1-a29d-0002a5d5c51b
    apiVersion: 00020000
    flags: 00000250
Library downmix
  Multichannel Downmix To Stereo / The Android Open Source Project
    UUID: 93f04452-e4fe-41cc-91f9-e475b6d1d69f
    TYPE: 381e49cc-a858-4aa2-87f6-e8388e7601b2
    apiVersion: 00020000
    flags: 00000008
Library visualizer
  Visualizer / The Android Open Source Project
    UUID: d069d9e0-8329-11df-9168-0002a5d5c51b
    TYPE: e46b26a0-dddd-11db-8afd-0002a5d5c51b
    apiVersion: 00020000
    flags: 00000008
Library reverb
  Insert Preset Reverb / NXP Software Ltd.
    UUID: 172cdf00-a3bc-11df-a72f-0002a5d5c51b
    TYPE: 47382d60-ddd8-11db-bf3a-0002a5d5c51b
    apiVersion: 00020000
    flags: 00000048
  Auxiliary Preset Reverb / NXP Software Ltd.
    UUID: f29a1400-a3bb-11df-8ddc-0002a5d5c51b
    TYPE: 47382d60-ddd8-11db-bf3a-0002a5d5c51b
    apiVersion: 00020000
    flags: 00000001
  Insert Environmental Reverb / NXP Software Ltd.
    UUID: c7a511a0-a3bb-11df-860e-0002a5d5c51b
    TYPE: c2e5d5f0-94bd-4763-9cac-4e234d06839e
    apiVersion: 00020000
    flags: 00000048
  Auxiliary Environmental Reverb / NXP Software Ltd.
    UUID: 4a387fc0-8ab3-11df-8bad-0002a5d5c51b
    TYPE: c2e5d5f0-94bd-4763-9cac-4e234d06839e
    apiVersion: 00020000
    flags: 00000001
Library bundle
  Volume / NXP Software Ltd.
    UUID: 119341a0-8469-11df-81f9-0002a5d5c51b
    TYPE: 09e8ede0-ddde-11db-b4f6-0002a5d5c51b
    apiVersion: 00020000
    flags: 00000050
Clients:
  pid: 3115
  pid: 3954
  pid: 6770
  pid: 9618
Notification Clients:
  pid: 3095
  pid: 3097
  pid: 3115
  pid: 3627
  pid: 3954
  pid: 4513
  pid: 6770
  pid: 9618
Global session refs:
  session   pid count packagename
       49  6770     1 com.sec.android.inputmethod
      105  3954     1 com.android.systemui
      105  3115     1 
      113  3954     1 com.android.systemui
      113  3115     1 
      873  3954     1 com.android.systemui
      897  9618     1 com.PlayfulPeople.Repertoire
      921  9618     1 com.PlayfulPeople.Repertoire
Hardware status: 0
Standby Time mSec: 3000

Output thread 0xf269ce00 type 0 (MIXER):
  Thread name: AudioOut_D
  I/O handle: 13
  TID: 3305
  Standby: no
  Sample rate: 48000 Hz
  HAL frame count: 960
  HAL format: 0x1 (pcm16)
  HAL buffer size: 3840 bytes
  Channel count: 2
  Channel mask: 0x00000003 (front-left, front-right)
  Processing format: 0x1 (pcm16)
  Processing frame size: 4 bytes
  Pending config events: none
  Output device: 0x2 (SPEAKER)
  Input device: 0 (NONE)
  Audio source: 0 (default)
  Normal frame count: 960
  Last write occurred (msecs): 4
  Total writes: 147964
  Delayed writes: 0
  Blocked in write: yes
  Suspend count: 0
  Sink buffer : 0xf2987000
  Mixer buffer: 0x0
  Effect buffer: 0x0
  Fast track availMask=0xfe
  Standby delay ns=3000000000
  AudioStreamOut: 0xf354a230 flags 0xa (PRIMARY|DEEP_BUFFER)
  Thread throttle time (msecs): 420
  AudioMixer tracks: 0x00000007
  Master mono: off
  FastMixer not initialized
  Stream volumes in dB: 0:-3, 1:-20, 2:-9, 3:0, 4:-9, 5:-9, 6:0, 7:0, 8:-8, 9:-inf, 10:0, 11:-3, 12:-3, 13:-inf, 14:0, 15:-57, 16:0
  Normal mixer raw underrun counters: partial=0 empty=0
  3 Tracks of which 1 are active
    Name Active Client Type      Fmt Chn mask Session fCount S F SRate  L dB  R dB    Server Main buf  Aux Buf Flags UndFrmCnt
       1     no   3115    1 00000001 00000003     113   8820 S 1 44100     0     0  00006035 0xf2987000 0x0 0x601      1768 
       2    yes   9618    3 00000001 00000003     897   2129 A 3 24000     0     0  00032F00 0xf2997000 0x0 0x000       484 
       0     no   3115    5 00000001 00000003     105   8820 S 1 44100     0     0  00018FDA 0xf2987000 0x0 0x601       886 
  0 Effect Chains
  EffectStages count : main 4, running sample : 48000
    Stage Name                      Stage Type    sessionId    activeCnt    Enable    stageBuffer    FrameCount    Volume(float)
    primary_output_main_normal   1                   -2              0           1           0xf2987000      960         1.000000    
    Converter Info - Count : 1
    Dst Sample    Dst Format    Src Sample    Src Format
    48000                    1               192000              4  
    Stage Name                      Stage Type    sessionId    activeCnt    Enable    stageBuffer    FrameCount    Volume(float)
    primary_output_sub1_normal   2                   -3              1           1           0xf2997000      960         1.000000    
    Stage Name                      Stage Type    sessionId    activeCnt    Enable    stageBuffer    FrameCount    Volume(float)
    primary_output_main_uhq  1                   -4              0           1           0xf26ed000      3840        1.000000    
    Converter Info - Count : 1
    Dst Sample    Dst Format    Src Sample    Src Format
    192000                   4               48000               1  
    Stage Name                      Stage Type    sessionId    activeCnt    Enable    stageBuffer    FrameCount    Volume(float)
    primary_output_sub1_uhq  2                   -5              0           1           0xf25ed000      3840        1.000000    
    Stage Name                      Stage Type    sessionId    activeCnt    Enable    stageBuffer    FrameCount    Volume(float)
    (null)   1                   0               0           1           0xf2603000      7680        1.000000    
    Converter Info - Count : 1
    Dst Sample    Dst Format    Src Sample    Src Format
    384000                   3               48000               1  
    Stage Name                      Stage Type    sessionId    activeCnt    Enable    stageBuffer    FrameCount    Volume(float)
    offload_output_main_normal   1                   -12                 0           1           0x0         0       1.000000    

Output thread 0xf1e03200 type 0 (MIXER):
  Thread name: AudioOut_15
  I/O handle: 21
  TID: 3307
  Standby: no
  Sample rate: 48000 Hz
  HAL frame count: 240
  HAL format: 0x1 (pcm16)
  HAL buffer size: 960 bytes
  Channel count: 2
  Channel mask: 0x00000003 (front-left, front-right)
  Processing format: 0x1 (pcm16)
  Processing frame size: 4 bytes
  Pending config events: none
  Output device: 0x2 (SPEAKER)
  Input device: 0 (NONE)
  Audio source: 0 (default)
  Normal frame count: 960
  Last write occurred (msecs): 3
  Total writes: 569963
  Delayed writes: 0
  Blocked in write: yes
  Suspend count: 0
  Sink buffer : 0xf23fd000
  Mixer buffer: 0x0
  Effect buffer: 0x0
  Fast track availMask=0xf8
  Standby delay ns=3000000000
  AudioStreamOut: 0xf354a700 flags 0x4 (FAST)
  Thread throttle time (msecs): 0
  AudioMixer tracks: 0x00000007
  Master mono: off
  FastMixer command=MIX_WRITE writeSequence=4559793 framesWritten=547175040
            numTracks=2 writeErrors=0 underruns=864 overruns=2588
            sampleRate=48000 frameCount=240 measuredWarmup=12.8 ms, warmupCycles=4
            mixPeriod=5.00 ms
  Simple moving statistics over last 163.8 seconds:
    wall clock time in ms per mix cycle:
      mean=5.00 min=0.41 max=9.65 stddev=0.21
    raw CPU load in us per mix cycle:
      mean=358 min=0 max=4511 stddev=94
  Distribution of mix cycle times in ms for the tails (> ~3 stddev outliers):
    left tail: mean=1.66 min=0.41 max=2.71 stddev=0.69
    right tail: mean=8.36 min=7.36 max=9.65 stddev=0.66
  Fast tracks: sMaxFastTracks=8 activeMask=0x3
  Index Active Full Partial Empty  Recent Ready
      0    yes  423       0     2    full  2400
      1    yes   37       4   650    full   480
      2     no   16       1    10   empty     0
      3     no   89       0   256   empty     0
      4     no    0       0     0    full     0
      5     no    0       0     0    full     0
      6     no    0       0     0    full     0
      7     no    0       0     0    full     0
  Stream volumes in dB: 0:-3, 1:-20, 2:-9, 3:0, 4:-9, 5:-9, 6:0, 7:0, 8:-8, 9:-inf, 10:0, 11:-3, 12:-3, 13:-inf, 14:0, 15:-57, 16:0
  Normal mixer raw underrun counters: partial=0 empty=2
  3 Tracks of which 1 are active
    Name Active Client Type      Fmt Chn mask Session fCount S F SRate  L dB  R dB    Server Main buf  Aux Buf Flags UndFrmCnt
       0     no   3954    1 00000001 00000003     873  12621 S 1 44100  -inf  -inf  0000629A 0xf23fd000 0x0 0x600         0 
    F  1    yes   9618    3 00000001 00000003     921    480 A 1 48000     0     0  0002D690 0xf23fd000 0x0 0x000         0 
    F  2     no   6770    1 00000001 00000003      49   4079 S 1 48000  -inf  -inf  00000FEF 0xf23fd000 0x0 0x600         0 
  0 Effect Chains

Record Clients Histories:
AudioDevice HAL::dump
1. Common part
    Mutex: locked
    Audio Mode: 0
    Audio Previous Mode: 0
    seamless enabled: False
    Voice Wakeup State: 0
    Wakeup Mic Enabled: False
    Wakeup Baby Cry Enabled: False
    Wakeup Stopped: False
    Voice Keyword Path: 
    Voice Grammar Path: 
    tx_data_inversion: False
    jack config: 0
    jack skip check: False
    jack end status: False

2. About Call part
    Mic Mute: False
    Call Active: False
    TTY mode: 16
    Voice Volume: 0.800000
    Voice Volume Max Index: 5
    wb_amr: 0
    dual_mic: 1
    HAC mode: False
    extra volume: False
    emergency status: False
    Mute Voice: False
    Call memo Enabled: 0x0
    current modem: 0
    call forwarding: False

3. About Communications part
    VoLTE state: False
    Previous VoLTE state: False
    dv_wb_amr: 0
    Voip wificalling: False
    Wificall path : False
    VoIP RX active: False
    Processing Type: 0
    Voice REC Type: 0
    PreProcesss EQ enableds: 1

4. About Connectivity part
    Bluetooth_nrec: -1
    Bluetooth samplerate: 0
    Bluetooth SCO in Turn on: False
    Bluetooth SCO out Turn on: False

5. About Device Routing part
    Audio Devices: out(2), in(0)
    Audio scenario: -1
    output device gain: gain-media-speaker
    input device gain: 
    Output Scenario: 0
    Input Scenario: -1
    Out SND Device: 2
    In SND Device: 0

6. About Offload Playback part
    Sound Speed: 1000
    Sound Balance: 50
    Mono Audio: 0
    Effect Enable: 0
    Offload Volume: 0 0

7. About Factory Test part
    Loopback Out Device: 0
    Loopback In Device: 0
    isLoopback enabled: False
    isLoopback routing: False
    Factory Route Mode: 0
    is RMS Enable: False
    factory loopback mode: 0
Audio Ril::dump
    ConnectedRild: -38
    Register Callback State: 0
    MultiSim Mode: 0
    Current Modem: 0
Audio Stream Out::dump
    Output Type 0
    Mutex: unlocked
    Channel Mask: 3
    Supported Channel Mask: 3, 0, 0
    Configs Channels: 2
    Configs Sample Rate: 48000
    Configs Period Size: 960
    Configs Period Count: 4
    Configs Format: 0
    Format: 0
    Flags: a
    Offload Fragment Size: 0
    Offload Fragments: 0
    Offload Playback Started: 0
    Offload Avail Write: False
    Standby: False
    PCM Device Id: 3
    Devices: 2
    Muted: False
    Outputs are disabled except HDMI multi CH output: False
    Is Format Changed: False
    Is DiamondVoice Initialized: True
Audio Stream Out::dump
    Output Type 1
    Mutex: unlocked
    Channel Mask: 3
    Supported Channel Mask: 3, 0, 0
    Configs Channels: 2
    Configs Sample Rate: 48000
    Configs Period Size: 240
    Configs Period Count: 2
    Configs Format: 0
    Format: 0
    Flags: 4
    Offload Fragment Size: 0
    Offload Fragments: 0
    Offload Playback Started: 0
    Offload Avail Write: False
    Standby: False
    PCM Device Id: 6
    Devices: 2
    Muted: False
    Outputs are disabled except HDMI multi CH output: False
    Is Format Changed: False
    Is DiamondVoice Initialized: False

Reroute submix audio module:
 route[0] rate in=0 out=0, addr=[]
 route[1] rate in=0 out=0, addr=[]
 route[2] rate in=0 out=0, addr=[]
 route[3] rate in=0 out=0, addr=[]
 route[4] rate in=0 out=0, addr=[]
 route[5] rate in=0 out=0, addr=[]
 route[6] rate in=0 out=0, addr=[]
 route[7] rate in=0 out=0, addr=[]
 route[8] rate in=0 out=0, addr=[]
 route[9] rate in=48000 out=48000, addr=[]

Is there something weird going on or am I misinterpreting the data above? Could this be something that is vendor specific?

Thanks, Bartosz

philburk commented 3 years ago

In AudioStreamOpenSLES::configureBufferSizes() we use the value DefaultStreamValues::FramesPerBurst.

https://github.com/google/oboe/blob/417c00cddc6c2e49a7f64be7b23dc9c525b4df52/src/opensles/AudioStreamOpenSLES.cpp#L104

It may be overwritten for versions >= 7.1. For 7.0 it should be honored, according to the code. (I think there may be a bug. It should ignore mPerformanceMode for <= API 7.0)

No matter what value I set, it seems to be ignored and the device opens an audio stream with the optimal frames per burst, which for this device is 240.

Note that the frameCount in the dumpsys is unrelated. That is set by the HAL, not Oboe.

You should check stream->getFramesPerBurst()