Android 4.2 low-latency audio

[GoogleCodeExporter]

I was reading this: http://developer.android.com/about/versions/jelly-bean.html 
and noticed there is now an option to take advantage of low-latency audio. Does 
CSipSimple utilize this? My Galaxy Nexus, with a low-latency patch, sounds 
considerably better than my Nexus 4 running 4.2.
I'm using the G729 codec, by the way.

The latency with my Galaxy Nexus is 18ms, while my N4 states 86ms.  

That's is what's stated regarding low-latency audio.

"Low-latency audio
Android 4.2 improves support for low-latency audio playback, starting from the 
improvements made in Android 4.1 release for audio output latency using OpenSL 
ES, Soundpool and tone generator APIs. These improvements depend on hardware 
support — devices that offer these low-latency audio features can advertise 
their support to apps through a hardware feature constant. New AudioManager 
APIs are provided to query the native audio sample rate and buffer size, for 
use on devices which claim this feature."

Any information would be great. CSipSimple works really well with my GNex, even 
over HSPA+, but doesn't sound quite right on my N4. 

Original issue reported on code.google.com by nb1...@gmail.com on 29 Nov 2012 at 8:33

[GoogleCodeExporter]

Thanks for the hint.

I didn't had time to test with my new N4 but I will. Normally as far as I've 
seen in API changes there is no special things that were not done previously. 
(OpenSL-ES api is already used and normally the sampling rate is not changed). 
The only thing that could be used in new API is AEC/NS but these modules are 
already included in csipsimple when software : it comes from webRTC project and 
are the same used in android source code.

I'm very interested in the way you use to measure the latency since you seems 
to get very precise values. It could help me to optimize stuff.

Original comment by r3gis...@gmail.com on 29 Nov 2012 at 10:11

[GoogleCodeExporter]

Thanks for the reply. I'm not sure how accurate those values are, but that's 
what is reported in Caustic 2, which is available in the Play store. 

Latency on 4.1, with the stock Gnex kernel, was 44ms, and on 4.2, it's nearly 
doubled. 

I'm fairly certain the poor audio quality I'm experiencing with my N4 is due to 
the audio latency. My Gnex, with the 18ms patch, sounds considerably better. 
I've tried turning the WebRTC implementation on, and off. I've used different 
echo cancellation methods, and I've played with the mic sensitivity. Nothing 
seems to help. 

I found the Galaxy Nexus patch here: 
http://forum.xda-developers.com/showthread.php?t=1777261 The author uses 
Caustic 2 to determine the latency. All you need to do is install the 
application, and go into settings. The audio latency is displayed at the 
bottom. 

I've scoured the internet looking for another method of estimating latency, but 
I haven't found anything yet. I found a program called "Pure Data Latency 
Tester" but it seems ridiculously inaccurate. Another program I found crashes, 
and fails to run. 

The developer said he's not going to be updating his patch to 4.2 and beyond, 
which is disappointing, because I definitely think my Galaxy Nexus sounds much 
better with it. 

I would test CSipSimple with your N4. I think you'll notice a difference 
compared to another device. Hopefully this can be patched, because I've been 
using your awesome SIP phone, without problems, for about 4 months now. 

Original comment by nb1...@gmail.com on 29 Nov 2012 at 11:21

[GoogleCodeExporter]

Ah, ok it mesures only the latency of the audio layer (I thought allowed to 
measure the latency when using csipsimple). 
It's maybe not totally relevant as the app may use a different audio mode than 
this app use : It's configurable inside csipsimple and by default uses 
"communication" mode which is announced by android sdk to be designed for low 
latency voip apps.

I did some tests with the N4 : I'm working on adding the high quality AAC-ELD 
(mpeg4) codec (HD codec) and for now I didn't noticed big latency (but I didn't 
focused on that and I'm working on direct wifi network).

Some thought too, maybe it could be interesting you try to upgrade to latest 
nightly build if you are using N4 (there's some fixes for JB) :
http://nightlies.csipsimple.com/trunk/CSipSimple-latest-trunk.apk

If you have some spare time, it could be insteresting to try to play with audio 
troubleshooting settings of the app.
It could help to find more optimized settings. (you have to switch to expert 
setting mode to see "audio troubleshooting" options -- in settings, press menu).
The clock rate/frequency option could also be interesting to change.

Original comment by r3gis...@gmail.com on 30 Nov 2012 at 10:21

• Changed state: Need-Details

[GoogleCodeExporter]

Thanks for the feedback. I'm mainly interested in G729, because I'm connected 
remotely to my PBX in a flash server, which is located at home, and G729 offers 
the best experience when on calling over HSPA+. PIAF is Asterisk based, so 
codecs like OPUS are not available yet. OPUS is supposed to be better for 
cellular data than G729, and OPUS is free. 

CSipSimple provides an excellent experience with my Galaxy Nexus and the G729 
codec. I have zero complaints -- it honestly sounds just as good as my 
expensive hardware SIP phones that use the G711/ulaw codec. 

I've upgraded to the latest nightly, and I'm still experiencing these problems. 
I've upped the clock rate to 44.1hHz, but the other party sounds a bit more 
distorted than before. I'm using WebRTC for echo cancellation, and I have 
WebRTC checked under "audio troubleshooting." Could I have an issue with echo 
cancellation? Will changing the echo cancellation trail do anything? I'm not 
sure what changing this value will do. 

The quality issues are not bad enough to stop using CSipSimple, and I haven't 
had anyone complain on the other end, but it's just not as good as the Gnex. I 
don't think it's the headset speaker, because circuit switched cellular calls 
actually sound better than my Gnex -- with the exception of the buzzing sound 
the headset speaker makes. 

I appreciate the feedback. Are you aware of any other settings I can play 
around with? Like I said, I'm exclusively using the G729 codec. I don't want to 
buy Bria, and I'd rather donate the $7.00 to you, and your cause. I'm just 
hoping this isn't a hardware issue. 

Original comment by nb1...@gmail.com on 30 Nov 2012 at 9:17

[GoogleCodeExporter]

Thanks for the tests and the feedback.
2 information you might be interested in about the settings you have :
For "webRTC" checked in "Audio troubleshooting" -> this option disable all 
other options (it's the way google implements the setup of the audio and it's 
not configurable ;) ). 
For the clock rate I thought more about reducing it.
In fact using higher values will require software to do more computation to 
resample : the gap between g729 that is 8kHz and the audio device clock rate is 
computed by software. So if you align the two (audio device clock rate and 
codec clock rate) it can gives better results. Having a higher value for audio 
device clock rate can virtually improve quality if audio driver is designed for 
this clock rate.

I'll continue doing tests on my N4 and keep you updated here if I find some 
interesting setting. I don't think it's linked to codec but rather something 
with audio layer for now but I may be wrong, so I'll also do more tests with 
g729 too.

Original comment by r3gis...@gmail.com on 30 Nov 2012 at 10:59

[GoogleCodeExporter]

I appreciate the response. I have played with many different settings, and 
nothing seems to improve the audio. I'm beginning to think the high pitched 
buzzing my headset speaker emits, even when the screen is off, is the culprit. 
Let me know if you're experiencing the same issue as it seems to be fairly 
widespread. 

Turn your screen off, and put your ear to the speaker. If you can hear a high 
pitched buzzing sound, your device has this defect too. 

After doing some testing, I've noticed that I cannot hear this buzzing sound 
when the phone is in airplane mode. It seems that there is an 
interference/shielding issue. If I turn WiFi back on while in airplane mode, 
the buzzing sound comes back.

I don't think CSipSimple is to blame for the poor audio. Although, I've been 
receiving a "404 not found" message when attempting to place a call on the 
latest nightly (r2036). I've been able to fix this issue by disconnecting from 
CSipSimple, and starting the program again. 

Original comment by nb1...@gmail.com on 3 Dec 2012 at 9:04

[GoogleCodeExporter]

Mmmh interesting point about the wifi interference.
BTW, some question, is bluetooth enabled ? (even if not connected, just enabled 
could have an impact)

There is some device that have BT and WiFi radio frequencies that conflicts 
(see 
http://code.google.com/p/csipsimple/issues/detail?id=2038&q=bluetooth%20wifi) 
-- on the issue it's with galaxy nexus, but maybe other device are impacted and 
maybe could impact even if no actual connection but just scanning.

Original comment by r3gis...@gmail.com on 3 Dec 2012 at 9:15

[GoogleCodeExporter]

I do not notice this issue on my Galaxy Nexus, EVO 4G LTE, or EVO 3d. The only 
time I cannot hear the buzzing sound is when airplane mode is enabled, and none 
of the interfaces (bt, wifi, or the radio) are enabled. Once I enable WiFi, or 
turn airplane mode off, I can hear the sound again. 

People are talking about it here: 
http://forum.xda-developers.com/showthread.php?t=1993668

Not all devices are affected, but many are. When you get a chance, test your 
device. 

Original comment by nb1...@gmail.com on 3 Dec 2012 at 9:19

[GoogleCodeExporter]

I had 22 milliseconds on my stock HTC One X (snap dragon,  not Tegra3 version) 
.  Put CM10 on it and it jumped to 122. My nexus 7 on Paranoid Android 3 gets 
70. Really want a portable drum machine brain for my Kork PadKontrol.  Want to 
make a condensed version of my MPC/Reason5. 

Original comment by RyanBrow...@gmail.com on 22 Feb 2013 at 2:02

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Is it possible to add 48khz sample rate to the expert audio options list?
Why do you ask? The Nexus 4 is native 48khz sampling unlike 44.1khz of most 
other devices. From what I can gather OpenSL fastpath is partly crippled 
through an internal software resampler if the sample rates aren't matched to 
the hardware.

See point 5 in post from Google's Android platform developer Ian Ni-Lewis:
http://stackoverflow.com/a/16205370

Original comment by darryl....@gmail.com on 5 Jun 2014 at 5:28

[GoogleCodeExporter]

I am happy to report that enabling 48khz sample rate has indeed improved full 
loop audio latency significantly from 426ms to 175ms on a LG Nexus 4. (stock 
4.4.3 (KTU48L), original kernel, May 2014 release Qualcomm drivers, rooted)

Longer explanation:
1. Sound measurements include both audio input, and output delays.
2. Select 44.1khz sample rate in settings, force close cSipSimple, open 
cSipSimple, open Audio test in settings. Open Audacity on PC, play a test tone 
or make a noise - observe and measure the captured wave forms between the test 
tone emission and its feedback from phone. Approximately 426ms observed.
3. Repeat as above, except, directly modify the preferences xml or recompile 
cSipsimple with option for 48khz sample rate. Repeat as above. Approximately 
175ms observed.

Device: LG Nexus 4, stock 4.4.3 (KTU48L), original kernel, rooted.

Forward:
I speculate there is still room for improvement. According to Glenn Kasten of 
Google, the "dude" in charge of the Android audio framework, there still needs 
to be alignment in terms of audio buffers. The Nexus 4 has a 5ms buffer (240 
frames at 48khz) at the driver level which should be closely matched by the 
OpenSL impl. A presentation at Google IO 13 details their Nexus (S, Gal, 4, 7 
,10) series and achieving 10ms output latency with extremely low jitter.

Their presentation is available below and is a good primer...
https://developers.google.com/events/io/sessions/325993827

Original comment by darryl....@gmail.com on 24 Jun 2014 at 12:12

[GoogleCodeExporter]

darry, so you hacked an 48 Khz setting into the code yourself? i hope 48 Khz 
might improve the situation with my moto g aswell.

Original comment by e...@gmx.de on 7 Aug 2014 at 6:35

[GoogleCodeExporter]

13: No more need - try the latest trunk build (r2427), r3gis added an "auto" 
option to select the correct hardware clock rate. (48khz for the N4, may differ 
on your moto g) You can see the selected rate in LogCat. 

For this option to provide any improvement, you must manually set the clock 
rate to AUTO (currently defaults to 8k?), have AndroidOS 4.2 or newer (4.4.4 
best), finally your device must support the new openSL low-latency audio 
stream. 

The usual developer build caveats apply.

Original comment by darryl....@gmail.com on 7 Aug 2014 at 6:59