Alternate Codec Support

[GoogleCodeExporter]

If possible, it would be REALLY helpful if multiple codec support was
included as there are a couple codecs that would be much better for a phone
like this. Or even set a fallback codec, however that is not even close to
mattering...

The codecs that would be EPIC to see supported are:

ILBC - Internet Low Bitrate Codec (Good for edge, or low signal)
GSM  
G729
G711

Thanks!

Original issue reported on code.google.com by andrusjo...@gmail.com on 9 Jun 2009 at 7:12

[GoogleCodeExporter]

G.711 is already supported. The other mentioned ones are not supported by 
Android OS. 
Implementing them within a Java application like Sipdroid is not possible.

Original comment by pmerl...@googlemail.com on 9 Jun 2009 at 7:54

• Changed state: WontFix

[GoogleCodeExporter]

From android officially supported codec table:
http://developer.android.com/guide/appendix/media-formats.html
there seems to be only audio encoder for "AMR-NB" with bitrates from 4.75 to 
12.2 
kbps sampled @ 8kHz. This should be more efficient than the raw G.711 data. 
(even if 
it needed a server side codec add-on).
Anyway don't know if SIP allows to negotiation of different codecs in each 
direction, 
but it would be nice to decode wide-band audio using like Vorbis or MP3 and 
encode 
with other limited codec.

What about a java implementation of Speex (there seems to be aleady a Java 
encoder/decoder):
http://freshmeat.net/projects/jspeex/
http://sourceforge.net/projects/jspeex/
don't know about real time performance in current android hardware, but 
performance 
wise, speex seems to be a very good speech codec (and free of 
patents/licenses)... if 
it prove to be successful, it could later implemented in android core media 
formats.

Original comment by mikexilva@gmail.com on 10 Jun 2009 at 1:06

[GoogleCodeExporter]

Today the NDK (native development kit) became available. That should allow 
developers 
to add low bitrate codecs.

Original comment by pmerl...@googlemail.com on 25 Jun 2009 at 10:15

• Changed state: New

[GoogleCodeExporter]

This is pretty important, as what good is it to have SIP if you can only use it 
at
the best of times...even one selectable lower bitrate codec would be sufficient.

Original comment by andrusjo...@gmail.com on 26 Jun 2009 at 4:49

[GoogleCodeExporter]

Original comment by pmerl...@googlemail.com on 26 Jul 2009 at 10:58

• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

I would like to suggest that g729 is particularly important for compatibility
reasons. Most SIP providers support g721 and g729. There are, however, licensing
issues with the g729, so I suggest selling a "codec upgrade package" on the 
market to
cover the licensing cost. It should be $10/channel.

Original comment by andr...@tastemycity.com on 2 Aug 2009 at 4:32

[GoogleCodeExporter]

I'd second the request for iLBC, Speex and GSM fullrate codecs, as they are 
freely
implementable and useable, and moreover they use much less bandwidth, making 
use over
GPRS or when there are limits in exchanged data feasible.

Original comment by marcu...@gmail.com on 22 Aug 2009 at 2:26

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

I'd also say a lower bitrate codec would be great for calls over edge, and I'd 
also
push for g729, even if I have to pay a fee, cause it's supported by my SIP 
provider.

Original comment by f8f...@gmail.com on 23 Aug 2009 at 3:23

[GoogleCodeExporter]

Yes, low bandwidth codes are a must. Mobile phones have come a long way in the 
last
few years. It is very clear that within the next year or so, everyone will be 
looking
at using their mobile phone for full fledged VOIP. A low bandwidth codec will 
be very
much welcomed by everyone.

Original comment by feroz.ba...@gmail.com on 26 Aug 2009 at 6:29

[GoogleCodeExporter]

+1 g729 is the only Free Codec my Provider has enabled ... even when paying the 
Monthly Option Very Few other Codecs :S

Original comment by masterc...@gmail.com on 28 Aug 2009 at 10:44

[GoogleCodeExporter]

I would once again like to ask please to have either G729 added or better 
yet...ILBC
- Internet Low Bitrate Codec. For those of us with not perfect edge even this 
would
really really help. And I would help if I knew how...

Original comment by andrusjo...@gmail.com on 14 Sep 2009 at 5:12

[GoogleCodeExporter]

g729 support is urgent :P

Original comment by giacomo1...@gmail.com on 24 Sep 2009 at 9:56

[GoogleCodeExporter]

I have been attempting jumping a somersault from the excitement I get from 
sipdroid.
 There is so much potential!  Thank you for all your work.  This is truly a FreeSky!

But alas, you need a codec for every season for sipdroid to be viable.  Please 
add
more codec support and make this the killer app!  I wish I could contribute 
more than
just stating the clearly obvious matter.  Escalate this to urgent?

Original comment by scotthaz...@gmail.com on 5 Oct 2009 at 1:40

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Android OS support the g729 codec?

Original comment by mhdazhar...@gmail.com on 5 Oct 2009 at 7:54

[GoogleCodeExporter]

It's possible to add support to Android, as it's essentially a java operating 
system
to some degree, and other symbian phones have g729 support. Symbian is a java
operating system as well, but I understand it would probably be alot of work. 
Any
work on it would be very awesome though!

Original comment by andrusjo...@gmail.com on 6 Oct 2009 at 9:26

[GoogleCodeExporter]

Neither android nor symbian are "java operating systems".
Android is linux, written in C-language with a proprietary-Java based UI stuff.
Symbian is entirely irrelevant. It is neither linux, nor Android-java.

As for whether "android supports" g729 or not...
No, of course not. And no, it doesn't matter.
You just need the source code for it (encoder and decoder) in C-language (use 
NDK
because Java is slow).

Original comment by andr...@tastemycity.com on 7 Oct 2009 at 9:01

[GoogleCodeExporter]

Symbian uses java-apps as it's primary application type, so it isn't entirely
irrelevant, as there are some sip java clients for Symbian, and finding such a 
client
that is open-source would save alot of work. APK's in android are in the devlik 
JAVA
VM. Unless you make something in the NDK like you said, it runs in the Java VM. 
I
know the operating systems aren't written in java, of course! That'd be 
rediculously
slow. I meant that they primarily are programmed FOR in java, as in the 
applications.

I agree with your last paragraph however.

Sorry for not having 'perfect' syntax, gosh.

Original comment by andrusjo...@gmail.com on 8 Oct 2009 at 8:39

[GoogleCodeExporter]

'And as ArsTechnica points out, despite being based on the Linux kernel, the 
open
source Android stack is essentially a creation of Java, and quite distinct from
Linux, making cross-platform application porting difficult.' 
-LinuxforDevices.com

As I pointed out, perhaps there is some open-source java code that can be 
repurposed,
such as g729 encoder and decoder. 

Link for quote:
http://www.linuxfordevices.com/c/a/News/Simulator-runs-Android-apps-on-Ubuntu/

Original comment by andrusjo...@gmail.com on 9 Oct 2009 at 4:22

[GoogleCodeExporter]

I did some research in the licensing of the g729 codec and it seems like it is a
proprietary codec that would need some fan-dangling much like cyanogen's most 
recent
mods. 

Cyanogen was forced to make a 'framework' to include google apps from HTC's ADB
image, and extract the apps into his ROM to sidestep 'distribution'. We would 
need to
do something similar with a 'codec api' much like Asterisk's implementation. 
Someone
would download SIPDROID with only the current included codec, and then they 
would be
able to download the snap-in codec module for the cost of licensing (usually
somewhere around $10 per implimentation - multiple lines equals more money). 

I got this information from a message by Matthew Rubenstein for the licensing 
issues
that SIP-Communicator faced for inclusion of G729. This message is pasted below 
and
shown here: http://markmail.org/message/xwducxagerofmicf

 'I read with interest bug #176, "support for g729", at
https://sip-communicator.dev.java.net/issues/show_bug.cgi?id=176 . I work with 
the
Asterisk PBX, and I can offer some insight into how SIP-Commuicator can support 
G.729
without violating legal constraints on either SIP-Communicator or G.729 .

G.729 is a patented codec algorithm. That means that any software that encodes 
data
into G.729 format, or that decodes G.729-encoded data into any other format, 
must
operate under an explicit license originating in the G.729 patent holder. It's 
not
clear (in murky US patent/DMCA/whatever law) whether you are prohibited from 
writing
and operating code that implements a G.729 codec (either/both encode and/or 
decode)
*if you do not distribute it, and do not derive any commercial gain from it*. In
other words, research and education *of yourself* are commonly believed to be 
safe,
even if you do not have a license to do so. This freedom might not be protected 
by
law - I am not a lawyer, and I don't know of any actual court cases, especially 
under
the G.729 patent - but the practice is widespread, specifically with G.729 
codecs. In
other words, the audio processing community "conventional wisdom" that is being 
used
by many developers and researchers around the world says that it's OK to do R&D 
by
writing and testing G.729 codecs without a patent license.

The patent is separate from the code. There are several "reference 
implementations"
of G.729 codecs in executable code distributed as source code. There are various
licenses that come with those code samples. Intel, for example, offers a code 
package
that is widely used, but which prohibits use in commercial applications. There 
are a
few other restrictions on the Intel code, but it is expressly supplied for R&D, 
and
used as such fairly widely. Other implementations are offered with similar 
freedom
for R&D. Again, they are covered under copyright restrictions, which are 
independent
(and in addition to) the patent license restrictions.

If you want to use someone's codec, even a commercially available binary codec, 
you
also need to have a license originating in the patent holder. Code offered
commercially for execution in commercial environments usually (always, AFAIK) 
comes
with a patent license. Sometimes those licenses are expensive, especially for
multiple concurrent instances of a running codec (the terms under which the 
patent
license is usually offered). However, Digium (the company that produces and 
publishes
the fairly open-source Asterisk PBX) sells G.729 codecs, with patent licenses 
they
relicense from the patent holder, for the lowest price I know. It's $10 per
concurrent stream (ie, 2 streams for a typical 2-person phonecall means 2 codecs
means $20 in licenses). The money pays for the license, and compensates Digium 
(which
it also invests in its other operations, including open Asterisk development). 
There
are other relicense sources, but they're all more expensive than buying them 
from
Digium, except at large numbers of concurrent streams/codecs (eg. 10,000 or so, 
maybe
fewer).

I would recommend SIP-Communicator operate in this Intellectual Property 
environment
in the following manner: Code SIP-Communicator with an interface for *any* 
codec as a
module, including G.729, the generic G.711, maybe GSM or some other popular 
codecs,
all with the single "codec" API. Developers can use the Intel or other 
reference code
in R&D to develop the API, as long as they do not distribute the G.729 code 
they used
in violation of its copyright license (eg. noncommercial). Offer G.729 as an 
addon
module. Then, with some finesse, perhaps the project (or someone using the 
project's
products) can make a deal with Digium or some patent relicenser to bundle a 
licensed
G.729 executable for a $fee, or include in an installer script an online 
ecommerce
transaction for purchasing licenses for bundled code, etc. The optional G.729 
codec
can be the kind of option that distinguishes commercial installs from 
noncommercial.
I recommend using Asterisk's G.729 inclusion techniques as a model:
http://www.google.com/search?q=site% 
3Avoip-info.org%20asterisk%20(g.729%20OR%20g729) .

It's worth it, because G.729 is high quality for the low bandwidth, and is very
popular with VoIP/PSTN gateways, and very widely supported in terminals (which 
means
minimal transcoding, therefore maximal performance). Some gateways, especially 
those
with low per-minute prices (and small minimum minutes committments), require 
G.729 to
connect. So including some way for SIP-Communicator to use G.729, despite the
patent/license hurdles, is a very valuable feature. I hope I can help this 
project to
achieve that goal.

-- 

(C) Matthew Rubenstein'

Original comment by andrusjo...@gmail.com on 9 Oct 2009 at 4:38

[GoogleCodeExporter]

And what about G726? It`s low bitrate codec too...

Original comment by kapo.ka...@gmail.com on 15 Oct 2009 at 3:28

[GoogleCodeExporter]

The usefulness of any particular audio encoding is based on use and 
availability.

Sure G726 might be a good low bitrate compression scheme, but how many VOIP 
providers
actually use it?

The reason G729 should be absolutely top priority is that it does the required 
job
perfectly, AND virtually *all* VOIP providers support it.

Remember that you can only use what your provider actually supports!

*HOWEVER*... an even better approach to this might be to include a generic 
plugin
interface. This would allow for the use of FREE compression schemes (like speex 
for
example) for those whose providers support them, but would also allow users to
actually pay for a license and use G729 if required.

Original comment by andr...@tastemycity.com on 15 Oct 2009 at 5:18

[GoogleCodeExporter]

Ya, that sounds good. It would be a good idea maybe to try to build a retro-fit 
for
the current plugins available for another open-source implimentation. Such as 
maybe
astrisx? It's an idea...

Original comment by andrusjo...@gmail.com on 17 Oct 2009 at 3:40

[GoogleCodeExporter]

I think "iLBC" is the best solution. Many top SIP providers, like Gizmo, 
support 
this codec and it is royalty free! Also, tests show iLBC actually performs 
better 
than G.729 in restrictive networks. As noted before, iLBC would work best in 
the 
native C code (via NDK). The key would be to create a UI for the C code in the 
Sipdroid app. Currently researching...

Original comment by jeremy.workman@gmail.com on 18 Oct 2009 at 4:52

[GoogleCodeExporter]

+1 Jeremy ! iLBC is the best solution, it's also a royalty free codec no fees, 
no patents, best quality in bad 
networks... We should ask iLBC support from android devs

Original comment by mouloud....@gmail.com on 18 Oct 2009 at 5:13

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Siphon, the SIP app for iphone (hosted on google code) has integrated g729 in 
its last version ! and this codec is 
not supported natively by the very proprietary iphone os, 
Why couldn't we do the same thing with g729 and ilbc on android ? (isn't 
android more open than iphone ?!)

http://code.google.com/p/siphon/issues/detail?id=129

Original comment by mouloud....@gmail.com on 18 Oct 2009 at 5:43

[GoogleCodeExporter]

Siphon is written in C and based on PJSIP, Sipdroid is written in Java and 
based on
MJSIP. There is very little comparison between the two unfortunately.

Original comment by duckmons...@gmail.com on 20 Oct 2009 at 5:44

[GoogleCodeExporter]

@duckmonster:
Not particularly relevant.
1) transcoding within java would be relatively heavy on the CPU. Best to use 
the NDK,
which means that it would be in C, which means that any C code containing the
required functions would be applicable.
2) pjsip vs mjsip is not particularly relevant since the objective is to build 
in the
transcoding components to the audio pipe. The actual protocol level changes 
required
to the sip stack (mjsip vs pjsip) are trivial. That means simply throwing in the
right header indicating the audio encoding to use and being able to receive and
understand the response header, which is used to apply the appropriate 
transcoders
when the audio streams are initiated.

*the audio transcoding is independent of SIP.
*think in terms of HTTP, i.e. request: "Accept-Encoding: gzip,deflate", 
response:
"Content-Encoding: gzip". The only real difference in SIP is that it is 2-way.

Original comment by andr...@tastemycity.com on 20 Oct 2009 at 1:18

[GoogleCodeExporter]

We should be able to use JNI bridges to access the Native Heap using a Long 
pointer. 
Codecs like iLBC are available in C packaging. We would then only need two 
components. The class file (ILBC.class) calls a native method, and the native 
library (ILBC.dll or rather ILBC.mk) implements the native method.

http://java.sun.com/docs/books/jni/html/jniTOC.html

Original comment by jeremy.workman@gmail.com on 22 Oct 2009 at 7:43

[GoogleCodeExporter]

That would be "ILBC.so"

Original comment by andr...@tastemycity.com on 22 Oct 2009 at 9:03

[GoogleCodeExporter]

Wouldn't speex be a better option than ILBC?  From what I understand, Speex is 
able
to operate at much lower bandwidth than ILBC, opening up the possibility of use 
over
GPRS (let's face it.  Even edge is not available everywhere).

Original comment by john.kie...@gmail.com on 26 Oct 2009 at 4:37

[GoogleCodeExporter]

Right but Speex is much less supported by SIP providers than iLBC ! so Ithink 
that the developpement should be 
focused on iLBC first (and then G729 if possible and speex, GSM...)

Original comment by mouloud....@gmail.com on 26 Oct 2009 at 4:42

[GoogleCodeExporter]

As I've already pointed out, the only two forms that you can count on are G711 
and
G729. The rest are *rare* and therefore not worth focusing on.

But really, the FACILITY to use ANY alternate encoding is what needs to be 
focused
on. Forget about implementing a specific encoding, fix the system first and then
implement encodings on an as-demanded basis.

Original comment by droidhac...@gmail.com on 26 Oct 2009 at 5:30

[GoogleCodeExporter]

iLBC is not rare - a lot of voip providers support it already
(http://www.slashphone.com/115/3229.html). 

As several other people here mentioned already, it has the major advantage that 
it is
free - which G729 is not. This makes it so much easier to add it to an open 
source
project.

Btw, i found the following (abandoned) project with a speex implementation for
android (quite rudimentary, though):
http://android.wooyd.org/files/VoiDroid-0.0.1.tar.bz2

Original comment by philipp....@gmail.com on 27 Oct 2009 at 9:06

[GoogleCodeExporter]

I would definitely second iLBC. Being free, and relatively widely used it would 
be
great. A codec framework would make more codecs easier, however iLBC I would 
say, is
more important...that's my opinion...

Original comment by andrusjo...@gmail.com on 27 Oct 2009 at 10:02

[GoogleCodeExporter]

Another vote for ilbc due to it's excelent packet loss concealment. This issue 
is the
only thing holding back android from being a truely disruptive technology on 3G
networks. Keep up the great work!

Original comment by jabawok....@gmail.com on 29 Oct 2009 at 8:38

[GoogleCodeExporter]

I think the docs say that sipdroid supports g711a.  Can we get support for g711u
(uLaw)? Thanks!

Original comment by robert.f...@gmail.com on 12 Nov 2009 at 6:02

[GoogleCodeExporter]

I was able to write a JNI linking library to the pjmedia-codec encode/decode
functions. The Sipdroid source I am providing is hard-coded for the GSM codec. 
I have
not experimented with compiling and linking in the Intel IPP extension for 
PJSIP,
which would provide the even higher compression G.729 and G.723 codecs.

ILBC and SPEEX are compiled in, but test very poorly using the PJSIP 
benchmarking
utility, so I stuck with GSM. There is one glaring bug, where the System.load() 
call
fails after Sipdroid has been Exited and started again.

I am releasing this mostly working version under the "release early release 
often"
principle. Hopefully someone out there knows how to find if Java has already 
loaded a
library, or how to somehow clear out the existing library from memory.

For those interested, the results of the PJSIP benchmarking application on a 
HTC Hero
are included in pjsip-test.txt.

Full sources at: http://rewire.org/sipdroid-gsm/source.zip (Can only upload 10M 
files
here)

To build, use the build-sipdroidlinker script as:

./build-sipdroidlinker --android-dir <your mydroid dir> --pjsip-dir 
pjproject-1.0.2
--sipdroidlinker-dir sipdroid-linker

Original comment by joe.n.ja...@gmail.com on 23 Nov 2009 at 1:20

Attachments:

• sipdroid-gsm.apk
• pjsip-test.txt

[GoogleCodeExporter]

Most of the ITSPs are using G.711 u/a-law under a wired environment (DSL or 
Cable), 
some use G.729a.
When using the Sipdroid on a mobile phone with HSPA/Edge/GPRS, it'd better to 
support G.729a.

Original comment by tommy.ht...@gmail.com on 23 Nov 2009 at 2:08

[GoogleCodeExporter]

@Joe, What Sipdroid source files did you change to create the interface with 
the 
PJSip JNI (RtpStreamReceiver, RtpStreamSender)? Does your interface still init 
the 
sipdroid/media/G711 in sipua/ui/InCallScreen.java? Could the ILBC and Speex 
benefit 
with a change to the buffer?

Original comment by jeremy.workman@gmail.com on 23 Nov 2009 at 5:19

[GoogleCodeExporter]

RtpStreamReceiver and RtpStreamSender call encode() and decode() respectively. 
The
JNI linker open() is called in Sipdroid.java, which is probably not where it 
should be.

The open() call takes a string which identifies the codec for PJSIP's codec 
search
function. Basically there is a good deal of work to be done in making Sipdroid 
keep
track of which codec it's using, and set the BUFFER_SIZE in the RTP files
appropriately, as well as the codec identification # and string in 
UserAgentProfile.java.

The problem with iLBC and Speex is how poorly they test in the PJSIP benchmark:

 8KHz codec encode/decode - iLBC              3037707  303.771 24611.42
 8KHz codec encode/decode - Speex 8Khz         481573   48.157 3901.69

iLBC takes 300% of the CPU, Speex takes 50%. G.729 seems like what people are
clamouring for, but there are both issues with compiling IPP for Android as 
well as
the licensing issue.

Also, this System.load() issue is a show stopper...

Original comment by joe.n.ja...@gmail.com on 23 Nov 2009 at 6:42

[GoogleCodeExporter]

Original comment by pmerl...@googlemail.com on 23 Nov 2009 at 8:51

• Changed state: Started

[GoogleCodeExporter]

man, with iLBC and Speex is poorly because they dont convert float point 
operation 
to fixed point operation which arm hardware doesnt come with float point.
so all are soft float point operations which consume a lot cpu.

fixed pointe version iLbc and G729 codec are the way out.

Original comment by yuxiao...@gmail.com on 24 Nov 2009 at 4:01

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

SIPdroid needs either iLBC or G729 to maximize it's potential.  Would it be 
hard to
integrate iLBC (to start), then later maybe add G729?   

Are there any plans to implement a different codec in SIPdroid?

Greg

Original comment by gregh...@gmail.com on 24 Nov 2009 at 2:09

[GoogleCodeExporter]

someone need to rewrite the rtpstreamsend and rtpstreamreceiver to make it easy 
to 
support other codec plug in. current code is for alaw only. it's a mess.

Original comment by yuxiao...@gmail.com on 27 Nov 2009 at 3:57

[GoogleCodeExporter]

joe.n.jackson, would you please check in your changes. I have added you as a 
project 
member. Thanks!

Original comment by pmerl...@googlemail.com on 27 Nov 2009 at 10:34

[GoogleCodeExporter]

Question above. That script your using there to compile in the pjsip stuff has 
some
damn weird toolchain assumptions.

What version of the NDK is that compiling with, because half of the link 
requirements
just dont seem to exist. :(

Original comment by duckmons...@gmail.com on 30 Nov 2009 at 8:53