Add AAC stream

[GoogleCodeExporter]

Hi, 

As we know Android can encoder the ADTS AAC on 4.0.3, 
how do we pack the aac stream for rtp streaming?

Original issue reported on code.google.com by ktasl.kt...@gmail.com on 25 May 2012 at 3:03

[GoogleCodeExporter]

Good point :)

Look what i've found in android sources:

        /** @hide AAC ADTS file format */
        public static final int AAC_ADTS = 6;

in frameworks/base/media/java/android/media/MediaRecorder.java

It's there, but not visisble yet in the documentation :/ I'm working on it, 
more feed back soon :)

Original comment by FyHertz on 30 May 2012 at 10:22

[GoogleCodeExporter]

Works on my phone ! (Galaxy SII with ICS 4.0.3), as a proof, here is the ouput 
of mediainfo:
abrac@dabra:~/Bureau$ mediainfo test.adts 
General
Complete name                            : test.adts
Format                                   : ADTS
Format/Info                              : Audio Data Transport Stream
File size                                : 11.1 KiB

Audio
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format version                           : Version 4
Format profile                           : LC
Bit rate mode                            : Variable / Variable
Minimum bit rate                         : 4 875 bps
Maximum bit rate                         : 109 Kbps
Channel(s)                               : 1 channel
Channel positions                        : Front: C
Sampling rate                            : 8 000 Hz
Compression mode                         : Lossy
Stream size                              : 11.1 KiB (100%)

You just need a MediaRecorder and to set the output format to 6: 
mr.setOuputFormat(6);

Original comment by FyHertz on 30 May 2012 at 11:07

[GoogleCodeExporter]

Hi, 

I want try it on my phone, but on HTC's phone , it has some error on H.264 
encode.
below attached is the logs.

And I found that use the phone to be the RTSP server, the packages are not 
stable.
Use the VLC watch the streaming, it will normal for only 1~3sec, and the video 
package will always error.

I mean the 3.1 and 3.2 version are not stabilizing as 1.6, why?

Original comment by ktasl.kt...@gmail.com on 31 May 2012 at 8:53

Attachments:

• logs.txt

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

This issue does not concern h264 support, it is about AAC dude :/
Anyway, i'm aware of this bug i might have a patch, i will release it in v3.3 :)
Thank you for your feedback, this kind of logs are really helpful !

Original comment by FyHertz on 31 May 2012 at 9:12

[GoogleCodeExporter]

v3.3 implements a trivial AAC ADTS packetizer for RTP, may not work well 
everywhere due to some MIME format parameters that may depend on the phone :/ 
I don't really know actually, the rfc 3640 refers to some specifications of AAC 
that i haven't tried to read yet ! 

Original comment by FyHertz on 31 May 2012 at 11:45

[GoogleCodeExporter]

At v3.3, can't see the AAC ADTS packetizer source code, I can't help the test.

Original comment by ktasl.kt...@gmail.com on 4 Jun 2012 at 2:55

[GoogleCodeExporter]

Fixed ! SVN plugin somehow forgot to version it :/ 

Original comment by FyHertz on 4 Jun 2012 at 12:00

[GoogleCodeExporter]

This post is for those brave developers who, after a short google research 
about AAC streaming on android, will end up on this page:

If you checkout the svn repository of this project, you will find out that 
streaming AAC from an android powered device running ICS is completly feasible, 
here is a breif description of how this is done in spydroid:

As it has already been mentionned on this thread, by setting the output format 
of a MediaRecorder to 6 you can record ADTS AAC. ADTS is already suitable for 
streaming, so what you could do is to simply set the output of the 
MediaRecorder to TCP socket. But what we do here is a little more complicated: 
AAC frames are extracted from the ADTS stream and rewrapped in an RTP stream, 
this is done by the AACADTSPacketizer.java class. 

What concerns me are those mime format parameters that are required to decode 
the RTP stream: some of them may depend on the phone (see comment 6).

Those parameters are:
    streamtype ?
    profile-level-id ?
    config ?
    Profile ?

(MIME format parameters are supposed to be sent to clients using SDP in the 
fmtp field)

Original comment by FyHertz on 11 Jun 2012 at 11:06

[GoogleCodeExporter]

Btw, I would really appreciate if some of you could tell me if AAC streaming is 
working on your phone with spydroid and if not, i could use a sample of ADTS 
ACC recorded with it :)

Original comment by FyHertz on 11 Jun 2012 at 11:09

[GoogleCodeExporter]

Hey, 

Sorry for late response.

Here are the logs using VLC connect device and AAC audio occurs 
ArrayIndexOutOfBoundsException

the devices still is HTC ONE V, and try the recorder inside the device, the AAC 
ATDS recorder is fine.

btw, I add some logs to help me change the code for WOWZA.. :)

Original comment by ktasl.kt...@gmail.com on 15 Jun 2012 at 7:08

Attachments:

• aacLog.txt

[GoogleCodeExporter]

I have the same problem. ArrayIndexOutOfBoundsException happened  when the aac 
frame size is bigger than the rtp packet size(1500)

Original comment by liuziwei...@gmail.com on 19 Jun 2012 at 3:28

[GoogleCodeExporter]

Okay :/ Access units may need to be splitted... I did not took this case into 
account in my basic implementation of the RFC3640
i could use a sample of ADTS ACC recorded with your phone ktasl.ktasl or 
liuziwei1982 :)
Anyway, thank you to you two for your feedback !

Original comment by FyHertz on 20 Jun 2012 at 9:43

[GoogleCodeExporter]

Hi Simon, 

I still study the AAC stream on Android 4.0.3, and I use the wireshark see the 
stream package, and I found that the inputstream.framelength in the RTP payload 
are strange.

Attached is the Log I export from wireShark. 

Please help about this, thank you.

Original comment by ktasl.kt...@gmail.com on 7 Aug 2012 at 8:43

Attachments:

• 08061639txt.txt

[GoogleCodeExporter]

I found out the solution......

Sometimes the recorder will record the ADTS with bad framelength, so drop the 
frame and the AAC stream will pass to server.

But I only record the audio without video....

Original comment by ktasl.kt...@gmail.com on 7 Aug 2012 at 9:29

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Hi, 
I noticed that both my phone (Galaxy SII and HTC V One) are often giving me bad 
frameLength in the AACPacketizer; either with a negative value (ie: -2, -7, 
...) or with values between 1500 and 8000. 

ktasl.ktasl what do you mean by drop the frame?

Original comment by ted.guen...@gmail.com on 20 Nov 2012 at 3:15

[GoogleCodeExporter]

Hi ted :)

Could you record some acc adts with your phone for me please ? You would simply 
need to create a basic application using a MediaRecorder with 
setAudioEncoder(MediaRecorder.AudioEncoder.AAC); and setOutputFormat(6);
It could really help me improve the acc packetizer :) 
I could use those samples to determine (using VLC) what are the MIME format 
parameters that would work for your phones... 

Regards, 

Original comment by FyHertz on 22 Nov 2012 at 8:26

[GoogleCodeExporter]

I recorded some audio using the following settings:
mRecorder = new MediaRecorder();
mRecorder.setAudioSource(MediaRecorder.AudioSource.MIC);
mRecorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);
mRecorder.setOutputFile(mFileName);
mRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);

And when I play it back with VLC, here's what I get:
Codec: MPEG AAC Audio (mp4a)
Channel: Stereo
Sampling rate: 16000 Hz

By the way, I could not manage to set the audio to MONO or set the sampling 
rate to 8000 (like it is in Spydroid). I tried using:
mRecorder.setAudioChannels(1);
mRecorder.setAudioSamplingRate(8000); 

Original comment by ted.guen...@gmail.com on 26 Nov 2012 at 2:05

Attachments:

• audiorecordtest.mp4

[GoogleCodeExporter]

Hi !
Thank you very much,
And yeah It seems that some (manny phones ?) it's impossible to record AAC with 
a sampling rate of 16kHz, I will submit a little patch soon and we will see if 
it works better

Original comment by FyHertz on 27 Nov 2012 at 7:53

[GoogleCodeExporter]

Original comment by FyHertz on 27 Nov 2012 at 8:52

• Added labels: Priority-High
• Removed labels: Priority-Medium

[GoogleCodeExporter]

Original comment by FyHertz on 27 Nov 2012 at 8:53

• Changed state: Accepted

[GoogleCodeExporter]

8kHz, i meant 8kHz in my previous reply.

Weird... The sample you posted has a sampling rate of 8kHz and I was able to 
stream it correctly with exactly the same mime type parameters that I use in 
Spydroid:
a=rtpmap:96 mpeg4-generic/8000
a=fmtp:96 streamtype=5; profile-level-id=15; mode=AAC-hbr; config=1588; 
SizeLength=13; IndexLength=3; IndexDeltaLength=3; Profile=1;

You can try the experiment yourself: use VLC to start an RTSP server and stream 
you file:
vlc audiorecordtest.mp4 --sout="#rtp{sdp=rtsp://:8554/}" -vvv
And then quickly connect to it:
vlc rtsp://127.0.0.1:8554/

Maybe AAC is recorded differently when the output format is ADTS. Could you 
record a video with mRecorder.setOutputFormat(6); ?

Original comment by FyHertz on 27 Nov 2012 at 10:01

[GoogleCodeExporter]

I tried your experiment and it worked fine. Here's the audio recorder with the 
following parameters:
mRecorder = new MediaRecorder();
mRecorder.setAudioSource(MediaRecorder.AudioSource.MIC);
mRecorder.setOutputFormat(6);
mRecorder.setOutputFile(mFileName);
mRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);
mRecorder.setAudioChannels(1);
mRecorder.setAudioSamplingRate(8000);

By the way, do you have a good ADTS parser? 

Original comment by ted.guen...@gmail.com on 29 Nov 2012 at 2:36

Attachments:

• audiorecordtest.mp4

[GoogleCodeExporter]

I'm working on it :) unfortunately I have exams coming so I don't have much 
free time
Thank you for the sample !

Original comment by FyHertz on 29 Nov 2012 at 5:36

[GoogleCodeExporter]

I finally figured it out. I had ADTS frames with frameLength of negative values 
or really high values (> 1000). The thing is when reading the ADTS header, 
sometimes I get a bunch of zeros between the different fields of the header and 
it would lead to a misinterpretation of the frameLength (which is the only 
relevant information in the ADTS header with the CRS). 
example: 
-1 0 0 0 0 0 0 0 -15 108 0 0 64 25 95 -4 (what I get)
-1 -15 108 64 25 95 -4 (what it should be)

I'll post the code when I'm done with it.

Original comment by ted.guen...@gmail.com on 30 Nov 2012 at 1:46

[GoogleCodeExporter]

Looks like the 0xFFF is useful after all... I did not expect phones to put 
garbage between frames :/
Thank you :)

Original comment by FyHertz on 30 Nov 2012 at 10:12

[GoogleCodeExporter]

Okay check out rev 213, I made some improvements !

Original comment by FyHertz on 1 Dec 2012 at 1:47