tizonia aac decoder: does not work on attached files

[CapOM]

Hello,

"OMX.Aratelia.audio_decoder.aac" does not work on the 2 files that I put all together in the attached archive. It is both mp4 containing aac. First one is 7 channels, second is 2 channels (7 channles mixed/converted into 2 channels).

They play well with a browser or with gstreamer using the faad element wrapper: gst-launch-1.0 filesrc location=7.1auditionOutLeader_v2_rtb.mp4 ! qtdemux ! aacparse ! faad ! audioconvert ! pulsesink

aac.tar.gz

Replacing faad with omxaacdec leads to errors. The attached patch helps because NeAACDecInit(v1) return 0 instead of 2, whereas NeAACDecInit2 does not return the number of bytes. With that patch it runs only on the 2ch file but not audible (just noise but it is not that far from being correct).

That would be great if you could make aacdecprc.c handle these files.

Out of curiosity could you share a few files on which you tested it ? I will give it a go with omxaacdec.

Thx a lot Julien

[juanrubio]

Hi Julien,

I'll have a look at those files.

Yes, I remember that the tizonia OMX AAC decoder does not support all the AAC features very well. From my testing, I remember I could not make it to decode streams with some of the newer AAC features.

Is the gst faad component also based on libfaad? If that is so, then I'll definitely have a look at that code, because that would mean that my understading of the libfaad API is flawed.

This is the location of the streams that are being used in Travis. I have not done extensive testing of the AAC decoder (like using conformance testing stream samples out there, etc) http://www.aratelia.com/tizonia/tizonia-test-media.tgz

Finally, is there a patch you were meaning to attach?

[CapOM]

0001-wip.patch.tar.gz

[CapOM]

Have a look to : https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/faad/gstfaad.c

[CapOM]

gstomxaacdec works on both tizonia-test-media/aac/riff_rock.aac and tizonia-test-media/aac/strum12str.aac without the attached patch I attached, but with this change:

In plugins/aac_decoder/src/aacdec.c, put pcmmode.eEndian = OMX_EndianLittle; instead of OMX_EndianBig .

With both patches it can play aac_2ch.mp4 so the big endian /little endian was the explanation for the noise I mentioned above.

So there are at least 2 issues, the endianness and needs to call NeAACDecInit2 or NeAACDecInit depending on some heuristic or try NeAACDecInit2 first and if fails try NeAACDecInit, something like that.

But for 7.1auditionOutLeader_v2_rtb.mp4 it looks it needs more than that.

Thx!

[juanrubio]

Hi Julien (@CapOM)

I remember now why I did not use at the time the NeAACDecInit2 call in the Tizonia AAC decoder. The reason is that this api is used when the first buffer in the stream contains the AAC AudioSpecificConfig element.

/* Init the library using a DecoderSpecificInfo */
char NEAACDECAPI NeAACDecInit2(NeAACDecHandle hDecoder,

The behaviour when there is a decoder specific info element in OpenMAX IL was specified in both 1.1.2 in 1.2):

/** Codec Config Buffer Flag:                                                                                                                                               
* OMX_BUFFERFLAG_CODECCONFIG is an optional flag that is set by an                                                                                                          
* output port when all bytes in the buffer form part or all of a set of                                                                                                     
* codec specific configuration data.  Examples include SPS/PPS nal units                                                                                                    
* for OMX_VIDEO_CodingAVC or AudioSpecificConfig data for                                                                                                                   
* OMX_AUDIO_CodingAAC.  Any component that for a given stream sets                                                                                                          
* OMX_BUFFERFLAG_CODECCONFIG shall not mix codec configuration bytes                                                                                                        
* with frame data in the same buffer, and shall send all buffers                                                                                                            
* containing codec configuration bytes before any buffers containing                                                                                                        
* frame data that those configurations bytes describe.                                                                                                                      
* If the stream format for a particular codec has a frame specific                                                                                                          
* header at the start of each frame, for example OMX_AUDIO_CodingMP3 or                                                                                                     
* OMX_AUDIO_CodingAAC in ADTS mode, then these shall be presented as                                                                                                        
* normal without setting OMX_BUFFERFLAG_CODECCONFIG.                                                                                                                        
 * @ingroup buf                                                                                                                                                             
 */
#define OMX_BUFFERFLAG_CODECCONFIG 0x00000080

Therefore, since Tizonia does not yet have an mp4 demuxer (see #392), the only streams I could support at the time were those that can be decoded with

/* Init the library based on info from the AAC file (ADTS/ADIF) */
long NEAACDECAPI NeAACDecInit(NeAACDecHandle hDecoder,

So before tizonia's AAC decoder can support both modes of operation, and decode aac_2ch.mp4, ideally I should try to implement the mp4 demuxer to get a testing pipeline to play with.

Do you know if the gst-omx demuxer outputs OMX_BUFFERFLAG_CODECCONFIG in its first buffer?

[juanrubio]

It is possible to do the AudioSpecificConfig work on the AAC decoder without the mp4 demuxer. The binary file reader with small modifications can be used to get the job done. I cannot promise to get this delivered right away, since I would like to resolve #393 as soon as possible.

[CapOM]

"Do you know if the gst-omx demuxer outputs OMX_BUFFERFLAG_CODECCONFIG in its first buffer?"

Yes it does https://cgit.freedesktop.org/gstreamer/gst-omx/tree/omx/gstomxaudiodec.c#n1175