Access Unit Delimiters are output in an unusual order

[willnewton]

The access unit delimiters output by the encoder are output alongside the picture data even if SPS/PPS is present, which leads to an unusual ordering of NAL units. For example x264 outputs:

AUD SPS PPS VCL AUD VCL AUD VCL ...

Whereas imxvpuenc_h264 outputs:

SPS PPS AUD VCL AUD VCL AUD VCL ...

I believe this is the root cause of https://github.com/Freescale/gstreamer-imx/issues/80

[willnewton]

Attached is a proof of concept patch. What do you think? It basically just stops the encoder emitting AUD NALs and does it all manually. libimxvpuapi_aud_patch.txt

[dv1]

This could work. I think access unit delimiters do not carry any extra info by themselves, they are essentially just markers, so there's no harm in hardcoding the AUD bits. My only potential concern would be performance, but stuffing in a few bytes should not harm performance. Still test if for some weird reason this worsens it. Other than that, add a comment right above the "(*encoder)->aud_enable = " line, which says something like:

/* The encoder outputs AUDs which are in an improper order. example:
 * SPS PPS AUD VCL AUD VCL AUD VCL ...
 *
 * whereas the proper order (as for example x264 does it) should be:
 * AUD SPS PPS VCL AUD VCL AUD VCL ...
 *
 * for this reason, the automatic AUD placement is not used; the AUDs
 * are inserted manually instead. */

Otherwise, it may be confusing why we do this.

[willnewton]

I added the comment and opened a pull request. I didn't notice any performance difference in testing on an iMX6.

[willnewton]

Pull request merged.