How to set `seq_param_set` and `pic_param_set` of `AacConfig` from raw H264 stream without input file?

[21pages]

I found that sps and pps can be get from input file in example mp4copy, but now I don't have input file, I just have h264 frames, how can I set seq_param_set and pic_param_set?

[alfg]

Hey, do you have an example for reference?

[DCNick3]

To mux raw h264 stream into an mp4 you would need to have do two things:

1. convert NALs from Annex B format to length-prefixed

So, apparently, for some reason, the Annex B ("raw h264") format and mp4 use different formats for NALs (Network Abstraction Layer, the smaller packets that are inside mp4 packets). Raw h264 NALs are identified by their "00 00 01" or "00 00 00 01" prefixes, while mp4 NALs are prefixed by their length (usually 4 bytes, always big endian)

If you want to mux it in, you would need to convert from one to another.

2. extract SPS and PPS NALs, remove them from the bitstream and put them into headers

In mp4 the SPS and PPS are stored out-of-band in the headers, that's why the library wants them. In the Annex B format the SPS and PPS are contained in-band, before most of the data. When extracting the raw stream ffmpeg actually inserts those.

Here's an example .h264 having those:

multi_512x512.h264.zip

mp4, on the other hand, is expected to not have those, so you would want to remove them.

Here's an example of code that does the reverse, converting the mp4 bitstream into Annex B format for openh264 library: https://github.com/ralfbiedert/openh264-rust/blob/21d0adf5348fa204ae2a68eac00e1ea455639007/openh264/src/utils/mp4_bitstream_converter.rs#L114

[21pages]

thanks