[GST Plugin] AAC Invalid Codec Private Data

[adamthesax]

Hi! I'm working on pumping video data (H264/AAC) from an upstream source to Kinesis Video Streams via the gstreamer plugin. While the gst pipeline runs and submits data successfully, when I try to generate an HLS playlist I get the following error:

$ aws kinesis-video-archived-media get-hls-streaming-session-url --endpoint $ENDPOINT_URL --stream-name aac-audio-issue --hls-fragment-selector FragmentSelectorType=SERVER_TIMESTAMP,TimestampRange={StartTimestamp=2021-04-06T15:05:00.007Z} --playback-mode LIVE_REPLAY

An error occurred (MissingCodecPrivateDataException) when calling the GetHLSStreamingSessionURL operation: Missing codec private data in fragment for track 2.

If I grab the fragment via get-media-for-fragment-list, the fragment AAC has an invalid sampling rate:

$ ffprobe -hide_banner fragment.webm 
[aac @ 0x7fdfb6820c00] invalid sampling rate index 13
[matroska,webm @ 0x7fdfb680de00] Failed to open codec in avformat_find_stream_info
[aac @ 0x7fdfb6820c00] invalid sampling rate index 13
    Last message repeated 1 times
[matroska,webm @ 0x7fdfb680de00] Failed to open codec in avformat_find_stream_info
[matroska,webm @ 0x7fdfb680de00] Could not find codec parameters for stream 1 (Audio: aac, 0 channels, fltp): unspecified sample rate
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, matroska,webm, from 'fragment.webm':
  Metadata:
    title           : Kinesis Video SDK
    encoder         : Kinesis Video SDK 1.1.0
    AWS_KINESISVIDEO_FRAGMENT_NUMBER: 91343852333181724596048935557323111792491521698
    AWS_KINESISVIDEO_SERVER_TIMESTAMP: 1617721523.251
    AWS_KINESISVIDEO_PRODUCER_TIMESTAMP: 1617721523.469
  Duration: N/A, start: 1617721523.469000, bitrate: N/A
    Stream #0:0(eng): Video: h264 (Constrained Baseline), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 30.30 fps, 30 tbr, 1k tbn, 60 tbc (default)
    Metadata:
      title           : kvs_video_track
    Stream #0:1(eng): Audio: aac, 0 channels, fltp (default)
    Metadata:
      title           : kvs_audio_track
[aac @ 0x7fdfb686a800] invalid sampling rate index 13
Could not open codec for input stream 1

To confirm that the GST pipeline isn't the culprit, I replaced kvsplugin with an matroskamux/filesink and the file works correctly (see pipeline.mkv). I also tried conforming the CPD by calling mkvgenGenerateAacCpd, but ran into the same error (see aac_cpd.patch)

This is my test gst pipeline:

gst-launch-1.0 \
    filesrc location=src.mkv do-timestamp=TRUE !  matroskademux name=demux ! identity sync=TRUE ! queue ! \
    video/x-h264,group-id=1 ! multiqueue name=q sync-by-running-time=true demux. ! audio/mpeg,group-id=1 ! q. \
    kvsplugin name=kvs aws-region=us-east-1 stream-name=aac-audio-issue enable-streaming=true connect-webrtc=false log-level=3 \
    q. ! video/x-h264 ! h264parse ! kvs. \
    q. ! audio/mpeg ! aacparse ! kvs.

All my test media and my unsuccessful attempt at a fix is attached: media.zip aac_cpd.patch.txt

Thank you in advanced for helping us get to the bottom of this one.

[disa6302]

Hello @adamthesax ,

Apologies for the delay in the response. This needs some deep dive and I will get back to you ASAP.

[MushMal]

Are you able to run the audio/video test application successfully? Can you run some of the a/v sample GST pipelines included in the docs?

[hassanctech]

Can you please

export KVS_DEBUG_DUMP_DATA_FILE_DIR and set it to a path, the Producer SDK will write out the MKV the same as what it will stream out to the back-end service. Then we can inspect that and see if there is an issue there too.

[hassanctech]

Also in your AAC patch can you please also add logging to print out the sample rate?

[adamthesax]

Thanks for following up! I added the sample rate to my AAC patch and the sample rate looks to be okay:

gst_kvs_plugin_handle_plugin_event(): Sample Rate: 44100

I also dumped the mkv with KVS_DEBUG_DUMP_DATA_FILE_DIR and it appears to be exhibiting the same error with ffprobe:

[aac @ 0x7ff88b0bc800] invalid sampling rate index 13

Attached are my updated patch and the media: aac-patch-with-logging.patch.txt aac-audio-issue_0.mkv.zip

[disa6302]

@adamthesax ,

Thank you for the logs. I am going to spend some time today/tomorrow to investigate the logs and get back to you. Will reach out if I need anything else.

[disa6302]

@adamthesax ,

I still do not see the logs with KVS_DEBUG_DUMP_DATA_FILE_DIR? Am I missing something?

[adamthesax]

Setting KVS_DEBUG_DUMP_DATA_FILE_DIR generated the above media (aac-audio-issue_0.mkv). I enabled verbose logging and attached is a logfile gstreamer.log

[disa6302]

@adamthesax ,

Thank you for the logs. Nothing in the logs stands out to me that could cause this issue. Did you test the pipeline with audio/video test src? Is the problem reproducible?

[disa6302]

Closing due to inactivity.