search

awslabs / amazon-kinesis-video-streams-producer-sdk-java

Allows developers to install and customize their connected camera and other devices to securely stream video, audio, and time-encoded data to Kinesis Video Streams
Apache License 2.0
78 stars 75 forks source link

Invalid Codec Private Data Exception when viewing KVS produced by a custom Android Producer #183

Closed jeryini closed 3 months ago

jeryini commented 1 year ago

Hi! We've implemented custom Android Producer, using examples as a reference point from:

The implementation should work correctly as in the AWS console we can see that data is being streamed: Screenshot from 2023-05-22 09-12-30

We're streaming H.265 encoded video stream and have found out the following CPD (Codec Private Data) in the stream:

00 00 00 01 40 01 0c 01 ff ff 01 40 00 00 03 00 
80 00 00 03 00 00 03 00 78 ac 0c 00 00 0f a0 00 
01 d4 c2 00 fa 28 00 00 00 01 42 01 01 01 40 00 
00 03 00 80 00 00 03 00 00 03 00 78 a0 03 c0 80 
10 e7 fa 2e ec 91 77 a2 5d 58 10 00 00 3e 80 00 
07 53 0c 40 00 00 00 01 44 01 c1 ad f0 13 64

It consists of Sequence Parameter Set (SPS), Picture Parameter Set (PPS) and Video Parameter Set (VPS) frames, each starting with start bytes 00 00 00 01. As specified in the reference manual for Stream Structures we've used this data to specify the final byte[] codecPrivateData when creating a new StreamInfo object, which is created in our implementation of custom MediaSource, overridden in method getStreamInfo. Based on above data in the stream we've passed it in the following way:

new byte[]{0x00, 0x00, 0x00, 0x01, 0x40, 0x01, 0x0c, 0x01, (byte) 0xff, (byte) 0xff, 0x01, 0x40, 0x00, 0x00, 0x03, 0x00, (byte) 0x80, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x78, (byte) 0xac, 0x0c, 0x00, 0x00, 0x0f, (byte) 0xa0, 0x00, 0x01, (byte) 0xd4, (byte) 0xc2, 0x00, (byte) 0xfa, 0x28, 0x00, 0x00, 0x00, 0x01, 0x42, 0x01, 0x01, 0x01, 0x40, 0x00, 0x00, 0x03, 0x00, (byte) 0x80, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x78, (byte) 0xa0, 0x03, (byte) 0xc0, (byte) 0x80, 0x10, (byte) 0xe7, (byte) 0xfa, 0x2e, (byte) 0xec, (byte) 0x91, 0x77, (byte) 0xa2, 0x5d, 0x58, 0x10, 0x00, 0x00, 0x3e, (byte) 0x80, 0x00, 0x07, 0x53, 0x0c, 0x40, 0x00, 0x00, 0x00, 0x01, 0x44, 0x01, (byte) 0xc1, (byte) 0xad, (byte) 0xf0, 0x13, 0x64 },

Also we've specified flag NAL_ADAPTATION_ANNEXB_CPD_AND_FRAME_NALS as last parameter when creating StreamInfo. According to the following document about NAL Adaptation Flags our stream is in Annex-B delimited format for NALUs, which means we need to use that flag.

But regardless of all that, we're still seeing the following error in KVS media viewer:

"message": "Could not parse CPD in HEVC (H.265) format.",
"code": "InvalidCodecPrivateDataException",

I've prepared a 10s long video stream in H.265 accessible here if it helps debug this issue and to check that CPD is indeed correct.

hassanctech commented 11 months ago

@jeryini If I try to playback the 10s video you provided in VLC player it is unable to do so. If I convert it to mp4 using ffmpeg I can play it and if I convert it back to raw h265 I can also play it:

ffmpeg -i video.h265 -c copy -movflags +faststart output.mp4
ffmpeg -i output.mp4 -c:v libx265 fout.h265

Also in the attached file I am not able to find the CPD data in Annex B format (i.e. I don't see the 40, 42, 44) sections but when I use ffmpeg to containerize it into MP4 then back to raw h265 I do see those start codes and the CPD data in Annex B format as I would expect.

I understand that the viewer shows you the InvalidCodecPrivateDataException error. Can you please share any logs you have including the actual request / response for the PutMedia call itself? Did you receive persisted acks on the producer side while calling PutMedia? In addition are you able to verify using the KVS Stream metrics that there are incoming bytes?

niyatim23 commented 3 months ago

Closing due to inactivity. Feel free to reopen if there's an update