search

awslabs / amazon-kinesis-video-streams-producer-sdk-cpp

Amazon Kinesis Video Streams Producer SDK for C++ is for developers to install and customize for their connected camera and other devices to securely stream video, audio, and time-encoded data to Kinesis Video Streams.
Apache License 2.0
373 stars 334 forks source link

[QUESTION] Streaming MP4 file data to Kinesis Video Streams results in Transferred a partial file error with CURL #1195

Open nilp-dromeda opened 1 month ago

nilp-dromeda commented 1 month ago

I am using AWS KVS to stream a live camera feed. As a quick MVP I have opted to replicate the camera feed with a MP4 file which I manually send to the AWS KVS stream frame-by-frame (I use OpenCV to read the MP4 and convert it to frames) using the putFrame function. I am not seeing any errors initially from logs, but after a couple seconds after putFrameing all the video frames I am given an error of the form:

[video_streamer_node-1] WARN - curlCompleteSync(): [<stream-name>] curl perform failed for url https://s-ca658586.kinesisvideo.ap-southeast-2.amazonaws.com/putMedia with result Transferred a partial file: transfer closed with outstanding read data remaining
[video_streamer_node-1] WARN - curlCompleteSync(): [<stream-name>] HTTP Error 0 : Response: (null)

On the AWS KVS CloudWatch metrics side I am able to see spikes in PutMedia Active Connections, PutMedia Success, HLS and MPEG-DASH Requests, and HLS and MPEG-DASH Success. I am getting 100% success rates with PutMedia Success, but 0% success rates with HLS and MPEG-DASH Success.

Could anyone give me any ideas on what could be causing this error? Would the data I send in the frame object I pass to putFrame be non-ideal - I have read that you can stream non-video data as well (which would mean what data I send shouldn't really affect the streaming?). Does the MP4 video encoding have any affect? I have uploaded the full logs below.

Logs

[video_streamer_node-1] [INFO] [1720758264.804119277] [kinesis_video_streamer_node]: Initialising streamer node...
[video_streamer_node-1] [INFO] [1720758264.804229816] [kinesis_video_streamer_node]: Initialising AWS Kinesis SDK
[video_streamer_node-1] [INFO] [1720758264.879923571] [kinesis_video_streamer_node]: Initialising kinesis video producer...
[video_streamer_node-1] INFO - createKinesisVideoClient(): Creating Kinesis Video Client
[video_streamer_node-1] WARN - fixupClientInfo(): Connection timeout is invalid...setting to default
[video_streamer_node-1] WARN - fixupClientInfo(): Completion timeout is invalid...setting to default
[video_streamer_node-1] INFO - heapInitialize(): Initializing native heap with limit size 268435456, spill ratio 0% and flags 0x00000001
[video_streamer_node-1] INFO - heapInitialize(): Creating AIV heap.
[video_streamer_node-1] INFO - heapInitialize(): Heap is initialized OK
[video_streamer_node-1] DEBUG - getSecurityTokenHandler invoked
[video_streamer_node-1] DEBUG - Refreshing credentials. Force refreshing: 0 Now time is: 1720758264881859265 Expiration: 0
[video_streamer_node-1] INFO - createDeviceResultEvent(): Create device result event.
[video_streamer_node-1] DEBUG - clientReadyHandler invoked
[video_streamer_node-1] [INFO] [1720758264.882003494] [kinesis_video_streamer_node]: Initialising kinesis video stream...
[video_streamer_node-1] INFO - Creating Kinesis Video Stream <stream-name>
[video_streamer_node-1] INFO - createKinesisVideoStream(): Creating Kinesis Video Stream.
[video_streamer_node-1] INFO - logStreamInfo(): SDK version:
[video_streamer_node-1] DEBUG - logStreamInfo(): Kinesis Video Stream Info
[video_streamer_node-1] DEBUG - logStreamInfo():        Stream name: <stream-name>
[video_streamer_node-1] DEBUG - logStreamInfo():        Streaming type: STREAMING_TYPE_REALTIME
[video_streamer_node-1] DEBUG - logStreamInfo():        Content type: video/h264
[video_streamer_node-1] DEBUG - logStreamInfo():        Max latency (100ns): 0
[video_streamer_node-1] DEBUG - logStreamInfo():        Fragment duration (100ns): 20000000
[video_streamer_node-1] DEBUG - logStreamInfo():        Key frame fragmentation: Yes
[video_streamer_node-1] DEBUG - logStreamInfo():        Use frame timecodes: Yes
[video_streamer_node-1] DEBUG - logStreamInfo():        Absolute frame timecodes: Yes
[video_streamer_node-1] DEBUG - logStreamInfo():        Nal adaptation flags: 40
[video_streamer_node-1] DEBUG - logStreamInfo():        Average bandwidth (bps): 4194304
[video_streamer_node-1] DEBUG - logStreamInfo():        Framerate: 25
[video_streamer_node-1] DEBUG - logStreamInfo():        Buffer duration (100ns): 1200000000
[video_streamer_node-1] DEBUG - logStreamInfo():        Replay duration (100ns): 400000000
[video_streamer_node-1] DEBUG - logStreamInfo():        Connection Staleness duration (100ns): 300000000
[video_streamer_node-1] DEBUG - logStreamInfo():        Store Pressure Policy: 1
[video_streamer_node-1] DEBUG - logStreamInfo():        View Overflow Policy: 1
[video_streamer_node-1] DEBUG - logStreamInfo():        Allow stream creation: Yes
[video_streamer_node-1] DEBUG - logStreamInfo():        Segment UUID: NULL
[video_streamer_node-1] DEBUG - logStreamInfo():        Frame ordering mode: 0
[video_streamer_node-1] DEBUG - logStreamInfo(): Track list
[video_streamer_node-1] DEBUG - logStreamInfo():        Track id: 1
[video_streamer_node-1] DEBUG - logStreamInfo():        Track name: kinesis_video
[video_streamer_node-1] DEBUG - logStreamInfo():        Codec id: V_MPEG4/ISO/AVC
[video_streamer_node-1] DEBUG - logStreamInfo():        Track type: TRACK_INFO_TYPE_VIDEO
[video_streamer_node-1] DEBUG - logStreamInfo():        Track cpd: NULL
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: user-agent -> AWS-SDK-KVS/1.5.2 GCC/13.2.0 Linux/5.15.153.1-microsoft-standard-WSL2 x86_64 CPPSDK
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: host -> kinesisvideo.ap-southeast-2.amazonaws.com/describeStream
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: X-Amz-Date -> 20240712T042424Z
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: content-type -> application/json
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: content-length -> 27
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: Authorization -> AWS4-HMAC-SHA256 Credential=<credentials>/ap-southeast-2/kinesisvideo/aws4_request, SignedHeaders=host;user-agent;x-amz-date, Signature=<signature>
[video_streamer_node-1] INFO - writeHeaderCallback(): RequestId: 36605001-63ab-4233-ad0c-b2cd5811344c
[video_streamer_node-1] DEBUG - describeStreamCurlHandler(): [<stream-name>] DescribeStream API response: {"StreamInfo":{"CreationTime":1.720751492346E9,"DataRetentionInHours":24,"DeviceName":"Kinesis_Video_Device","IngestionConfiguration":null,"KmsKeyId":"arn:aws:kms:ap-southeast-2:210362323009:alias/aws/kinesisvideo","MediaType":"video/h264","Status":"ACTIVE","StreamARN":"arn:aws:kinesisvideo:ap-southeast-2:210362323009:stream/<stream-name>/1720751492346","StreamName":"<stream-name>","Version":"<version>"}}
[video_streamer_node-1] INFO - describeStreamResultEvent(): Describe stream result event.
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: user-agent -> AWS-SDK-KVS/1.5.2 GCC/13.2.0 Linux/5.15.153.1-microsoft-standard-WSL2 x86_64 CPPSDK
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: host -> kinesisvideo.ap-southeast-2.amazonaws.com/tagStream
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: X-Amz-Date -> 20240712T042425Z
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: content-type -> application/json
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: content-length -> 132
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: Authorization -> AWS4-HMAC-SHA256 Credential=<credentials>/ap-southeast-2/kinesisvideo/aws4_request, SignedHeaders=host;user-agent;x-amz-date, Signature=<signature>
[video_streamer_node-1] INFO - writeHeaderCallback(): RequestId: e8905042-9e0f-418f-ba60-b9915bb54ca6
[video_streamer_node-1] INFO - tagResourceResultEvent(): Tag resource result event.
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: user-agent -> AWS-SDK-KVS/1.5.2 GCC/13.2.0 Linux/5.15.153.1-microsoft-standard-WSL2 x86_64 CPPSDK
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: host -> kinesisvideo.ap-southeast-2.amazonaws.com/getDataEndpoint
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: X-Amz-Date -> 20240712T042425Z
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: content-type -> application/json
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: content-length -> 52
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: Authorization -> AWS4-HMAC-SHA256 Credential=<credentials>/ap-southeast-2/kinesisvideo/aws4_request, SignedHeaders=host;user-agent;x-amz-date, Signature=<signature>
[video_streamer_node-1] INFO - writeHeaderCallback(): RequestId: 77971e4e-a881-4344-bbb3-4b2871fcd48e
[video_streamer_node-1] DEBUG - getStreamingEndpointCurlHandler(): [<stream-name>] GetStreamingEndpoint API response: {"DataEndpoint":"https://s-ca658586.kinesisvideo.ap-southeast-2.amazonaws.com"}
[video_streamer_node-1] INFO - getStreamingEndpointResultEvent(): Get streaming endpoint result event.
[video_streamer_node-1] DEBUG - getStreamingTokenHandler invoked
[video_streamer_node-1] DEBUG - Refreshing credentials. Force refreshing: 1 Now time is: 1720758265390606953 Expiration: 1720758564
[video_streamer_node-1] INFO - getStreamingTokenResultEvent(): Get streaming token result event.
[video_streamer_node-1] DEBUG - streamReadyHandler invoked
[video_streamer_node-1] [INFO] [1720758265.390754832] [kinesis_video_streamer_node]: Initialising image transport callback...
<stream some data with putFrame>
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: user-agent -> AWS-SDK-KVS/1.5.2 GCC/13.2.0 Linux/5.15.153.1-microsoft-standard-WSL2 x86_64 CPPSDK
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: x-amzn-stream-name -> <stream-name>
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: x-amzn-producer-start-timestamp -> 1720758394.270
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: x-amzn-fragment-acknowledgment-required -> 1
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: x-amzn-fragment-timecode-type -> ABSOLUTE
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: transfer-encoding -> chunked
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: connection -> keep-alive
[video_streamer_node-1] INFO - putStreamResultEvent(): Put stream result event. New upload handle 0
[video_streamer_node-1] WARN - notifyDataAvailable(): [<stream-name>] Failed to un-pause curl with error: 43. Curl object 0x7f12fc010b50
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: host -> s-ca658586.kinesisvideo.ap-southeast-2.amazonaws.com/putMedia
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: X-Amz-Date -> 20240712T042634Z
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: content-type -> application/json
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: Authorization -> AWS4-HMAC-SHA256 Credential=<credentials>/ap-southeast-2/kinesisvideo/aws4_request, SignedHeaders=connection;host;transfer-encoding;user-agent;x-amz-date;x-amzn-fragment-acknowledgment-required;x-amzn-fragment-timecode-type;x-amzn-producer-start-timestamp;x-amzn-stream-name, Signature=<signature>
<stream some data with putFrame>
[video_streamer_node-1] INFO - writeHeaderCallback(): RequestId: c7cbe5da-833c-2c79-98c1-5766cd414979
<stream more data with putFrame>
[video_streamer_node-1] DEBUG - postReadCallback(): [AbiCam] Pausing CURL read for upload handle: 0
<stream more data with putFrame>
[video_streamer_node-1] WARN - curlCompleteSync(): [<stream-name>] curl perform failed for url https://s-ca658586.kinesisvideo.ap-southeast-2.amazonaws.com/putMedia with result Transferred a partial file: transfer closed with outstanding read data remaining
[video_streamer_node-1] WARN - curlCompleteSync(): [<stream-name>] HTTP Error 0 : Response: (null)
[video_streamer_node-1] Request URL: https://s-ca658586.kinesisvideo.ap-southeast-2.amazonaws.com/putMedia
[video_streamer_node-1] Request Headers:
[video_streamer_node-1]     Authorization: AWS4-HMAC-SHA256 Credential=<credentials>/20240712/ap-southeast-2/kinesisvideo/aws4_request, SignedHeaders=connection;host;transfer-encoding;user-agent;x-amz-date;x-amzn-fragment-acknowledgment-required;x-amzn-fragment-timecode-type;x-amzn-producer-start-timestamp;x-amzn-stream-name, Signature=<signature>
[video_streamer_node-1] DEBUG - putStreamCurlHandler(): Network thread for Kinesis Video stream: <stream-name> with upload handle: 0 exited. http status: 0
[video_streamer_node-1] WARN - putStreamCurlHandler(): [<stream-name>] Stream with streamHandle 20284784 uploadHandle 0 has exited without triggering end-of-stream. Service call result: 10006
[video_streamer_node-1] INFO - kinesisVideoStreamTerminated(): Stream 0x1358570 terminated upload handle 0 with service call result 10006.
[video_streamer_node-1] DEBUG - defaultStreamStateTransitionHook(): Stream state machine retry count: 0
[video_streamer_node-1] DEBUG - defaultStreamStateTransitionHook():
[video_streamer_node-1]  KinesisVideoStream base result is [10006]. Executing KVS retry handler of retry strategy type [1]
[video_streamer_node-1] DEBUG - defaultStreamStateTransitionHook(): Stream state machine retry count: 1
[video_streamer_node-1] DEBUG - defaultStreamStateTransitionHook():
[video_streamer_node-1]  KinesisVideoStream base result is [10006]. Executing KVS retry handler of retry strategy type [1]
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: user-agent -> AWS-SDK-KVS/1.5.2 GCC/13.2.0 Linux/5.15.153.1-microsoft-standard-WSL2 x86_64 CPPSD
nilp-dromeda commented 1 month ago

After some more tinkering, I resorted to directly using the camera stream after encoding it using software H264 encoding (libx264) from ffmpeg. However, now I seem to be dropping a lot of frames with the below errors occurring:

[video_streamer_node-1] WARN - viewItemRemoved(): Reporting a dropped frame/fragment.

and

[video_streamer_node-1] DEBUG - storageOverflowPressureHandler invoked

Which might be a result of this error:

postWriteCallback(): Curl post body write function for stream with handle: <stream-name> and upload handle: 1 returned: {"EventType":"ERROR","FragmentTimecode":0,"FragmentNumber":"91343852333181491818755708519218936382025283882","ErrorCode":"FRAGMENT_TIMECODE_LESSER_THAN_PREVIOUS","ErrorId":4004}

And then I am also getting insane fps for some reason:

[video_streamer_node-1] DEBUG - Kinesis Video client and stream metrics for <stream-name>
[video_streamer_node-1]         >> Overall storage byte size: 268435456
[video_streamer_node-1]         >> Available storage byte size: 1291042
[video_streamer_node-1]         >> Allocated storage byte size: 267144414
[video_streamer_node-1]         >> Total view allocation byte size: 144080
[video_streamer_node-1]         >> Total streams elementary frame rate (fps): 10000000
[video_streamer_node-1]         >> Total streams transfer rate (bps): 14424456 (14086 Kbps)
[video_streamer_node-1]         >> Current view duration (ms): 0
[video_streamer_node-1]         >> Overall view duration (ms): 0
[video_streamer_node-1]         >> Current view byte size: 267096126
[video_streamer_node-1]         >> Overall view byte size: 267096126
[video_streamer_node-1]         >> Current elementary frame rate (fps): 1e+07
[video_streamer_node-1]         >> Current transfer rate (bps): 14424456 (14086 Kbps)

Another question with regards to the initial title of this question - is there a way to see if data is being received by the AWS Kinesis Video Stream online even though it might not be seen through the console as it is not H264 encoded?

nilp-dromeda commented 3 weeks ago

The extremely high fps was due to incorrect time stamps being sent to KVS in the producer side - this was corrected by using the correct time stamps. I am now running into the problem of the stream on KVS side not being extremely stable. It runs for a few seconds and then starts to buffer with the following message "Searching for media - Your media preview will start once data is retrieved from the video" for around 1 second and then plays a chunk of the stream and them begins to buffer again. Also, the video is extremely patchy, with a lot of frames being dropped. I do sometimes see the overflow pressure callback being invoked and the dropped frame callback also being invoked, but I have kept the buffer size on the producer size to 200 seconds, which is an extremely large amount of time.