awslabs / amazon-kinesis-video-streams-producer-sdk-cpp

Amazon Kinesis Video Streams Producer SDK for C++ is for developers to install and customize for their connected camera and other devices to securely stream video, audio, and time-encoded data to Kinesis Video Streams.
Apache License 2.0
374 stars 336 forks source link

ErrorCode":"INVALID_MKV_DATA","ErrorId":4006 #315

Closed indoorrobot closed 4 years ago

indoorrobot commented 4 years ago

Hi I am constantly streaming video to KVS (from 2 h264 cameras) using gst, but after a few seconds / minutes I am getting the error:

{"EventType":"ERROR","ErrorCode":"INVALID_MKV_DATA","ErrorId":4006}

See logs:

Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - defaultDroppedFrameReportCallback(): Reported droppedFrame callback for stream handle 27000953. Dropped frame timecode in 100ns: 10950330000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - droppedFrameReportHandler invoked
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] WARN - Reported droppedFrame callback for stream handle 27000953. Dropped frame timecode in 100ns: 10950330000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - defaultStreamLatencyPressureCallback(): Reported streamLatencyPressure callback for stream handle 27000953. Current buffer duration in 100ns: 927670000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - streamLatencyPressureHandler invoked
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - Reported streamLatencyPressure callback for stream handle 27000953. Current buffer duration in 100ns: 927670000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] WARN - viewItemRemoved(): Reporting a dropped frame/fragment.
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - defaultDroppedFrameReportCallback(): Reported droppedFrame callback for stream handle 27000953. Dropped frame timecode in 100ns: 10950660000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - droppedFrameReportHandler invoked
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] WARN - Reported droppedFrame callback for stream handle 27000953. Dropped frame timecode in 100ns: 10950660000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - defaultStreamLatencyPressureCallback(): Reported streamLatencyPressure callback for stream handle 27000953. Current buffer duration in 100ns: 927710000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - streamLatencyPressureHandler invoked
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - Reported streamLatencyPressure callback for stream handle 27000953. Current buffer duration in 100ns: 927710000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] WARN - viewItemRemoved(): Reporting a dropped frame/fragment.
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - defaultDroppedFrameReportCallback(): Reported droppedFrame callback for stream handle 27000953. Dropped frame timecode in 100ns: 10950990000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - droppedFrameReportHandler invoked
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] WARN - Reported droppedFrame callback for stream handle 27000953. Dropped frame timecode in 100ns: 10950990000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - defaultStreamLatencyPressureCallback(): Reported streamLatencyPressure callback for stream handle 27000953. Current buffer duration in 100ns: 927700000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - streamLatencyPressureHandler invoked
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - Reported streamLatencyPressure callback for stream handle 27000953. Current buffer duration in 100ns: 927700000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] WARN - viewItemRemoved(): Reporting a dropped frame/fragment.
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - defaultDroppedFrameReportCallback(): Reported droppedFrame callback for stream handle 27000953. Dropped frame timecode in 100ns: 10951330000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - droppedFrameReportHandler invoked
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] WARN - Reported droppedFrame callback for stream handle 27000953. Dropped frame timecode in 100ns: 10951330000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - defaultStreamLatencyPressureCallback(): Reported streamLatencyPressure callback for stream handle 27000953. Current buffer duration in 100ns: 927700000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:459913431840512] DEBUG - Reported streamLatencyPressure callback for stream handle 10741817. Current buffer duration in 100ns: 924580000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913431840512] DEBUG - postReadCallback(): Wrote 16372 bytes to Kinesis Video. Upload stream handle: 205
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913320244992] DEBUG - postWriteCallback(): Curl post body write function for stream with handle: Device05_rc and upload handle: 205 returned: {"EventType":"ERROR","ErrorCode":"INVALID_MKV_DATA","ErrorId":4006}
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913320244992] DEBUG - defaultStreamErrorReportCallback(): Reported streamError callback for stream handle 10741817. Upload handle 205. Fragment timecode in 100ns: 10620330000. Error status: 0x52000062
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913320244992] DEBUG - streamErrorHandler invoked
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913320244992] ERROR - Reported stream error. Errored timecode: 10620330000 Status: 0x52000062
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913320244992] WARN - continuousRetryStreamErrorReportHandler(): Reporting stream error. Errored timecode: 10620330000 Status: 0x52000062
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913320244992] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000100, Next state: 0x0000000000000200
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913320244992] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000200, Next state: 0x0000000000000001
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913320244992] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000001, Next state: 0x0000000000000002
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913320244992] DEBUG - fragmentAckReceivedHandler invoked
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913320244992] WARN - getStreamData(): Indicating an end-of-stream for a terminated stream upload handle 205
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913320244992] INFO - postReadCallback(): Reported end-of-stream for stream Device05_rc. Upload handle: 205
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913320244992] DEBUG - postReadCallback(): Wrote 0 bytes to Kinesis Video. Upload stream handle: 205
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913320244992] DEBUG - putStreamCurlHandler(): Network thread for Kinesis Video stream: Device05_rc with upload handle: 205 exited. http status: 200
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913320244992] DEBUG - defaultStreamLatencyPressureCallback(): Reported streamLatencyPressure callback for stream handle 10741817. Current buffer duration in 100ns: 925180000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913431840512] DEBUG - streamLatencyPressureHandler invoked
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913431840512] DEBUG - Reported streamLatencyPressure callback for stream handle 10741817. Current buffer duration in 100ns: 925180000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913431840512] DEBUG - defaultStreamLatencyPressureCallback(): Reported streamLatencyPressure callback for stream handle 10741817. Current buffer duration in 100ns: 925500000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913431840512] DEBUG - streamLatencyPressureHandler invoked
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913431840512] DEBUG - Reported streamLatencyPressure callback for stream handle 10741817. Current buffer duration in 100ns: 925500000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913431840512] DEBUG - defaultStreamLatencyPressureCallback(): Reported streamLatencyPressure callback for stream handle 10741817. Current buffer duration in 100ns: 925820000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913431840512] DEBUG - streamLatencyPressureHandler invoked
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913431840512] DEBUG - Reported streamLatencyPressure callback for stream handle 10741817. Current buffer duration in 100ns: 925820000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913431840512] DEBUG - defaultStreamLatencyPressureCallback(): Reported streamLatencyPressure callback for stream handle 10741817. Current buffer duration in 100ns: 926100000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913431840512] DEBUG - streamLatencyPressureHandler invoked
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913431840512] DEBUG - Reported streamLatencyPressure callback for stream handle 10741817. Current buffer duration in 100ns: 926100000
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139913431840512] DEBUG - defaultStreamLatencyPressureCallback(): Reported streamLatencyPressure callback for stream handle 10741817. Current buffer duration in 100 [139807149221632] DEBUG - streamLatencyPressureHandler invoked
Dec 26 10:50:45 Device05 robot[1875]: 2019-12-26 10:50:45 [139807149221632] DEBUG - Reported streamLatencyPressure callback for stream handle 27000953. Current buffer duration in 100ns: 927700000

The way I am running the stream is using a bash script:

#!/bin/bash
while true
do
    if ! pgrep -f "_rc access-key=" > /dev/null
    then
        echo "Running rear stream"
        gst-launch-1.0 -v v4l2src device=/dev/video1 ! h264parse ! video/x-h264,stream-frmat=avc,alignment=au,width=1920,height=1080,framerate=30/1 ! kvssink name=sink stream-name=${ROBOT_CLOUD_ID}_rc access-key=$AWS_ACCESS_KEY_ID secret-key=$AWS_SECRET_ACCESS_KEY  > /dev/null 2>&1 &
    fi
    if ! pgrep -f "_fc access-key=" > /dev/null
    then
        echo "Running front stream"
        gst-launch-1.0 -v v4l2src device=/dev/video3 ! h264parse ! video/x-h264,stream-frmat=avc,alignment=au,width=1920,height=1080,framerate=30/1 ! kvssink name=sink stream-name=${ROBOT_CLOUD_ID}_fc access-key=$AWS_ACCESS_KEY_ID secret-key=$AWS_SECRET_ACCESS_KEY  > /dev/null 2>&1 &
    fi
    sleep 4s
done

What am I missing?

zhiyua-git commented 4 years ago

Hi @indoorrobot ,

To work around this you can make code changes as below, this will stop the gstreamer pipeline when it hits stream latency pressure. You can tune the setting by adding "max-latency=n" to your gstreamer pipeline to get stream latency setting to n seconds like gst-launch-1.0 -v v4l2src device=/dev/video3 ! h264parse ! video/x-h264,stream-frmat=avc,alignment=au,width=1920,height=1080,framerate=30/1 ! kvssink name=sink stream-name=${ROBOT_CLOUD_ID}_fc access-key=$AWS_ACCESS_KEY_ID secret-key=$AWS_SECRET_ACCESS_KEY max-latency=10. Please note in this case, the data in the buffer remaining for that many seconds would be dropped.

diff --git a/kinesis-video-gstreamer-plugin/src/gstkvssink.cpp b/kinesis-video-gstreamer-plugin/src/gstkvssink.cpp
index 046f5fa..2d556a9 100644
--- a/kinesis-video-gstreamer-plugin/src/gstkvssink.cpp
+++ b/kinesis-video-gstreamer-plugin/src/gstkvssink.cpp
@@ -1053,9 +1053,11 @@ gst_kvs_sink_handle_buffer (GstCollectPads * pads,
             goto CleanUp;
         } else {
             // resetStream, note that this will flush out producer buffer
-            data->kinesis_video_stream->resetStream();
+            //data->kinesis_video_stream->resetStream();
             // reset state
-            data->stream_status = STATUS_SUCCESS;
+            //data->stream_status = STATUS_SUCCESS;
+            ret = GST_FLOW_ERROR;
+            goto CleanUp;
         }
     }
diff --git a/kinesis-video-gstreamer-plugin/src/KvsSinkStreamCallbackProvider.cpp b/kinesis-video-gstreamer-plugin/src/KvsSinkStreamCallbackProvider.cpp
index afbe275..9d3560b 100644
--- a/kinesis-video-gstreamer-plugin/src/KvsSinkStreamCallbackProvider.cpp
+++ b/kinesis-video-gstreamer-plugin/src/KvsSinkStreamCallbackProvider.cpp
@@ -34,7 +34,7 @@ STATUS
 KvsSinkStreamCallbackProvider::droppedFrameReportHandler(UINT64 custom_data,
                                                          STREAM_HANDLE stream_handle,
                                                          UINT64 dropped_frame_timecode) {
-    UNUSED_PARAM(custom_data);
+    ((CustomData*) (custom_data))->stream_status = STATUS_INTERNAL_ERROR;
     LOG_WARN("Reported droppedFrame callback for stream handle " << stream_handle << ". Dropped frame timecode in 100ns: " << dropped_frame_timecode);
     return STATUS_SUCCESS; // continue streaming
 }
@@ -43,7 +43,7 @@ STATUS
 KvsSinkStreamCallbackProvider::streamLatencyPressureHandler(UINT64 custom_data,
                                                             STREAM_HANDLE stream_handle,
                                                             UINT64 current_buffer_duration) {
-    UNUSED_PARAM(custom_data);
+    ((CustomData*) (custom_data))->stream_status = STATUS_INTERNAL_ERROR;
     LOG_DEBUG("Reported streamLatencyPressure callback for stream handle " << stream_handle << ". Current buffer duration in 100ns: " << current_buffer_duration);
     return STATUS_SUCCESS;
 }

We will work on improving the producer SDK retry behavior to auto-recover on these case.

Thanks, -ZH

Nomidia commented 4 years ago

@zhiyua-git Hi, I have the same error code when I use kinesis-video-c-producer on my embeded device. it appears as soon as the program starts. I have no idea why. But it succeed on my ubuntu. Can you please give me some advices?Thanks

16:33:30 2020-01-10 08:33:28 DEBUG Key frame file /var/sdcard/h264SampleFrames/frame-001.h264, size 119387 16:33:30 2020-01-10 08:33:28 DEBUG stepStateMachine(): State Machine - Current state: 0x0000000000000040, Next state: 0x0000000000000080 16:33:30 2020-01-10 08:33:28 DEBUG putStreamResultEvent(): Put stream result event. 16:33:30 2020-01-10 08:33:28 DEBUG stepStateMachine(): State Machine - Current state: 0x0000000000000080, Next state: 0x0000000000000100 16:33:30 2020-01-10 08:33:28 DEBUG logStreamMetric(): Kinesis Video client and stream metrics: 16:33:30 Overall storage byte size: 10485760 16:33:30 Available storage byte size: 10366044 16:33:30 Allocated storage byte size: 119716 16:33:30 Total view allocation byte size: 345672 16:33:30 Total streams frame rate (fps): 120 16:33:30 Total streams transfer rate (bps): 16777216 (16384 Kbps) 16:33:30 Current view duration (ms): 0 16:33:30 Overall view duration (ms): 0 16:33:30 Current view byte size: 0 16:33:30 Overall view byte size: 0 16:33:30 Current frame rate (fps): 120.000000 16:33:30 Current transfer rate (bps): 16777216 (16384 Kbps) 16:33:32 2020-01-10 08:33:29 DEBUG writeHeaderCallback(): RequestId: e10e28bb-249e-c84a-b79c-11a076cf00e6 16:33:32 2020-01-10 08:33:29 DEBUG postReadCallback(): Wrote 16372 bytes to Kinesis Video. Upload stream handle: 0 16:33:32 2020-01-10 08:33:30 DEBUG postReadCallback(): Wrote 16372 bytes to Kinesis Video. Upload stream handle: 0 16:33:32 2020-01-10 08:33:30 DEBUG postReadCallback(): Wrote 16372 bytes to Kinesis Video. Upload stream handle: 0 16:33:32 2020-01-10 08:33:30 DEBUG postWriteCallback(): Curl post body write function for stream with handle: ExampleStream and upload handle: 0 returned: {"EventType":"ERROR","ErrorCode":"INVALID_MKV_DATA","ErrorId":4006} 16:33:32 2020-01-10 08:33:30 DEBUG defaultStreamErrorReportCallback(): Reported streamError callback for stream handle 33717816. Upload handle 0. Fragment timecode in 100ns: 15786452083350000. Error status: 0x52000062 16:33:32 2020-01-10 08:33:30 DEBUG continuousRetryStreamErrorReportHandler(): Reporting stream error. Errored timecode: 15786452083350000 Status: 0x52000062

zhiyua-git commented 4 years ago

Hi @Nomidia ,

Can you provide more information like your stream arn that you can find in the log and the platform of your embedded device? Thanks.

indoorrobot commented 4 years ago

Hi @zhiyua-git

After implementing the changes you suggested, I see that continuousRetryStreamErrorReportHandler is still being called and the process retries. How can I disable it so I would rerun the process manually?

zhiyua-git commented 4 years ago

Hi @indoorrobot ,

Remove this line https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/blob/master/kinesis-video-producer/src/DefaultCallbackProvider.cpp#L378 then it will be disabled. Thanks.

Nomidia commented 4 years ago

Hi @Nomidia ,

Can you provide more information like your stream arn that you can find in the log and the platform of your embedded device? Thanks.

Please take a look at this log, tks. "StreamARN":"arn:aws:kinesisvideo:us-east-1:491796068942:stream/ExampleStream/1576149365367","StreamName":"ExampleStream" user-agent: AWS-SDK-KVS/2.1.0 GCC/4.6.1 Linux/3.4.43-gk armv6l

kvsVideoOnly.log

zhiyua-git commented 4 years ago

Hi @Nomidia ,

Service log shows that parser failed when it try to parse the data you sent which contains 0x1549a9d0, 0x1549a9f4, 0x1549a994 or 0x1549a91c for an MKV element id. This looks like a memory corruption happening for a pre-set MKV element Segment Info which is [15][49][A9][66].

Are you running the sample on master branch/release version or other branches? With that information we can try to reproduce and fix the issue. Thanks.

Nomidia commented 4 years ago

Hi @zhiyua-git , I use the branch aligned_mem. https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/tree/aligned_mem

MushMal commented 4 years ago

@Nomidia this branch will not work - it's only designed to support WebRTC. We do not currently support aligned only memory access

Nomidia commented 4 years ago

@MushMal Yes, at the same time I am using webrtc. So I am trying to use aligned_mem to run kvs. What should I do now?

MushMal commented 4 years ago

WebRTC will use aligned_mem branch but it would only link specific sub-projects which have been modified to work with aligned memory. Whereas KVS Producer SDK will utilize all of the projects - including the content view, content store (heap) and mkv generator which were crafter to be fast but they work with unaligned memory. We are going to provide a proper handling for aligned only memory architectures.

Are you attempting to use KVS Producer SDK on the same device as WebRTC? Is there a way to enable kernel mode fix-up on this device?

Nomidia commented 4 years ago

Yes, I'm testing KVS and webrtc for cloud storage and real time preview, etc. there are some memory access issues. At the same time I will ask the chip manufacturer if they support setting up non-aligned memory access. Tks

indoorrobot commented 4 years ago

@MushMal per your suggestion I am now testing the aws WebRTC sdk, however it is not clear from the tutorial how to set the source as the camera (instead of the sample data). Is there any documentation I can read about it?

MushMal commented 4 years ago

@indoorrobot there is a GStreamer master sample available in the same directory - it demonstrates the source integration with GStreamer. What's your media pipeline? If it's something custom then you can use the Master sample as a template and extract frames from your custom media pipeline and feed them into the transeiver. Can you give more details please?

indoorrobot commented 4 years ago

Right now i am using the v4l2src to stream 2 h264 usb cams to kvs and would like to add a webrtc for the live view.

This is the pipeline i am currently using per camera for kvs:

gst-launch-1.0 -v v4l2src device=/dev/video3 ! h264parse ! video/x-h264,stream-frmat=avc,alignment=au,width=1920,height=1080,framerate=30/1 ! kvssink name=sink stream-name=${ROBOT_CLOUD_ID}_fc access-key=$AWS_ACCESS_KEY_ID secret-key=$AWS_SECRET_ACCESS_KEY

MushMal commented 4 years ago

Thanks. My question is more about what would you like the desired behavior to be of the KVS sink for your application 1) in case of no network temporarily 2) in case of no network for the device (no connectivity at all for long period of time) 3) in case of the network bandwidth not being enough - the buffer keeps filling up and needs to drop the frame

There are various remedies for this. Some applications call for 1) auto-recovery at all costs for scenario #2) attempt to re-stream but the application later picks up and sends an alarm to the operator or something (hard to implement in kvs plugin without GStreamer event bus support) 3) similar to #2 or automagically attempt to "thin" out the frames

indoorrobot commented 4 years ago

Hi @MushMal, of course it would be best if the stream would auto-recover by itself and flush all the aggregated buffer. The reason I opened this ticket in the first place was because sometimes the gst proc would simply get stuck - without streaming and the only way restart the stream would be restating the gst process. Since the process runs in a different thread, without seeing the logs coming out, and the fact that there is no other real way to know if it is streaming or not (at least not one that I could find in the documentation), I had to go to aws KVS console and actually check if there is a stream going on, or check the process's bandwidth usage and see that the process in not uploading any data. Once I saw that it did not work, I had to restart it.

Regarding my use case: I need a robust and as much as possible no-loss video stream. So even if something fails - low bandwidth / pipeline breaks / no connectivity / etc... I would need to make a local backup of the video (with a size cap of course) and whenever the connection is back or bandwidth is high again - flush it back to KVS and delete the local file. If it exceeds the cap, keep the local file and kill to process (can even add a cloudwatch alert).

Therefore, it can be every helpful to have an API for polling stream status, so that it can be invoked in order to know if there is data coming in. In addition, it could be quite useful in times when stream fails, to ingest video with some metadata such as start time and add it to the desired stream after time.

If you would like I can elaborate more in a private chat.

MushMal commented 4 years ago

@indoorrobot Thanks for the detailed info. We would love to hear more about your use case - please email kinesis-video-support@amazon.com and we would reply directly.

We are assessing the different approaches. As you've mentioned, the SDK should be resilient for temporary network outage - this is built in and should work as long as there is an ability to "catch-up" when the network is back and the throughput allows. There is also default "continuous" retry logic by default which should work in most cases. In case of 10 second max latency, the logic will break due to "pogo-stick" on recovery.

Some solutions we are looking at is to introduce different retry/streaming policies with different buffer configurations. For example, ability to configure a huge temporal buffer (multiple hours or days long) with a hybrid file-backed content store that can collect hours worth of data and stream faster than real-time to catch up when the networking is back.

Other solutions - as you've suggested is a policy of "back-fill" where the "dropped frames" are added to the persistent storage (on the flash/disk) and a secondary stream can be created by the application to stream at a later time/parallel to the real-time tip streaming.

In either case, we should look at allowing GStreamer pipeline events which would be sent as the SDK calls the callbacks on pressures/dropped frames so an application that solely uses GST plugin can react to.

We would love to discuss these approaches further.

chehefen commented 4 years ago

hi @indoorrobot, regarding an api for polling stream status, you can listen to the FragmentAckReceived callback: https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/blob/master/kinesis-video-c-producer/src/source/StreamCallbacksProvider.c#L77. As long as there are peristed ack coming in, you are guaranteed to be able to view the stream in the console. The ack type is a member of PFragmentAck.

MushMal commented 4 years ago

I am going to resolve this as stale.

Two updates: 1) We fully support aligned memory only model and 2) the code has been updated with multiple fixes and feature enhancements. Please give it a try.