[QUESTION] Command for streaming file source

[ishangupta3]

I am trying to stream from a local file source to aws Kinesis video stream. I believe I am using the wrong command, can someone share the appropriate command to stream from a local file source (.mp4).

gst-launch-1.0 filesrc location=/deepstream-5.0/samples/streams/sample_1080p_h264.mp4 ! videoconvert ! video/x-raw,width=1920,height=1080 ! x264enc bframes=0 key-int-max=45 bitrate=512 tune=zerolatency ! h264parse ! video/x-h264,stream-format=avc,alignment=au,profile=baseline ! kvssink stream-name="MyKVStream" storage-size=512 access-key= secret-key= aws-region="us-west-2

[disa6302]

Hello @ishangupta3,

Could you try this pipeline?

https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/blob/master/docs/macos.md#running-the-gst-launch-10-command-to-upload-mp4-file-that-contains-both-audio-and-video-in-mac-os . This is for macos, but you could try it for the system you are running in.

One more thing, I have edited your question to remove the access key and secret key you added. Make sure you do not disclose those details in the future. Thanks!

[ishangupta3]

Thank you for doing that. I will try that command, I am using Ubuntu 18.04.

[ishangupta3]

@disa6302 I ran the command for the mp4, while it did say it was writing packets to AWS Kinesis Video Stream - sorry for the long error message. But it seems to send packets successfully then it just stops with an error:

Current transfer rate (bps): 31876704 (31129 Kbps) 2020-07-29 19:34:25 [547630256624] INFO - stopKinesisVideoStreamSync(): Synchronously stopping Kinesis Video Stream 00000055853ba5a8. 2020-07-29 19:34:25 [547621863920] DEBUG - postReadCallback(): Wrote 65524 bytes to Kinesis Video. Upload stream \ 2020-07-29 19:34:36 [547524432368] DEBUG - postReadCallback(): Wrote 65524 bytes to Kinesis Video. Upload stream handle: 0 2020-07-29 19:34:36 [547524432368] DEBUG - postReadCallback(): Wrote 65524 bytes to Kinesis Video. Upload stream handle: 0 2020-07-29 19:34:40 [547524432368] DEBUG - postReadCallback(): Wrote 65524 bytes to Kinesis Video. Upload stream handle: 0 2020-07-29 19:34:40 [547524432368] DEBUG - postReadCallback(): Wrote 65524 bytes to Kinesis Video. Upload stream handle: 0 2020-07-29 19:34:40 [547524432368] DEBUG - postReadCallback(): Wrote 65524 bytes to Kinesis Video. Upload stream handle: 0 2020-07-29 19:34:40 [547524432368] DEBUG - postReadCallback(): Wrote 65524 bytes to Kinesis Video. Upload stream handle: 0 2020-07-29 19:34:40 [547524432368] DEBUG - postReadCallback(): Wrote 65524 bytes to Kinesis Video. Upload stream handle: 0 2020-07-29 19:34:40 [547524432368] DEBUG - postWriteCallback(): Curl post body write function for stream with handle: MyKVStream and upload handle: 0 returned: {"EventType":"ERROR","FragmentTimecode":1596076463021,"FragmentNumber":"91343852333181447247962534054916022264923153650","ErrorCode":"MAX_FRAGMENT_DURATION_REACHED","ErrorId":4002}

2020-07-29 19:34:40 [547524432368] DEBUG - streamErrorHandler invoked 2020-07-29 19:34:40 [547524432368] ERROR - Reported stream error. Errored timecode: 0 Status: 0x5200005e 2020-07-29 19:34:40 [547524432368] WARN - continuousRetryStreamErrorReportHandler(): Reporting stream error. Errored timecode: 0 Status: 0x5200005e 2020-07-29 19:34:40 [547524432368] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000100, Next state: 0x0000000000000200 2020-07-29 19:34:40 [547524432368] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000200, Next state: 0x0000000000000001 2020-07-29 19:34:40 [547524432368] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000001, Next state: 0x0000000000000002 2020-07-29 19:34:40 [547524432368] INFO - describeStreamResultEvent(): Describe stream result event. 2020-07-29 19:34:40 [547524432368] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000002, Next state: 0x0000000000000020 2020-07-29 19:34:40 [547524432368] INFO - getStreamingEndpointResultEvent(): Get streaming endpoint result event. 2020-07-29 19:34:40 [547524432368] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000020, Next state: 0x0000000000000010 2020-07-29 19:34:40 [547524432368] DEBUG - getStreamingTokenHandler invoked 2020-07-29 19:34:40 [547524432368] DEBUG - Refreshing credentials. Force refreshing: 1 Now time is: 1596076480881602000 Expiration: 18446744073709551615 2020-07-29 19:34:40 [547524432368] INFO - getStreamingTokenResultEvent(): Get streaming token result event. 2020-07-29 19:34:40 [547524432368] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000010, Next state: 0x0000000000000040 2020-07-29 19:34:40 [547524432368] DEBUG - streamReadyHandler invoked 2020-07-29 19:34:40 [547524432368] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000040, Next state: 0x0000000000000080 2020-07-29 19:34:40 [547524432368] INFO - putStreamResultEvent(): Put stream result event. New upload handle 1 2020-07-29 19:34:40 [547524432368] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000080, Next state: 0x0000000000000100 2020-07-29 19:34:40 [547524432368] DEBUG - fragmentAckReceivedHandler invoked 2020-07-29 19:34:40 [547524432368] WARN - getStreamData(): Indicating an end-of-stream for a terminated stream upload handle 0 2020-07-29 19:34:40 [547524432368] INFO - postReadCallback(): Reported end-of-stream for stream MyKVStream. Upload handle: 0 2020-07-29 19:34:40 [547524432368] DEBUG - postReadCallback(): Wrote 0 bytes to Kinesis Video. Upload stream handle: 0 2020-07-29 19:34:40 [547524432368] DEBUG - putStreamCurlHandler(): Network thread for Kinesis Video stream: MyKVStream with upload handle: 0 exited. http status: 200 2020-07-29 19:34:40 [547524432368] INFO - writeHeaderCallback(): RequestId: e7b63c34-34bc-2163-b02a-083566748f0e 2020-07-29 19:34:41 [547516039664] DEBUG - postReadCallback(): Wrote 65524 bytes to Kinesis Video. Upload stream handle: 1 2020-07-29 19:34:41 [547516039664] DEBUG - postReadCallback(): Wrote 65524 bytes to Kinesis Video. Upload stream handle: 1 2020-07-29 19:34:41 [547516039664] DEBUG - postWriteCallback(): Curl post body write function for stream with handle: MyKVStream and upload handle: 1 returned: {"EventType":"ERROR","ErrorCode":"INVALID_MKV_DATA","ErrorId":4006}

2020-07-29 19:34:41 [547516039664] DEBUG - streamErrorHandler invoked 2020-07-29 19:34:41 [547516039664] ERROR - Reported stream error. Errored timecode: 0 Status: 0x52000062 2020-07-29 19:34:41 [547516039664] WARN - continuousRetryStreamErrorReportHandler(): Reporting stream error. Errored timecode: 0 Status: 0x52000062 2020-07-29 19:34:41 [547516039664] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000100, Next state: 0x0000000000000200 2020-07-29 19:34:41 [547516039664] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000200, Next state: 0x0000000000000001 2020-07-29 19:34:41 [547516039664] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000001, Next state: 0x0000000000000002 2020-07-29 19:34:41 [547516039664] INFO - describeStreamResultEvent(): Describe stream result event. 2020-07-29 19:34:41 [547516039664] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000002, Next state: 0x0000000000000020 2020-07-29 19:34:41 [547516039664] INFO - getStreamingEndpointResultEvent(): Get streaming endpoint result event. 2020-07-29 19:34:41 [547516039664] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000020, Next state: 0x0000000000000010 2020-07-29 19:34:41 [547516039664] DEBUG - getStreamingTokenHandler invoked 2020-07-29 19:34:41 [547516039664] DEBUG - Refreshing credentials. Force refreshing: 1 Now time is: 1596076481307465573 Expiration: 18446744073709551615 2020-07-29 19:34:41 [547516039664] INFO - getStreamingTokenResultEvent(): Get streaming token result event. 2020-07-29 19:34:41 [547516039664] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000010, Next state: 0x0000000000000040 2020-07-29 19:34:41 [547516039664] DEBUG - streamReadyHandler invoked 2020-07-29 19:34:41 [547516039664] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000040, Next state: 0x0000000000000080 2020-07-29 19:34:41 [547516039664] INFO - putStreamResultEvent(): Put stream result event. New upload handle 2 2020-07-29 19:34:41 [547516039664] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000080, Next state: 0x0000000000000100 2020-07-29 19:34:41 [547516039664] DEBUG - fragmentAckReceivedHandler invoked 2020-07-29 19:34:41 [547516039664] WARN - getStreamData(): Indicating an end-of-stream for a terminated stream upload handle 1 2020-07-29 19:34:41 [547516039664] INFO - postReadCallback(): Reported end-of-stream for stream MyKVStream. Upload handle: 1 2020-07-29 19:34:41 [547516039664] DEBUG - postReadCallback(): Wrote 0 bytes to Kinesis Video. Upload stream handle: 1 2020-07-29 19:34:41 [547516039664] DEBUG - putStreamCurlHandler(): Network thread for Kinesis Video stream: MyKVStream with upload handle: 1 exited. http status: 200 2020-07-29 19:34:41 [547516039664] INFO - writeHeaderCallback(): RequestId: d3f0731a-2cec-5d89-846c-471b7dff844d 2020-07-29 19:34:41 [547524432368] DEBUG - postReadCallback(): Wrote 65524 bytes to Kinesis Video. Upload stream handle: 2 2020-07-29 19:34:41 [547524432368] DEBUG - postReadCallback(): Wrote 65524 bytes to Kinesis Video. Upload stream handle: 2 2020-07-29 19:34:41 [547524432368] DEBUG - postReadCallback(): Wrote 65524 bytes to Kinesis Video. Upload stream handle: 2 2020-07-29 19:34:41 [547524432368] DEBUG - postWriteCallback(): Curl post body write function for stream with handle: MyKVStream and upload handle: 2 returned: {"EventType":"ERROR","ErrorCode":"INVALID_MKV_DATA","ErrorId":4006}

[MushMal]

You are getting STATUS_ACK_ERR_FRAGMENT_DURATION_REACHED which indicates thats your pipeline is not producing I-frames which will drive the fragmentation. You might want to re-encode your file to have frequent fragments (less than 10 seconds long).

[ishangupta3]

@MushMal thanks for the response, I will do that. Also, do you have any resources where I can better understand the different errors for this plugin?

[MushMal]

The documentation covers most of them: https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/producer-sdk-errors.html

However, the documentation could go stale due to fast paced development so if there is an error you don’t see there you can check the public include files in the native repository

[ishangupta3]

yessir thank you @MushMal and @disa6302

[MushMal]

I am going to resolve this issue but I encourage re-opening issues or cutting new ones to get help directly from us or from community members.