search

awslabs / amazon-kinesis-video-streams-producer-sdk-cpp

Amazon Kinesis Video Streams Producer SDK for C++ is for developers to install and customize for their connected camera and other devices to securely stream video, audio, and time-encoded data to Kinesis Video Streams.
Apache License 2.0
377 stars 337 forks source link

Streaming to KVS using gst and iot-certificate [QUESTION] #609

Closed indoorrobot closed 3 years ago

indoorrobot commented 3 years ago

Hi, I am trying to stream to KVS from an IoT device without the need of settings AWS credentials.

I followed https://docs.aws.amazon.com/kinesisvideostreams/ and now using the following command:

gst-launch-1.0 rtspsrc location="rtsp://127.0.0.1:$FRONT_PORT/front" short-header=TRUE user-id="$FRONT_USERNAME" user-pw="$FRONT_PASS" ! rtph264depay ! video/x-h264, format=avc,alignment=au ! h264parse ! kvssink name=sink stream-name="_fc" iot-certificate="iot-certificate, endpoint=XXXXXXXXX.credentials.iot.eu-central-1.amazonaws.com, cert-path=$PATH_TO_CERT/certificate.pem.crt,key-path=$PATH_TO_CERT/private.pem.key,ca-path=$PATH_TO_CERT/rootCA.pem, role-aliases=KvsCameraIoTRoleAlias" retention-period=4320

What I get in response is:

Setting pipeline to PAUSED ...
log4cplus:ERROR could not open file ./kvs_log_configuration
DEBUG - Creating IoT auth callbacks.
2020-11-02 23:45:00 [139930241742592] 2020-11-02 21:45:00 ERROR   blockingCurlCall(): Curl call response failed with http status 403
ERROR - Unable to create Iot Credential provider. Error status: 0x15000011
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstKvsSink:sink: Could not initialize supporting library.
Additional debug info:
/amazon-kinesis-video-streams-producer-sdk-cpp/src/gstreamer/gstkvssink.cpp(1375): gst_kvs_sink_change_state (): /GstPipeline:pipeline0/GstKvsSink:sink:
Failed to init kvs producer. Error: Unable to create Iot Credential provider. Error status: 0x15000011
2020-11-02 23:45:00 [139930241742592] Setting pipeline to NULL ...
Freeing pipeline ...

the policy json I am using for KvsCameraIoTRoleAlias:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "kinesisvideo:*",
            "Resource": "*"
        }
    ]
}

What am I missing?

MushMal commented 3 years ago

You are doing the right thing it seems but you are getting permissions denied likely due to the policy document. Here is the actual documentation to generate the IoT credentials with the step-by-step instructions and samples.

https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/how-iot.html

indoorrobot commented 3 years ago

Hi @MushMal thanks for the quick reply! :)

This is what I followed and get the above response... I do not use the Resource Tag in the policy json (since the stream name is customized), is this required?

MushMal commented 3 years ago

That's a good question... Not entirely sure as I found out that the resulting policy is very "sensitive" to the JSON format. Do you mind following this guide very closely to get this working and then modify to your needs.

disa6302 commented 3 years ago

@indoorrobot ,

Any updates?

disa6302 commented 3 years ago

Closing since it has gone stale.

clogwog commented 2 years ago

i've got the same issue https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/781 seems to be related to not having the stream-name being the same as the thing name. @indoorrobot try changing your stream-name to be the same as the thingname...

sxj731533730 commented 1 year ago

i have also keep same name ,which use the stream-name being the same as the thing name , but i still encounter error

ubuntu@ubuntu:~/amazon-kinesis-video-streams-producer-sdk-cpp/build$ ./kvs_gstreamer_sample ai_facilities_eb4d475452 /home/ubuntu/Downloads/test_video/0.MOV [INFO ] [24-08-2023 04:42:34:324.942 GMT] Using region: ap-east-1 [INFO ] [24-08-2023 04:42:34:324.979 GMT] Using IoT credentials for Kinesis Video Streams [DEBUG] [24-08-2023 04:42:34:325.439 GMT] Creating IoT auth callbacks. 2023-08-24 04:42:34 ERROR blockingCurlCall(): Curl call response failed with http status 403 [ERROR] [24-08-2023 04:42:34:842.510 GMT] Unable to create Iot Credential provider. Error status: 0x15000025 [ERROR] [24-08-2023 04:42:34:842.889 GMT] Failed to initialize kinesis video with an exception: Unable to create Iot Credential provider. Error status: 0x15000025

sirknightj commented 1 year ago

Hi @sxj731533730, please try the following gstreamer command, which includes the iot-thing-name key-value pair in the iot-certificate parameter.

gst-launch-1.0 -v videotestsrc is-live=true \
  ! video/x-raw,framerate=30/1,width=720,height=480  \
  ! clockoverlay time-format="%a %B %d, %Y %I:%M:%S %p"  \
  ! x264enc  \
  ! h264parse  \
  ! kvssink stream-name="YourStreamName" iot-certificate="iot-certificate,endpoint=xx.credentials.iot.us-west-2.amazonaws.com,cert-path=xxx-certificate.pem.crt,key-path=xxx-private.pem.key,ca-path=AmazonRootCA1.pem,role-aliases=YourRoleAlias,iot-thing-name=YourThingName"

If the iot-thing-name parameter isn't present, kvssink will use the value of the stream-name parameter as the x-amzn-iot-thingname header in the curl call here, which may explain the 403 that you are encountering.

You can also manually make the CURL call yourself to confirm, and changing the value of the header.

curl --header "x-amzn-iot-thingname:YourThingName" \
  --cert xxx-certificate.pem.crt \
  --key xxx-private.pem.key \
  --cacert AmazonRootCA1.pem \
  https://xxx.credentials.iot.us-west-2.amazonaws.com/role-aliases/YourRoleAlias/credentials
sxj731533730 commented 1 year ago

Hello, as you said, I have added environment variables and I have been able to successfully push local mkv videos, but I am still unable to push rtsp and mp4 videos

Jeremy Gunawan @.***> 于2023年8月24日周四 14:25写道：

Hi @sxj731533730 https://github.com/sxj731533730, please try the following gstreamer command, which includes the iot-thing-name key-value pair in the iot-certificate parameter.

gst-launch-1.0 -v videotestsrc is-live=true \ ! video/x-raw,framerate=30/1,width=720,height=480 \ ! clockoverlay time-format="%a %B %d, %Y %I:%M:%S %p" \ ! x264enc \ ! h264parse \ ! kvssink stream-name="YourStreamName" iot-certificate="iot-certificate,endpoint=xx.credentials.iot.us-west-2.amazonaws.com,cert-path=xxx-certificate.pem.crt,key-path=xxx-private.pem.key,ca-path=AmazonRootCA1.pem,role-aliases=YourRoleAlias,iot-thing-name=YourThingName"

If the iot-thing-name parameter isn't present, kvssink will use the value of the stream-name parameter as the x-amzn-iot-thingname header in the curl call here https://docs.aws.amazon.com/iot/latest/developerguide/authorizing-direct-aws.html, which may explain the 403 that you are encountering.

You can also manually make the CURL call yourself to confirm, and changing the value of the header.

curl --header "x-amzn-iot-thingname:YourThingName" \ --cert xxx-certificate.pem.crt \ --key xxx-private.pem.key \ --cacert AmazonRootCA1.pem \ https://xxx.credentials.iot.us-west-2.amazonaws.com/role-aliases/YourRoleAlias/credentials

— Reply to this email directly, view it on GitHub https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609#issuecomment-1691075144, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGKU6EPRW4TO4ADCAFMXRB3XW3XURANCNFSM4TO45VMQ . You are receiving this because you were mentioned.Message ID: <awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609/1691075144 @github.com>

sxj731533730 commented 1 year ago

I fully refer to the operation of this video https://www.youtube.com/watch?v=nVxwX7Q9nPU&t=172s But I am running on a local physical machine

江流儿 @.***> 于2023年8月26日周六 09:49写道：

Hello, as you said, I have added environment variables and I have been able to successfully push local mkv videos, but I am still unable to push rtsp and mp4 videos

Jeremy Gunawan @.***> 于2023年8月24日周四 14:25写道：

Hi @sxj731533730 https://github.com/sxj731533730, please try the following gstreamer command, which includes the iot-thing-name key-value pair in the iot-certificate parameter.

gst-launch-1.0 -v videotestsrc is-live=true \ ! video/x-raw,framerate=30/1,width=720,height=480 \ ! clockoverlay time-format="%a %B %d, %Y %I:%M:%S %p" \ ! x264enc \ ! h264parse \ ! kvssink stream-name="YourStreamName" iot-certificate="iot-certificate,endpoint=xx.credentials.iot.us-west-2.amazonaws.com,cert-path=xxx-certificate.pem.crt,key-path=xxx-private.pem.key,ca-path=AmazonRootCA1.pem,role-aliases=YourRoleAlias,iot-thing-name=YourThingName"

If the iot-thing-name parameter isn't present, kvssink will use the value of the stream-name parameter as the x-amzn-iot-thingname header in the curl call here https://docs.aws.amazon.com/iot/latest/developerguide/authorizing-direct-aws.html, which may explain the 403 that you are encountering.

You can also manually make the CURL call yourself to confirm, and changing the value of the header.

curl --header "x-amzn-iot-thingname:YourThingName" \ --cert xxx-certificate.pem.crt \ --key xxx-private.pem.key \ --cacert AmazonRootCA1.pem \ https://xxx.credentials.iot.us-west-2.amazonaws.com/role-aliases/YourRoleAlias/credentials

— Reply to this email directly, view it on GitHub https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609#issuecomment-1691075144, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGKU6EPRW4TO4ADCAFMXRB3XW3XURANCNFSM4TO45VMQ . You are receiving this because you were mentioned.Message ID: <awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609/1691075144 @github.com>

sxj731533730 commented 1 year ago

when i run *.mp4 is error ,meanwhile the rtsp run ralso is error @.***:~/Downloads/amazon-kinesis-video-streams-producer-sdk-cpp/build$ ./kvs_gstreamer_sample sxj731533730_thing /home/ubuntu/demo.mp4 [INFO ] [26-08-2023 02:00:55:227.748 GMT] Using region: ap-east-1 [INFO ] [26-08-2023 02:00:55:227.795 GMT] Using IoT credentials for Kinesis Video Streams [DEBUG] [26-08-2023 02:00:55:228.417 GMT] Creating IoT auth callbacks. [INFO ] [26-08-2023 02:00:55:866.615 GMT] createKinesisVideoClient(): Creating Kinesis Video Client [INFO ] [26-08-2023 02:00:55:866.725 GMT] heapInitialize(): Initializing native heap with limit size 134217728, spill ratio 0% and flags 0x00000001 [INFO ] [26-08-2023 02:00:55:866.731 GMT] heapInitialize(): Creating AIV heap. [INFO ] [26-08-2023 02:00:55:866.754 GMT] heapInitialize(): Heap is initialized OK [INFO ] [26-08-2023 02:00:55:866.875 GMT] createDeviceResultEvent(): Create device result event. [DEBUG] [26-08-2023 02:00:55:866.895 GMT] clientReadyHandler invoked [DEBUG] [26-08-2023 02:00:55:866.929 GMT] Client is ready [INFO ] [26-08-2023 02:00:55:866.982 GMT] Creating Kinesis Video Stream sxj731533730_thing [INFO ] [26-08-2023 02:00:55:867.003 GMT] createKinesisVideoStream(): Creating Kinesis Video Stream. [INFO ] [26-08-2023 02:00:55:867.018 GMT] logStreamInfo(): SDK version: e8125ee363c4e77e11b15398a8b77f4e8a5791a5 [DEBUG] [26-08-2023 02:00:55:867.024 GMT] logStreamInfo(): Kinesis Video Stream Info [DEBUG] [26-08-2023 02:00:55:867.029 GMT] logStreamInfo(): Stream name: sxj731533730_thing [DEBUG] [26-08-2023 02:00:55:867.031 GMT] logStreamInfo(): Streaming type: STREAMING_TYPE_OFFLINE [DEBUG] [26-08-2023 02:00:55:867.037 GMT] logStreamInfo(): Content type: video/h264 [DEBUG] [26-08-2023 02:00:55:867.040 GMT] logStreamInfo(): Max latency (100ns): 0 [DEBUG] [26-08-2023 02:00:55:867.043 GMT] logStreamInfo(): Fragment duration (100ns): 20000000 [DEBUG] [26-08-2023 02:00:55:867.046 GMT] logStreamInfo(): Key frame fragmentation: Yes [DEBUG] [26-08-2023 02:00:55:867.055 GMT] logStreamInfo(): Use frame timecode: Yes [DEBUG] [26-08-2023 02:00:55:867.059 GMT] logStreamInfo(): Absolute frame timecode: Yes [DEBUG] [26-08-2023 02:00:55:867.063 GMT] logStreamInfo(): Nal adaptation flags: 0 [DEBUG] [26-08-2023 02:00:55:867.067 GMT] logStreamInfo(): Average bandwith (bps): 4194304 [DEBUG] [26-08-2023 02:00:55:867.070 GMT] logStreamInfo(): Framerate: 25 [DEBUG] [26-08-2023 02:00:55:867.073 GMT] logStreamInfo(): Buffer duration (100ns): 1200000000 [DEBUG] [26-08-2023 02:00:55:867.075 GMT] logStreamInfo(): Replay duration (100ns): 400000000 [DEBUG] [26-08-2023 02:00:55:867.078 GMT] logStreamInfo(): Connection Staleness duration (100ns): 0 [DEBUG] [26-08-2023 02:00:55:867.081 GMT] logStreamInfo(): Store Pressure Policy: 1 [DEBUG] [26-08-2023 02:00:55:867.083 GMT] logStreamInfo(): View Overflow Policy: 1 [DEBUG] [26-08-2023 02:00:55:867.086 GMT] logStreamInfo(): Segment UUID: NULL [DEBUG] [26-08-2023 02:00:55:867.089 GMT] logStreamInfo(): Frame ordering mode: 0 [DEBUG] [26-08-2023 02:00:55:867.091 GMT] logStreamInfo(): Track list [DEBUG] [26-08-2023 02:00:55:867.094 GMT] logStreamInfo(): Track id: 1 [DEBUG] [26-08-2023 02:00:55:867.097 GMT] logStreamInfo(): Track name: kinesis_video [DEBUG] [26-08-2023 02:00:55:867.099 GMT] logStreamInfo(): Codec id: V_MPEG4/ISO/AVC [DEBUG] [26-08-2023 02:00:55:867.102 GMT] logStreamInfo(): Track type: TRACK_INFO_TYPE_VIDEO [DEBUG] [26-08-2023 02:00:55:867.105 GMT] logStreamInfo(): Track cpd: NULL [INFO ] [26-08-2023 02:00:56:624.539 GMT] writeHeaderCallback(): RequestId: fc30711a-3628-4345-a442-8dc2c3cd0012 [DEBUG] [26-08-2023 02:00:56:624.598 GMT] describeStreamCurlHandler(): DescribeStream API response: [INFO ] [26-08-2023 02:00:56:625.247 GMT] describeStreamResultEvent(): Describe stream result event. [WARN ] [26-08-2023 02:00:56:625.275 GMT] describeStreamResult(): Retention period returned from the DescribeStream call doesn't match the one specified in the StreamInfo [WARN ] [26-08-2023 02:00:56:625.280 GMT] describeStreamResult(): Content type returned from the DescribeStream call doesn't match the one specified in the StreamInfo [INFO ] [26-08-2023 02:00:57:175.724 GMT] writeHeaderCallback(): RequestId: f17b41b4-016c-44b4-b574-a710a25e317d [DEBUG] [26-08-2023 02:00:57:175.776 GMT] getStreamingEndpointCurlHandler(): GetStreamingEndpoint API response: {"DataEndpoint":"https://s-24c98279.kinesisvideo.ap-east-1.amazonaws.com"} [INFO ] [26-08-2023 02:00:57:176.185 GMT] getStreamingEndpointResultEvent(): Get streaming endpoint result event. [INFO ] [26-08-2023 02:00:57:176.213 GMT] getStreamingTokenResultEvent(): Get streaming token result event. [DEBUG] [26-08-2023 02:00:57:176.228 GMT] streamReadyHandler invoked [DEBUG] [26-08-2023 02:00:57:176.332 GMT] Stream is ready [DEBUG] [26-08-2023 02:00:57:176.356 GMT] Attempt to upload file: /home/ubuntu/demo.mp4 [INFO ] [26-08-2023 02:00:57:186.379 GMT] Streaming from file source Error received from element demux: This file is invalid and cannot be played. Debugging information: qtdemux.c(775): gst_qtdemux_pull_atom (): /GstPipeline:file-kinesis-pipeline/GstQTDemux:demux: atom has bogus size 1229802312 [INFO ] [26-08-2023 02:00:57:191.967 GMT] Finished sending file to kvs producer: /home/ubuntu/demo.mp4 [INFO ] [26-08-2023 02:00:57:191.999 GMT] stopKinesisVideoStreamSync(): Synchronously stopping Kinesis Video Stream 0000555ddf7aa490. [DEBUG] [26-08-2023 02:00:57:192.022 GMT] streamClosedHandler invoked for upload handle: 18446744073709551615 [INFO ] [26-08-2023 02:00:57:192.038 GMT] All files have been persisted [INFO ] [26-08-2023 02:00:57:192.049 GMT] Freeing Kinesis Video Stream sxj731533730_thing [INFO ] [26-08-2023 02:00:57:192.065 GMT] freeKinesisVideoStream(): Freeing Kinesis Video stream. [DEBUG] [26-08-2023 02:00:57:192.074 GMT] curlApiCallbacksShutdownActiveRequests(): pActiveRequests hashtable is empty [INFO ] [26-08-2023 02:00:57:592.314 GMT] Freeing Kinesis Video Stream sxj731533730_thing [INFO ] [26-08-2023 02:00:57:592.383 GMT] freeKinesisVideoClient(): Freeing Kinesis Video Client [DEBUG] [26-08-2023 02:00:57:592.397 GMT] curlApiCallbacksShutdownActiveRequests(): pActiveRequests hashtable is empty [DEBUG] [26-08-2023 02:00:57:992.623 GMT] freeKinesisVideoClientInternal(): Total allocated memory 0 [WARN ] [26-08-2023 02:00:57:992.692 GMT] curlApiCallbacksShutdown(): curlApiCallbacksShutdown called when already in progress of shutting down

江流儿 @.***> 于2023年8月26日周六 09:51写道：

I fully refer to the operation of this video https://www.youtube.com/watch?v=nVxwX7Q9nPU&t=172s But I am running on a local physical machine

江流儿 @.***> 于2023年8月26日周六 09:49写道：

Hello, as you said, I have added environment variables and I have been able to successfully push local mkv videos, but I am still unable to push rtsp and mp4 videos

Jeremy Gunawan @.***> 于2023年8月24日周四 14:25写道：

Hi @sxj731533730 https://github.com/sxj731533730, please try the following gstreamer command, which includes the iot-thing-name key-value pair in the iot-certificate parameter.

gst-launch-1.0 -v videotestsrc is-live=true \ ! video/x-raw,framerate=30/1,width=720,height=480 \ ! clockoverlay time-format="%a %B %d, %Y %I:%M:%S %p" \ ! x264enc \ ! h264parse \ ! kvssink stream-name="YourStreamName" iot-certificate="iot-certificate,endpoint=xx.credentials.iot.us-west-2.amazonaws.com,cert-path=xxx-certificate.pem.crt,key-path=xxx-private.pem.key,ca-path=AmazonRootCA1.pem,role-aliases=YourRoleAlias,iot-thing-name=YourThingName"

If the iot-thing-name parameter isn't present, kvssink will use the value of the stream-name parameter as the x-amzn-iot-thingname header in the curl call here https://docs.aws.amazon.com/iot/latest/developerguide/authorizing-direct-aws.html, which may explain the 403 that you are encountering.

You can also manually make the CURL call yourself to confirm, and changing the value of the header.

curl --header "x-amzn-iot-thingname:YourThingName" \ --cert xxx-certificate.pem.crt \ --key xxx-private.pem.key \ --cacert AmazonRootCA1.pem \ https://xxx.credentials.iot.us-west-2.amazonaws.com/role-aliases/YourRoleAlias/credentials

— Reply to this email directly, view it on GitHub https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609#issuecomment-1691075144, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGKU6EPRW4TO4ADCAFMXRB3XW3XURANCNFSM4TO45VMQ . You are receiving this because you were mentioned.Message ID: <awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609/1691075144 @github.com>

sxj731533730 commented 1 year ago

the rtsp running is also error but the mkv video is normal for me to playback in aws @.:~/Downloads/amazon-kinesis-video-streams-producer-sdk-cpp/build$ ./kvs_gstreamer_sample sxj731533730_thing rtsp:// @.:554/h264/ch33/main/av_stream [INFO ] [26-08-2023 02:03:36:675.752 GMT] Using region: ap-east-1 [INFO ] [26-08-2023 02:03:36:675.790 GMT] Using IoT credentials for Kinesis Video Streams [DEBUG] [26-08-2023 02:03:36:678.067 GMT] Creating IoT auth callbacks. [INFO ] [26-08-2023 02:03:37:713.553 GMT] createKinesisVideoClient(): Creating Kinesis Video Client [INFO ] [26-08-2023 02:03:37:713.647 GMT] heapInitialize(): Initializing native heap with limit size 134217728, spill ratio 0% and flags 0x00000001 [INFO ] [26-08-2023 02:03:37:713.654 GMT] heapInitialize(): Creating AIV heap. [INFO ] [26-08-2023 02:03:37:713.674 GMT] heapInitialize(): Heap is initialized OK [INFO ] [26-08-2023 02:03:37:713.763 GMT] createDeviceResultEvent(): Create device result event. [DEBUG] [26-08-2023 02:03:37:713.778 GMT] clientReadyHandler invoked [DEBUG] [26-08-2023 02:03:37:713.810 GMT] Client is ready [INFO ] [26-08-2023 02:03:37:713.862 GMT] Creating Kinesis Video Stream sxj731533730_thing [INFO ] [26-08-2023 02:03:37:713.879 GMT] createKinesisVideoStream(): Creating Kinesis Video Stream. [INFO ] [26-08-2023 02:03:37:713.886 GMT] logStreamInfo(): SDK version: e8125ee363c4e77e11b15398a8b77f4e8a5791a5 [DEBUG] [26-08-2023 02:03:37:713.890 GMT] logStreamInfo(): Kinesis Video Stream Info [DEBUG] [26-08-2023 02:03:37:713.895 GMT] logStreamInfo(): Stream name: sxj731533730_thing [DEBUG] [26-08-2023 02:03:37:713.899 GMT] logStreamInfo(): Streaming type: STREAMING_TYPE_REALTIME [DEBUG] [26-08-2023 02:03:37:713.904 GMT] logStreamInfo(): Content type: video/h264 [DEBUG] [26-08-2023 02:03:37:713.908 GMT] logStreamInfo(): Max latency (100ns): 600000000 [DEBUG] [26-08-2023 02:03:37:713.913 GMT] logStreamInfo(): Fragment duration (100ns): 20000000 [DEBUG] [26-08-2023 02:03:37:713.918 GMT] logStreamInfo(): Key frame fragmentation: Yes [DEBUG] [26-08-2023 02:03:37:713.922 GMT] logStreamInfo(): Use frame timecode: Yes [DEBUG] [26-08-2023 02:03:37:713.926 GMT] logStreamInfo(): Absolute frame timecode: Yes [DEBUG] [26-08-2023 02:03:37:713.930 GMT] logStreamInfo(): Nal adaptation flags: 0 [DEBUG] [26-08-2023 02:03:37:713.935 GMT] logStreamInfo(): Average bandwith (bps): 4194304 [DEBUG] [26-08-2023 02:03:37:713.939 GMT] logStreamInfo(): Framerate: 25 [DEBUG] [26-08-2023 02:03:37:713.944 GMT] logStreamInfo(): Buffer duration (100ns): 1200000000 [DEBUG] [26-08-2023 02:03:37:713.949 GMT] logStreamInfo(): Replay duration (100ns): 400000000 [DEBUG] [26-08-2023 02:03:37:713.953 GMT] logStreamInfo(): Connection Staleness duration (100ns): 600000000 [DEBUG] [26-08-2023 02:03:37:713.957 GMT] logStreamInfo(): Store Pressure Policy: 1 [DEBUG] [26-08-2023 02:03:37:713.962 GMT] logStreamInfo(): View Overflow Policy: 1 [DEBUG] [26-08-2023 02:03:37:713.967 GMT] logStreamInfo(): Segment UUID: NULL [DEBUG] [26-08-2023 02:03:37:713.971 GMT] logStreamInfo(): Frame ordering mode: 0 [DEBUG] [26-08-2023 02:03:37:713.975 GMT] logStreamInfo(): Track list [DEBUG] [26-08-2023 02:03:37:713.982 GMT] logStreamInfo(): Track id: 1 [DEBUG] [26-08-2023 02:03:37:713.987 GMT] logStreamInfo(): Track name: kinesis_video [DEBUG] [26-08-2023 02:03:37:713.991 GMT] logStreamInfo(): Codec id: V_MPEG4/ISO/AVC [DEBUG] [26-08-2023 02:03:37:713.996 GMT] logStreamInfo(): Track type: TRACK_INFO_TYPE_VIDEO [DEBUG] [26-08-2023 02:03:37:714.000 GMT] logStreamInfo(): Track cpd: NULL [INFO ] [26-08-2023 02:03:38:279.605 GMT] writeHeaderCallback(): RequestId: dc3f1e3d-d648-4f35-8540-4618b18daf77 [DEBUG] [26-08-2023 02:03:38:279.687 GMT] describeStreamCurlHandler(): DescribeStream API response: {"StreamInfo":{"CreationTime":1.692878742352E9,"DataRetentionInHours":8760,"DeviceName":null,"IngestionConfiguration":null,"KmsKeyId":"arn:aws:kms:ap-east-1:733421296020:alias/aws/kinesisvideo","MediaType":null,"Status":"ACTIVE","StreamARN":"arn:aws:kinesisvideo:ap-east-1:733421296020:stream/sxj731533730_thing/1692878742352","StreamName":"sxj731533730_thing","Version":"A2Dmc6X5EfKcIkfLSflW"}} [INFO ] [26-08-2023 02:03:38:280.361 GMT] describeStreamResultEvent(): Describe stream result event. [WARN ] [26-08-2023 02:03:38:280.383 GMT] describeStreamResult(): Retention period returned from the DescribeStream call doesn't match the one specified in the StreamInfo [WARN ] [26-08-2023 02:03:38:280.405 GMT] describeStreamResult(): Content type returned from the DescribeStream call doesn't match the one specified in the StreamInfo [INFO ] [26-08-2023 02:03:38:729.998 GMT] writeHeaderCallback(): RequestId: 09fbe3e2-b6f1-4bdc-987d-a6d090c358aa [DEBUG] [26-08-2023 02:03:38:730.058 GMT] getStreamingEndpointCurlHandler(): GetStreamingEndpoint API response: {"DataEndpoint":"https://s-24c98279.kinesisvideo.ap-east-1.amazonaws.com"} [INFO ] [26-08-2023 02:03:38:730.450 GMT] getStreamingEndpointResultEvent(): Get streaming endpoint result event. [INFO ] [26-08-2023 02:03:38:730.466 GMT] getStreamingTokenResultEvent(): Get streaming token result event. [DEBUG] [26-08-2023 02:03:38:730.478 GMT] streamReadyHandler invoked [DEBUG] [26-08-2023 02:03:38:730.576 GMT] Stream is ready [INFO ] [26-08-2023 02:03:38:740.705 GMT] Streaming from rtsp source New pad found: recv_rtp_src_0_1336496155_96 [INFO ] [26-08-2023 02:03:38:885.120 GMT] Pad link failed New pad found: recv_rtp_src_1_1151119672_104 [INFO ] [26-08-2023 02:03:38:965.554 GMT] Pad link failed Error received from element udpsrc3: Internal data stream error. Debugging information: gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:rtsp-kinesis-pipeline/GstRTSPSrc:source/GstUDPSrc:udpsrc3: streaming stopped, reason not-linked (-1) [INFO ] [26-08-2023 02:03:39:037.722 GMT] stopKinesisVideoStreamSync(): Synchronously stopping Kinesis Video Stream 000055dbd3c9f650. [DEBUG] [26-08-2023 02:03:39:037.780 GMT] streamClosedHandler invoked for upload handle: 18446744073709551615 [INFO ] [26-08-2023 02:03:39:037.796 GMT] Freeing Kinesis Video Stream sxj731533730_thing [INFO ] [26-08-2023 02:03:39:037.809 GMT] freeKinesisVideoStream(): Freeing Kinesis Video stream. [DEBUG] [26-08-2023 02:03:39:037.815 GMT] curlApiCallbacksShutdownActiveRequests(): pActiveRequests hashtable is empty [INFO ] [26-08-2023 02:03:39:438.062 GMT] Freeing Kinesis Video Stream sxj731533730_thing [INFO ] [26-08-2023 02:03:39:438.113 GMT] freeKinesisVideoClient(): Freeing Kinesis Video Client [DEBUG] [26-08-2023 02:03:39:438.122 GMT] curlApiCallbacksShutdownActiveRequests(): pActiveRequests hashtable is empty [DEBUG] [26-08-2023 02:03:39:838.334 GMT] freeKinesisVideoClientInternal(): Total allocated memory 0 [WARN ] [26-08-2023 02:03:39:838.422 GMT] curlApiCallbacksShutdown(): curlApiCallbacksShutdown called when already in progress of shutting down @.*** :~/Downloads/amazon-kinesis-video-streams-producer-sdk-cpp/build$

江流儿 @.***> 于2023年8月26日周六 10:02写道：

when i run *.mp4 is error ,meanwhile the rtsp run ralso is error @.***:~/Downloads/amazon-kinesis-video-streams-producer-sdk-cpp/build$ ./kvs_gstreamer_sample sxj731533730_thing /home/ubuntu/demo.mp4 [INFO ] [26-08-2023 02:00:55:227.748 GMT] Using region: ap-east-1 [INFO ] [26-08-2023 02:00:55:227.795 GMT] Using IoT credentials for Kinesis Video Streams [DEBUG] [26-08-2023 02:00:55:228.417 GMT] Creating IoT auth callbacks. [INFO ] [26-08-2023 02:00:55:866.615 GMT] createKinesisVideoClient(): Creating Kinesis Video Client [INFO ] [26-08-2023 02:00:55:866.725 GMT] heapInitialize(): Initializing native heap with limit size 134217728, spill ratio 0% and flags 0x00000001 [INFO ] [26-08-2023 02:00:55:866.731 GMT] heapInitialize(): Creating AIV heap. [INFO ] [26-08-2023 02:00:55:866.754 GMT] heapInitialize(): Heap is initialized OK [INFO ] [26-08-2023 02:00:55:866.875 GMT] createDeviceResultEvent(): Create device result event. [DEBUG] [26-08-2023 02:00:55:866.895 GMT] clientReadyHandler invoked [DEBUG] [26-08-2023 02:00:55:866.929 GMT] Client is ready [INFO ] [26-08-2023 02:00:55:866.982 GMT] Creating Kinesis Video Stream sxj731533730_thing [INFO ] [26-08-2023 02:00:55:867.003 GMT] createKinesisVideoStream(): Creating Kinesis Video Stream. [INFO ] [26-08-2023 02:00:55:867.018 GMT] logStreamInfo(): SDK version: e8125ee363c4e77e11b15398a8b77f4e8a5791a5 [DEBUG] [26-08-2023 02:00:55:867.024 GMT] logStreamInfo(): Kinesis Video Stream Info [DEBUG] [26-08-2023 02:00:55:867.029 GMT] logStreamInfo(): Stream name: sxj731533730_thing [DEBUG] [26-08-2023 02:00:55:867.031 GMT] logStreamInfo(): Streaming type: STREAMING_TYPE_OFFLINE [DEBUG] [26-08-2023 02:00:55:867.037 GMT] logStreamInfo(): Content type: video/h264 [DEBUG] [26-08-2023 02:00:55:867.040 GMT] logStreamInfo(): Max latency (100ns): 0 [DEBUG] [26-08-2023 02:00:55:867.043 GMT] logStreamInfo(): Fragment duration (100ns): 20000000 [DEBUG] [26-08-2023 02:00:55:867.046 GMT] logStreamInfo(): Key frame fragmentation: Yes [DEBUG] [26-08-2023 02:00:55:867.055 GMT] logStreamInfo(): Use frame timecode: Yes [DEBUG] [26-08-2023 02:00:55:867.059 GMT] logStreamInfo(): Absolute frame timecode: Yes [DEBUG] [26-08-2023 02:00:55:867.063 GMT] logStreamInfo(): Nal adaptation flags: 0 [DEBUG] [26-08-2023 02:00:55:867.067 GMT] logStreamInfo(): Average bandwith (bps): 4194304 [DEBUG] [26-08-2023 02:00:55:867.070 GMT] logStreamInfo(): Framerate: 25 [DEBUG] [26-08-2023 02:00:55:867.073 GMT] logStreamInfo(): Buffer duration (100ns): 1200000000 [DEBUG] [26-08-2023 02:00:55:867.075 GMT] logStreamInfo(): Replay duration (100ns): 400000000 [DEBUG] [26-08-2023 02:00:55:867.078 GMT] logStreamInfo(): Connection Staleness duration (100ns): 0 [DEBUG] [26-08-2023 02:00:55:867.081 GMT] logStreamInfo(): Store Pressure Policy: 1 [DEBUG] [26-08-2023 02:00:55:867.083 GMT] logStreamInfo(): View Overflow Policy: 1 [DEBUG] [26-08-2023 02:00:55:867.086 GMT] logStreamInfo(): Segment UUID: NULL [DEBUG] [26-08-2023 02:00:55:867.089 GMT] logStreamInfo(): Frame ordering mode: 0 [DEBUG] [26-08-2023 02:00:55:867.091 GMT] logStreamInfo(): Track list [DEBUG] [26-08-2023 02:00:55:867.094 GMT] logStreamInfo(): Track id: 1 [DEBUG] [26-08-2023 02:00:55:867.097 GMT] logStreamInfo(): Track name: kinesis_video [DEBUG] [26-08-2023 02:00:55:867.099 GMT] logStreamInfo(): Codec id: V_MPEG4/ISO/AVC [DEBUG] [26-08-2023 02:00:55:867.102 GMT] logStreamInfo(): Track type: TRACK_INFO_TYPE_VIDEO [DEBUG] [26-08-2023 02:00:55:867.105 GMT] logStreamInfo(): Track cpd: NULL [INFO ] [26-08-2023 02:00:56:624.539 GMT] writeHeaderCallback(): RequestId: fc30711a-3628-4345-a442-8dc2c3cd0012 [DEBUG] [26-08-2023 02:00:56:624.598 GMT] describeStreamCurlHandler(): DescribeStream API response: {"StreamInfo":{"CreationTime":1.692878742352E9,"DataRetentionInHours":8760,"DeviceName":null,"IngestionConfiguration":null,"KmsKeyId":"arn:aws:kms:ap-east-1:733421296020:alias/aws/kinesisvideo","MediaType":null,"Status":"ACTIVE","StreamARN":"arn:aws:kinesisvideo:ap-east-1:733421296020:stream/sxj731533730_thing/1692878742352","StreamName":"sxj731533730_thing","Version":"A2Dmc6X5EfKcIkfLSflW"}} [INFO ] [26-08-2023 02:00:56:625.247 GMT] describeStreamResultEvent(): Describe stream result event. [WARN ] [26-08-2023 02:00:56:625.275 GMT] describeStreamResult(): Retention period returned from the DescribeStream call doesn't match the one specified in the StreamInfo [WARN ] [26-08-2023 02:00:56:625.280 GMT] describeStreamResult(): Content type returned from the DescribeStream call doesn't match the one specified in the StreamInfo [INFO ] [26-08-2023 02:00:57:175.724 GMT] writeHeaderCallback(): RequestId: f17b41b4-016c-44b4-b574-a710a25e317d [DEBUG] [26-08-2023 02:00:57:175.776 GMT] getStreamingEndpointCurlHandler(): GetStreamingEndpoint API response: {"DataEndpoint":"https://s-24c98279.kinesisvideo.ap-east-1.amazonaws.com"} [INFO ] [26-08-2023 02:00:57:176.185 GMT] getStreamingEndpointResultEvent(): Get streaming endpoint result event. [INFO ] [26-08-2023 02:00:57:176.213 GMT] getStreamingTokenResultEvent(): Get streaming token result event. [DEBUG] [26-08-2023 02:00:57:176.228 GMT] streamReadyHandler invoked [DEBUG] [26-08-2023 02:00:57:176.332 GMT] Stream is ready [DEBUG] [26-08-2023 02:00:57:176.356 GMT] Attempt to upload file: /home/ubuntu/demo.mp4 [INFO ] [26-08-2023 02:00:57:186.379 GMT] Streaming from file source Error received from element demux: This file is invalid and cannot be played. Debugging information: qtdemux.c(775): gst_qtdemux_pull_atom (): /GstPipeline:file-kinesis-pipeline/GstQTDemux:demux: atom has bogus size 1229802312 [INFO ] [26-08-2023 02:00:57:191.967 GMT] Finished sending file to kvs producer: /home/ubuntu/demo.mp4 [INFO ] [26-08-2023 02:00:57:191.999 GMT] stopKinesisVideoStreamSync(): Synchronously stopping Kinesis Video Stream 0000555ddf7aa490. [DEBUG] [26-08-2023 02:00:57:192.022 GMT] streamClosedHandler invoked for upload handle: 18446744073709551615 [INFO ] [26-08-2023 02:00:57:192.038 GMT] All files have been persisted [INFO ] [26-08-2023 02:00:57:192.049 GMT] Freeing Kinesis Video Stream sxj731533730_thing [INFO ] [26-08-2023 02:00:57:192.065 GMT] freeKinesisVideoStream(): Freeing Kinesis Video stream. [DEBUG] [26-08-2023 02:00:57:192.074 GMT] curlApiCallbacksShutdownActiveRequests(): pActiveRequests hashtable is empty [INFO ] [26-08-2023 02:00:57:592.314 GMT] Freeing Kinesis Video Stream sxj731533730_thing [INFO ] [26-08-2023 02:00:57:592.383 GMT] freeKinesisVideoClient(): Freeing Kinesis Video Client [DEBUG] [26-08-2023 02:00:57:592.397 GMT] curlApiCallbacksShutdownActiveRequests(): pActiveRequests hashtable is empty [DEBUG] [26-08-2023 02:00:57:992.623 GMT] freeKinesisVideoClientInternal(): Total allocated memory 0 [WARN ] [26-08-2023 02:00:57:992.692 GMT] curlApiCallbacksShutdown(): curlApiCallbacksShutdown called when already in progress of shutting down

江流儿 @.***> 于2023年8月26日周六 09:51写道：

I fully refer to the operation of this video https://www.youtube.com/watch?v=nVxwX7Q9nPU&t=172s But I am running on a local physical machine

江流儿 @.***> 于2023年8月26日周六 09:49写道：

Hello, as you said, I have added environment variables and I have been able to successfully push local mkv videos, but I am still unable to push rtsp and mp4 videos

Jeremy Gunawan @.***> 于2023年8月24日周四 14:25写道：

Hi @sxj731533730 https://github.com/sxj731533730, please try the following gstreamer command, which includes the iot-thing-name key-value pair in the iot-certificate parameter.

gst-launch-1.0 -v videotestsrc is-live=true \ ! video/x-raw,framerate=30/1,width=720,height=480 \ ! clockoverlay time-format="%a %B %d, %Y %I:%M:%S %p" \ ! x264enc \ ! h264parse \ ! kvssink stream-name="YourStreamName" iot-certificate="iot-certificate,endpoint=xx.credentials.iot.us-west-2.amazonaws.com,cert-path=xxx-certificate.pem.crt,key-path=xxx-private.pem.key,ca-path=AmazonRootCA1.pem,role-aliases=YourRoleAlias,iot-thing-name=YourThingName"

If the iot-thing-name parameter isn't present, kvssink will use the value of the stream-name parameter as the x-amzn-iot-thingname header in the curl call here https://docs.aws.amazon.com/iot/latest/developerguide/authorizing-direct-aws.html, which may explain the 403 that you are encountering.

You can also manually make the CURL call yourself to confirm, and changing the value of the header.

curl --header "x-amzn-iot-thingname:YourThingName" \ --cert xxx-certificate.pem.crt \ --key xxx-private.pem.key \ --cacert AmazonRootCA1.pem \ https://xxx.credentials.iot.us-west-2.amazonaws.com/role-aliases/YourRoleAlias/credentials

— Reply to this email directly, view it on GitHub https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609#issuecomment-1691075144, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGKU6EPRW4TO4ADCAFMXRB3XW3XURANCNFSM4TO45VMQ . You are receiving this because you were mentioned.Message ID: <awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609/1691075144 @github.com>

clogwog commented 1 year ago

gst-launch-1.0 -e filesrc location="/path/to/mp4/file/with/h264encoding.mp4" ! qtdemux name="mux1" mux1.video_0 ! h264parse config-interval=-1 ! "video/x-h264,stream-format=avc,alignment=au" ! kvs1.video_0 kvssink streaming-type=offline file-start-time=[epochtimeofwheretostorethisbitofvideointhestream] name=kvs1 storage-size=512 iot-certificate="iot-certificate,endpoint=xx.credentials.iot.us-west-2.amazonaws.com,cert-path=xxx-certificate.pem.crt,key-path=xxx-private.pem.key,ca-path=AmazonRootCA1.pem,role-aliases=YourRoleAlias,iot-thing-name=YourThingName" stream-name="YourStreamName"

is working for me (i have a different way of authorisation though as i'm using greengrass to get a session kye, but the iot-certificate was working for us before as well)

pretty sure it doesn't make any difference, but i still have an idea in my head that the stream-name needs to be the last parameter to the kvssink (it makes no sense, but i thought i'd mention it)

sxj731533730 commented 1 year ago

I fully refer to the operation of this video https://www.youtube.com/watch?v=nVxwX7Q9nPU&t=172s But I am running on a local physical machine

clogwog @.***> 于2023年8月26日周六 11:06写道：

gst-launch-1.0 -e filesrc location="/path/to/mp4/file/with/h264encoding.mp4" ! qtdemux name="mux1" mux1.video_0 ! h264parse config-interval=-1 ! "video/x-h264,stream-format=avc,alignment=au" ! kvs1.video_0 kvssink streaming-type=offline file-start-time=[epochtimeofwheretostorethisbitofvideointhestream] name=kvs1 storage-size=512 iot-certificate="iot-certificate,endpoint= xx.credentials.iot.us-west-2.amazonaws.com,cert-path=xxx-certificate.pem.crt,key-path=xxx-private.pem.key,ca-path=AmazonRootCA1.pem,role-aliases=YourRoleAlias,iot-thing-name=YourThingName" stream-name="YourStreamName"

is working for me (i have a different way of authorisation though as i'm using greengrass to get a session kye, but the iot-certificate was working for us before as well)

pretty sure it doesn't make any difference, but i still have an idea in my head that the stream-name needs to be the last parameter to the kvssink (it makes no sense, but i thought i'd mention it)

— Reply to this email directly, view it on GitHub https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609#issuecomment-1694133996, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGKU6EPMW6H5XWQGU2TWNALXXFRZPANCNFSM4TO45VMQ . You are receiving this because you were mentioned.Message ID: <awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609/1694133996 @github.com>

sxj731533730 commented 1 year ago

what is it the file-start-time=[epochtimeofwheretostorethisbitofvideointhestream] ?

clogwog commented 1 year ago

It’s the epoch time of where you want to upload that particular file to in the stream. (This example is for an upload of a file in offline mode)

Checkout gst-describe kvssink For a description of all parameters.

On Sat, 26 Aug 2023 at 1:39 pm, sxj731533730 @.***> wrote:

what is it the file-start-time=[epochtimeofwheretostorethisbitofvideointhestream] ?

— Reply to this email directly, view it on GitHub https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609#issuecomment-1694143207, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAUAA5VZKTQD25BSBT2HLQLXXFVYDANCNFSM4TO45VMQ . You are receiving this because you commented.Message ID: <awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609/1694143207 @github.com>

sxj731533730 commented 1 year ago

hello, i solve the demo.mp4 push aws because of the demo.mp4 need to be setted framerate it is valid for me that i use the ffmpeg tools to set the mp4 framerate https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/1054

clogwog @.***> 于2023年8月26日周六 11:54写道：

It’s the epoch time of where you want to upload that particular file to in the stream. (This example is for an upload of a file in offline mode)

Checkout gst-describe kvssink For a description of all parameters.

On Sat, 26 Aug 2023 at 1:39 pm, sxj731533730 @.***> wrote:

what is it the file-start-time=[epochtimeofwheretostorethisbitofvideointhestream] ?

— Reply to this email directly, view it on GitHub < https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609#issuecomment-1694143207>,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAUAA5VZKTQD25BSBT2HLQLXXFVYDANCNFSM4TO45VMQ>

. You are receiving this because you commented.Message ID:

<awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609/1694143207

@github.com>

— Reply to this email directly, view it on GitHub https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609#issuecomment-1694146015, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGKU6EMOODDWYA2CWY3W5I3XXFXORANCNFSM4TO45VMQ . You are receiving this because you were mentioned.Message ID: <awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609/1694146015 @github.com>

sxj731533730 commented 1 year ago

but i still not push rtsp url, i am solving it are you give some suggestion ?

江流儿 @.***> 于2023年9月4日周一 17:13写道：

hello, i solve the demo.mp4 push aws because of the demo.mp4 need to be setted framerate it is valid for me that i use the ffmpeg tools to set the mp4 framerate https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/1054

clogwog @.***> 于2023年8月26日周六 11:54写道：

It’s the epoch time of where you want to upload that particular file to in the stream. (This example is for an upload of a file in offline mode)

Checkout gst-describe kvssink For a description of all parameters.

On Sat, 26 Aug 2023 at 1:39 pm, sxj731533730 @.***> wrote:

what is it the file-start-time=[epochtimeofwheretostorethisbitofvideointhestream] ?

— Reply to this email directly, view it on GitHub < https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609#issuecomment-1694143207>,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAUAA5VZKTQD25BSBT2HLQLXXFVYDANCNFSM4TO45VMQ>

. You are receiving this because you commented.Message ID:

<awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609/1694143207

@github.com>

— Reply to this email directly, view it on GitHub https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609#issuecomment-1694146015, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGKU6EMOODDWYA2CWY3W5I3XXFXORANCNFSM4TO45VMQ . You are receiving this because you were mentioned.Message ID: <awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/issues/609/1694146015 @github.com>