search

awslabs / amazon-kinesis-video-streams-producer-sdk-cpp

Amazon Kinesis Video Streams Producer SDK for C++ is for developers to install and customize for their connected camera and other devices to securely stream video, audio, and time-encoded data to Kinesis Video Streams.
Apache License 2.0
379 stars 334 forks source link

[QUESTION] raspberry pi bullseye - kvssink not streaming video #811

Closed EnricoFortunato closed 2 years ago

EnricoFortunato commented 2 years ago

Hello, I am trying to publish from raspberrypi (bullseye) to kinesis video stream using gstreamer plugin and libcamerasrc. The pipeline starts executing but it stops after few millisecond without providing errors. Any help will be greatly appreciated. Thanks.

Pipeline

gst-launch-1.0 libcamerasrc ! video/x-raw,format=I420,width=640,height=480,framerate=30/1! videoconvert ! x264enc bframes=0 key-int-max=30 bitrate=500 ! h264parse ! video/x-h264,stream-format=avc,alignment=au,profile=baseline ! kvssink stream-name="Hawkeye-video-stream" storage-size=512 access-key=$ACCESS_KEY secret-key=$SECRET_KEY aws-region="eu-west-1"

Logging

pi@raspberrypi:~/Documents/hawkeye/src/raspberrypi/example/kvs/amazon-kinesis-video-streams-producer-sdk-cpp $ gst-launch-1.0 libcamerasrc ! video/x-raw,format=I420,width=640,height=480,framerate=30/1! videoconvert ! x264enc bframes=0 key-int-max=30 bitrate=500 ! h264parse ! video/x-h264,stream-format=avc,alignment=au,profile=baseline ! kvssink stream-name="Hawkeye-video-stream" storage-size=512 access-key=$ACCESS_KEY secret-key=$SECRET_KEY aws-region="eu-west-1" Setting pipeline to PAUSED ... log4cplus:ERROR could not open file ./kvs_log_configuration INFO - createKinesisVideoClient(): Creating Kinesis Video Client 2022-01-23 11:47:50 [3069619968] INFO - heapInitialize(): Initializing native heap with limit size 536870912, spill ratio 0% and flags 0x00000001 2022-01-23 11:47:50 [3069619968] INFO - heapInitialize(): Creating AIV heap. 2022-01-23 11:47:50 [3069619968] INFO - heapInitialize(): Heap is initialized OK 2022-01-23 11:47:50 [3069619968] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000001, Next state: 0x0000000000000002 2022-01-23 11:47:50 [3069619968] DEBUG - getSecurityTokenHandler invoked 2022-01-23 11:47:50 [3069619968] DEBUG - Refreshing credentials. Force refreshing: 0 Now time is: 1642934870901478224 Expiration: 0 2022-01-23 11:47:50 [3069619968] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000002, Next state: 0x0000000000000010 2022-01-23 11:47:50 [3069619968] INFO - createDeviceResultEvent(): Create device result event. 2022-01-23 11:47:50 [3069619968] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000010, Next state: 0x0000000000000040 2022-01-23 11:47:50 [3069619968] DEBUG - clientReadyHandler invoked 2022-01-23 11:47:50 [3069619968] INFO - try creating stream 2022-01-23 11:47:50 [3069619968] INFO - Creating Kinesis Video Stream Hawkeye-video-stream 2022-01-23 11:47:50 [3069619968] INFO - createKinesisVideoStream(): Creating Kinesis Video Stream. 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Kinesis Video Stream Info 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Stream name: Hawkeye-video-stream 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Streaming type: STREAMING_TYPE_REALTIME 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Content type: video/h264 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Max latency (100ns): 600000000 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Fragment duration (100ns): 20000000 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Key frame fragmentation: Yes 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Use frame timecode: Yes 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Absolute frame timecode: Yes 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Nal adaptation flags: 0 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Average bandwith (bps): 4194304 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Framerate: 25 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Buffer duration (100ns): 1200000000 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Replay duration (100ns): 400000000 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Connection Staleness duration (100ns): 600000000 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Store Pressure Policy: 1 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): View Overflow Policy: 1 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Segment UUID: NULL 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Frame ordering mode: 0 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Track list 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Track id: 1 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Track name: kinesis_video 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Codec id: V_MPEG4/ISO/AVC 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Track type: TRACK_INFO_TYPE_VIDEO 2022-01-23 11:47:50 [3069619968] DEBUG - logStreamInfo(): Track cpd: NULL 2022-01-23 11:47:50 [3069619968] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000001, Next state: 0x0000000000000002 2022-01-23 11:47:50 [3069619968] INFO - writeHeaderCallback(): RequestId: cf9ff271-5229-442b-a3b1-9fb6d5afe20a 2022-01-23 11:47:51 [2484986944] DEBUG - describeStreamCurlHandler(): DescribeStream API response: {"StreamInfo":{"CreationTime":1.642195005898E9,"DataRetentionInHours":1,"DeviceName":null,"IngestionConfiguration":null,"KmsKeyId":"arn:aws:kms:eu-west-1:081972465094:alias/aws/kinesisvideo","MediaType":null,"Status":"ACTIVE","StreamARN":"arn:aws:kinesisvideo:eu-west-1:081972465094:stream/Hawkeye-video-stream/1642195005898","StreamName":"Hawkeye-video-stream","Version":"byhYhzsOaFJnHVfFYbdT"}} 2022-01-23 11:47:51 [2484986944] INFO - describeStreamResultEvent(): Describe stream result event. 2022-01-23 11:47:51 [2484986944] WARN - describeStreamResult(): Retention period returned from the DescribeStream call doesn't match the one specified in the StreamInfo 2022-01-23 11:47:51 [2484986944] WARN - describeStreamResult(): Content type returned from the DescribeStream call doesn't match the one specified in the StreamInfo 2022-01-23 11:47:51 [2484986944] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000002, Next state: 0x0000000000000020 2022-01-23 11:47:51 [2484986944] INFO - writeHeaderCallback(): RequestId: 981dd5e2-54e6-46df-a15f-c787df07e0eb 2022-01-23 11:47:51 [2474636352] DEBUG - getStreamingEndpointCurlHandler(): GetStreamingEndpoint API response: {"DataEndpoint":"https://s-5ac677c0.kinesisvideo.eu-west-1.amazonaws.com"} 2022-01-23 11:47:51 [2474636352] INFO - getStreamingEndpointResultEvent(): Get streaming endpoint result event. 2022-01-23 11:47:51 [2474636352] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000020, Next state: 0x0000000000000010 2022-01-23 11:47:51 [2474636352] DEBUG - getStreamingTokenHandler invoked 2022-01-23 11:47:51 [2474636352] DEBUG - Refreshing credentials. Force refreshing: 1 Now time is: 1642934871283500220 Expiration: 18446744073709551615 2022-01-23 11:47:51 [2474636352] INFO - getStreamingTokenResultEvent(): Get streaming token result event. 2022-01-23 11:47:51 [2474636352] DEBUG - stepStateMachine(): State Machine - Current state: 0x0000000000000010, Next state: 0x0000000000000040 2022-01-23 11:47:51 [2474636352] DEBUG - streamReadyHandler invoked 2022-01-23 11:47:51 [2474636352] Stream is ready [0:26:33.874317944] [8415] INFO Camera camera_manager.cpp:293 libcamera v0.0.0+3156-f4070274 Pipeline is live and does not need PREROLL ... Pipeline is PREROLLED ... Setting pipeline to PLAYING ... New clock: GstSystemClock Redistribute latency... [0:26:33.924067646] [8424] INFO Camera camera.cpp:937 configuring streams: (0) 640x480-YUV420 [0:26:33.924989227] [8421] INFO RPI raspberrypi.cpp:122 Mode: 3280x2464 fmt RG10 Score: 2662.49 (best 2662.49) [0:26:33.925112244] [8421] INFO RPI raspberrypi.cpp:122 Mode: 1920x1080 fmt RG10 Score: 2136.67 (best 2136.67) [0:26:33.925145780] [8421] INFO RPI raspberrypi.cpp:122 Mode: 1640x1232 fmt RG10 Score: 1944.49 (best 1944.49) [0:26:33.925177521] [8421] INFO RPI raspberrypi.cpp:122 Mode: 640x480 fmt RG10 Score: 1500 (best 1500) [0:26:33.925209576] [8421] INFO RPI raspberrypi.cpp:122 Mode: 3280x2464 fmt pRAA Score: 2162.49 (best 1500) [0:26:33.925282668] [8421] INFO RPI raspberrypi.cpp:122 Mode: 1920x1080 fmt pRAA Score: 1636.67 (best 1500) [0:26:33.925342334] [8421] INFO RPI raspberrypi.cpp:122 Mode: 1640x1232 fmt pRAA Score: 1444.49 (best 1444.49) [0:26:33.925404851] [8421] INFO RPI raspberrypi.cpp:122 Mode: 640x480 fmt pRAA Score: 1000 (best 1000) [0:26:33.925470906] [8421] INFO RPI raspberrypi.cpp:122 Mode: 3280x2464 fmt RGGB Score: 3662.49 (best 1000) [0:26:33.925530590] [8421] INFO RPI raspberrypi.cpp:122 Mode: 1920x1080 fmt RGGB Score: 3136.67 (best 1000) [0:26:33.925586756] [8421] INFO RPI raspberrypi.cpp:122 Mode: 1640x1232 fmt RGGB Score: 2944.49 (best 1000) [0:26:33.925643719] [8421] INFO RPI raspberrypi.cpp:122 Mode: 640x480 fmt RGGB Score: 2500 (best 1000) [0:26:33.925820809] [8421] INFO RPI raspberrypi.cpp:620 Sensor: /base/soc/i2c0mux/i2c@1/imx219@10 - Selected mode: 640x480-pRAA [0:26:33.937174998] [8421] INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0 [0:26:33.937247460] [8421] INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0 [0:26:33.937293941] [8421] INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0 [0:26:33.993320508] [8421] INFO RPI raspberrypi.cpp:1636 Dropping frame at the request of the IPA (6 left) [0:26:34.022872430] [8421] INFO RPI raspberrypi.cpp:1636 Dropping frame at the request of the IPA (5 left) [0:26:34.056644261] [8421] INFO RPI raspberrypi.cpp:1636 Dropping frame at the request of the IPA (4 left) [0:26:34.089641509] [8421] INFO RPI raspberrypi.cpp:1636 Dropping frame at the request of the IPA (3 left) [0:26:34.123052382] [8421] INFO RPI raspberrypi.cpp:1636 Dropping frame at the request of the IPA (2 left) [0:26:34.159430569] [8421] INFO RPI raspberrypi.cpp:1636 Dropping frame at the request of the IPA (1 left) [0:26:34.192809905] [8421] INFO RPI raspberrypi.cpp:1636 Dropping frame at the request of the IPA (0 left) 0:00:00.4 / 99:99:99.

OS

PRETTY_NAME="Raspbian GNU/Linux 11 (bullseye)"NAME="Raspbian GNU/Linux" VERSION_ID="11" VERSION="11 (bullseye)" VERSION_CODENAME=bullseye ID=raspbian

Additional Trials performed

I am successfully able to run the following pipelines and see the video: gst-launch-1.0 libcamerasrc ! video/x-raw,format=I420,width=640,height=480,framerate=30/1 ! videoconvert ! queue ! autovideosink (I can see the video on the raspberry pi) gst-launch-1.0 libcamerasrc ! video/x-raw,format=I420,width=640,height=480,framerate=30/1 ! x264enc ! h264parse ! udpsink host=192.168.0.23 port=8888 (I can see the streaming video on my windows pc)

niyatim23 commented 2 years ago

Hi, thank you for your question. Can you please answer the following questions for me?

  1. What version of the SDK are you using?
  2. Are you viewing/able to see in the frames in the aws-console?
  3. Does "Dropping frame at the request of the IPA" have custom callback functions to drop frames? What is it exactly doing?
  4. Can you please share VERBOSE logs for the SDK as well as GStreamer?
EnricoFortunato commented 2 years ago

Hi, thanks for the reply.

  1. I have recently cloned the repo following the instructions so I assume it's the latest (3.2.0)
  2. No I cannot see the frames in the console.
  3. I cannot find a proper description of this behavior. However this is happening also even when running the sample libcamera app or any other functioning pipeline with gstreamer launch.
  4. Here additional logs. logs.log.txt
niyatim23 commented 2 years ago

Hi @EnricoFortunato, I see that the media-type is being set to NULL in your logs (1:081972465094:alias/aws/kinesisvideo","MediaType":null,"Status":"ACTIVE","StreamARN":"arn:aws:kinesisvideo:eu-west-) Have you tried sources other than libcamerasrc? If your camera is USB-based then can you try the sample pipelines here?

disa6302 commented 2 years ago

Closing due to no response. Feel free to reach out if you have more questions.