search

awslabs / amazon-kinesis-video-streams-producer-sdk-cpp

Amazon Kinesis Video Streams Producer SDK for C++ is for developers to install and customize for their connected camera and other devices to securely stream video, audio, and time-encoded data to Kinesis Video Streams.
Apache License 2.0
378 stars 336 forks source link

[BUG] playback jitter in stream ingested using kvssink from RSTP / h264 stream from Xprotect #816

Closed mleopold closed 2 years ago

mleopold commented 2 years ago

I'm trying to ingest a live camera stream originating from a stream in the VMS solution Xprotect, but when playing back the stream from KVS it shows some form of "jitter". I have used the kvssink Gstreamer plugin from the C++ examples. I have reproduced the problem by exporting the clip from the VMS and ingested it using the kvs_gstreamer_file_uploader_sample. The clip originates from a 'fake' camera running a stored video in a loop (known as StableFPS in Xprotect). I tried creating a new KVS stream with no change.

Visually it appears to me that the video "jumps" back every 0.5 s or so - I've attached an exported clip as well as a screen recording showing the issue. I'm using Google Chrome on Ubuntu and issue persists when using the KVS console, the 'amazon-kinesis-video-streams-media-viewer' example and VLC. The videostream is exposed from Xprotect using an add-on called Onvif Bridge as RTSP / h264 / 555x333 resolution. Xprotect is running on a Windows EC2 instance and is fetched between by a Ubuntu EC2 instance in the same subnet. The command line used for the kvssink Gstreamer example: gst-launch-1.0 rtspsrc protocols=tcp location=rtsp://XXX:YYY@10.0.0.123:554/live/03598800-0dd1-XXXX-YYYY-67ffe43a2b8a ! rtph264depay ! h264parse ! kvssink stream-name="MLTest" storage-size=512 aws-region="eu-central-1"

Exporting a clip from the VMS and uploading using kvs_gstreamer_file_uploader_sample reproduces the issue (with a new stream). ./kvs_gstreamer_file_uploader_sample MLTest2 ~/MLTest.mp4 `date +%s`

Transcoding the video (reducing height/width 1 pixel) seems to make the problem go away gst-launch-1.0 rtspsrc protocols=tcp do-timestamp=TRUE do-rtcp=TRUE short-header=TRUE onvif-mode=TRUE location=rtsp://XXX:YYYY@10.0.0.123:554/live/03598800-0dd1-XXXX-YYYY-67ffe43a2b8a ! queue ! decodebin ! videoscale ! 'video/x-raw,width=554,height=332' ! x264enc ! kvssink stream-name="MLTest3" storage-size=512 aws-region="eu-central-1"

To Reproduce

  1. Install Xprotect (available from Marketplace) and upgrade license to allow use of Onvif bridge
  2. Install and configure the Onvif bridge with username/password
  3. Export RTSP stream URL
  4. Clone and build KVS GStreamer plugin
  5. Create KVS stream
  6. Ingest using GST pipeline as shown above
  7. View video in online-viewer

SDK version number

I cloned the repo November 29 b15283d0ad01173abe2cfe8adf4c1eb31d5c9477

Screenshots

I have captured a screen-cast of the console player side-by-side with the example player from the GitHub project showing the same video. Also I have included a clip of the original video from Xprotect (the one used with the file_uploader).

Screencast 2021-12-10.mp4 https://milestonesys365-my.sharepoint.com/:v:/g/personal/martin_leopold_milestone_dk/EbQTGQ-0xH9Ahk83zq-KrrIBXxKZCuGKqkgmB92Krjri3Q?e=La5qgE

XprotectExporth.mp4 https://milestonesys365-my.sharepoint.com/:v:/g/personal/martin_leopold_milestone_dk/EU2WfeCxillJq-bhWySvL5QBnvJNvytdrg1xCMtVbHA9cg?e=eximor

Logs I have included logs-snippets from the two examples above: one with and one without transcoding gst-onvif-snippet.txt gst-onvif-transcode.txt

unicornss commented 2 years ago

Hi @mleopold , In real-time streaming ingestion, times codes or B frames could cause such stuttering. Given that re-encoding (reducing width/height) fixes the problem, the frame time codes and playback order are to be looked at the source. KVS does not process the media frames so, this needs to be adjusted in the stream source; One possibility is to configure the RTSP stream settings to disable B-frames and see if that helps. Thanks.

mleopold commented 2 years ago

Hi, What i was using was an encoded stream from a camera presented as a camera (using what is called the StableFPS driver in Xprotect), not an live-feed, so I imagine that encoding has been done with B-frames. I was unable adjust the time-codes in the source (it simply relays the pre-encoded video), but switching to a live feed from a camera (presumably without B-frames) shows no stuttering.

I'm not sure how the time-codes/B-frames in the first video are encoded causing the issue - they seem to play alright in VLC, but my case will use live-feeds and it seems this won't be an issue.

So you can close the bug - thanks for your help.