[BUG] `kvssink` occasionally generates non-monotonic PTS

[siddhantwahal]

Logging

Log snippet with error:

[DEBUG] [21-05-2024 09:26:41:811.151 GMT] [xxx-redacted-stream-name-xxx] pts: 17162836049677987, dts: 21840000, duration: 0, size: 6057, trackId: 1, isKey: 0
[DEBUG] [21-05-2024 09:26:41:811.608 GMT] [xxx-redacted-stream-name-xxx] pts: 17162836049988335, dts: 21860000, duration: 0, size: 5961, trackId: 1, isKey: 0 
[DEBUG] [21-05-2024 09:26:41:813.590 GMT] postReadCallback(): [xxx-redacted-stream-name-xxx] Wrote 41862 bytes to Kinesis Video. Upload stream handle: 0 
[DEBUG] [21-05-2024 09:26:41:829.753 GMT] [xxx-redacted-stream-name-xxx] pts: 17162836050298238, dts: 21880000, duration: 0, size: 5948, trackId: 1, isKey: 0
[DEBUG] [21-05-2024 09:26:41:830.221 GMT] [xxx-redacted-stream-name-xxx] pts: 17162836037652573, dts: 21900000, duration: 0, size: 5884, trackId: 1, isKey: 0
[WARN ] [21-05-2024 09:26:41:830.243 GMT] putKinesisVideoFrame(): [xxx-redacted-stream-name-xxx] Failed to submit frame to Kinesis Video client. status: 0x32000002 decoding timestamp: 21900000 presentation timestamp: 17162836037652573
[ERROR] [21-05-2024 09:26:41:830.254 GMT] Put frame for xxx-redacted-stream-name-xxx failed with 0x32000002                                                          

Detailed logs: kvs_pts.log original_pts.log

Describe the bug

Occasionally, the PTS generated by kvssink is not monotonic (see the log snippet above). This in turn leads to failures submitting frames to KVS. This failure does not occur at all when the the original PTS of the RTSP stream is used (by setting the use-original-pts option for kvssink to true).

To Reproduce

We're emulating IP cameras so our streaming setup is a little involved. We have two machines, machine1 and machine2. Both machines host RTSP servers on port 8554.

On machine1:

We're streaming an H265 video to the local RTSP server on machine1 via ffmepg:

ffmpeg -re -stream_loop -1  -i vid.mp4  -c copy -f rtsp -rtsp_transport tcp rtsp://localhost:8554/stream_1

We then relay this local RTSP stream to machine2:

ffmpeg -rtsp_transport tcp -i rtsp://localhost:8554/stream_1 -c copy -f rtsp -rtsp_transport tcp rtsp://machine2:8554/stream_2

Here is the ffrobe output of the H265 video:

  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42hvc1
    creation_time   : 2023-08-09T12:49:47.000000Z
  Duration: 01:00:07.00, start: 0.000000, bitrate: 3914 kb/s
  Stream #0:0(eng): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv), 2048x1536, 3868 kb/s, 30 fps, 30 tbr, 90k tbn, 30 tbc (default)
    Metadata:
      creation_time   : 2023-08-09T12:49:47.000000Z
      vendor_id       : [0][0][0][0]

It does not have any B-frames and the GOP is 1s.

On machine2:

The RTSP stream relayed from machine should be available at rtsp://localhost:8554/stream_1. We use the following GStreamer pipeline to upload it to KVS:

 gst-launch-1.0 -v rtspsrc location='rtsp://host.docker.internal:8554/stream_1' protocols=GST_RTSP_LOWER_TRANS_TCP short-header=TRUE ! rtpjitterbuffer ! rtph265depay ! video/x-h265,framerate=30/1 !  h265parse ! kvssink stream-name='xxx-redacted-stream-name-xxx' storage-size=512  aws-region='ap-northeast-1' access-key=$AWS_ACCESS_KEY_ID secret-key=$AWS_SECRET_ACCESS_KEY framerate=30 use-original-pts=false"

Expected behavior

kvssink should assign monotonically increasing PTS.

SDK version number

3.4.1

Platform (please complete the following information):

• OS: [e.g. Linux, Windows]: Docker ubuntu:22.04
• Processor: [e.g. ARMv5] Intel(R) Xeon(R) Platinum 8175M

[siddhantwahal]

I've just noticed that although using use-original-pts=true does not produce any 0x32000002 d errors, it generates a stream with an (incorrect) framerate of 500fps and a fragment duration of 0.1s.

@MushMal @disa6302 Do you have any pointers to help debug or mitigate this issue?