Stream mp4 file as if it was live (drop frames if necessary)

[Kajatin]

Hello @dusty-nv ,

When I stream the input from an mp4 file, it looks as if it was processing every frame from the video file, which makes the application look like it was running in slow motion. My pipeline can handle about 10 FPS, but when I stream from a 30 FPS mp4 file, it'll essentially run 1/3 of the speed of real time.

Is there a way to configure the gstreamer pipeline string to force it to play it back live and instead skip/drop frames that are not processed/queried in time?

Thanks! Roland

https://github.com/dusty-nv/jetson-utils/blob/0d3f59f5c0967a108ec1cd58a518ee5ad817d35b/codec/gstDecoder.cpp#L456

[dusty-nv]

Hi @Kajatin, gstDecoder and the ringbuffer code that it uses actually already does return the latest frame available and skip/drop frames that aren't processed in time. So my guess is that the entire system is slowing down as opposed to the video reader getting ahead.

What is the resolution of your mp4 file? You may want to try using a smaller resolution with the --input-width and --input-height arguments. This can significantly reduce the overhead (and the DNN models take in small resolution anyways)

[Kajatin]

It's interesting; I haven't thought of the whole system slowing down. It's just that looking at the frames it doesn't seem to be skipping any of them, that's why I thought that it was simply not dropping them when the consumer was slow.

I will investigate whether it's possible that the whole system is just slow.

The video file has a resolution of 2560x1920 @ 20FPS. I have a custom pipeline, so I'm only linking the jetson-utils library and use some of the functionality it provides, including setting up an RTSP stream.

[johnnynunez]

@dusty-nv is it possible to keep all frames with flag? I mean if I want process all frame due benchmarks, datasets etc

[dusty-nv]

videoSource interface uses a ringbuffer internally to limit memory usage - so just use jetson_utils.cudaMemcpy(img) for each img from capture() and save it as long as you want

---

From: Johnny @.> Sent: Monday, May 6, 2024 7:54:45 AM To: dusty-nv/jetson-utils @.> Cc: Dustin Franklin @.>; Mention @.> Subject: Re: [dusty-nv/jetson-utils] Stream mp4 file as if it was live (drop frames if necessary) (Issue #106)

@dusty-nvhttps://github.com/dusty-nv is it possible to keep all frames with flag? I mean if I want process all frame due benchmarks, datasets etc

— Reply to this email directly, view it on GitHubhttps://github.com/dusty-nv/jetson-utils/issues/106#issuecomment-2095837148, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADVEGK63ZJZJWJ6442CXQK3ZA5VQLAVCNFSM5MGR5IJ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBZGU4DGNZRGQ4A. You are receiving this because you were mentioned.Message ID: @.***>

[johnnynunez]

so, @dusty-nv if i doo a loop:

while True:
    if not self.paused:
        try:
            rgb_img = self.frame_capture.Capture()
            print("\n")
            print(f"Frame: {self.frame_capture.GetFrameCount()}\n")
            # here should I do:
            rgb_img_copy = jetson_utils.cudaMemcpy(rgb_img)

but it is possible that "while" go to next frame? I mean Capture is decoding on background, not stop if memory is full, dropping frames?

[dusty-nv]

IIRC it will keep going in background as fast as gstreamer publishes them. Jetson_utils is meant for live streaming, not seeking/pausing

---

From: Johnny @.> Sent: Monday, May 6, 2024 8:47:39 AM To: dusty-nv/jetson-utils @.> Cc: Dustin Franklin @.>; Mention @.> Subject: Re: [dusty-nv/jetson-utils] Stream mp4 file as if it was live (drop frames if necessary) (Issue #106)

so, @dusty-nvhttps://github.com/dusty-nv if i doo a loop:

while True: if not self.paused: try: rgb_img = self.frame_capture.Capture() print("\n") print(f"Frame: {self.frame_capture.GetFrameCount()}\n")

here should I do:

         jetson_utils.cudaMemcpy(img)

but it is possible that "while" go to next frame? I mean Capture is decoding on background, not stop if memory is full, dropping frames?

— Reply to this email directly, view it on GitHubhttps://github.com/dusty-nv/jetson-utils/issues/106#issuecomment-2095940282, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADVEGK7656MMT7GUVZBYQPLZA53WXAVCNFSM5MGR5IJ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBZGU4TIMBSHAZA. You are receiving this because you were mentioned.Message ID: @.***>

[johnnynunez]

IIRC it will keep going in background as fast as gstreamer publishes them. Jetson_utils is meant for live streaming, not seeking/pausing … ____ From: Johnny @.> Sent: Monday, May 6, 2024 8:47:39 AM To: dusty-nv/jetson-utils @.> Cc: Dustin Franklin @.>; Mention @.> Subject: Re: [dusty-nv/jetson-utils] Stream mp4 file as if it was live (drop frames if necessary) (Issue #106) so, @dusty-nvhttps://github.com/dusty-nv if i doo a loop: while True: if not self.paused: try: rgb_img = self.frame_capture.Capture() print("\n") print(f"Frame: {self.frame_capture.GetFrameCount()}\n") # here should I do: jetson_utils.cudaMemcpy(img) but it is possible that "while" go to next frame? I mean Capture is decoding on background, not stop if memory is full, dropping frames? — Reply to this email directly, view it on GitHub<#106 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADVEGK7656MMT7GUVZBYQPLZA53WXAVCNFSM5MGR5IJ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBZGU4TIMBSHAZA. You are receiving this because you were mentioned.Message ID: @.***>

thank you @dusty-nv !