What can i do for get the newest frame from stream media.

[summm]

Really strange, the decoding speed is ten times faster than the other, but with this code, the final video displayed in front of me is 50-100ms later than the other, probably a 3-6 frame delay. How can I get the latest frame of the streaming media as quickly as possible?

video_path = "http://192.168.0.5/test.flv"

nvDec = nvc.PyNvDecoder(video_path, 0,)#{'max_delay': '1', 'bufsize': '1k'}
# nvCvt = nvc.PySurfaceResizer(1080, 720, nvDec.Format(), 0)
x_width, x_height = nvDec.Width(), nvDec.Height()
to_rgb = cconverter(x_width, x_height, 0)
to_rgb.add(nvc.PixelFormat.NV12, nvc.PixelFormat.YUV420)
to_rgb.add(nvc.PixelFormat.YUV420, nvc.PixelFormat.RGB)
#to_rgb.add(nvc.PixelFormat.RGB, nvc.PixelFormat.RGB_PLANAR)
nvDwn = nvc.PySurfaceDownloader(x_width, x_height, nvc.PixelFormat.RGB, 0)
while True:
    #t1 = time.time()
    src_surface = nvDec.DecodeSingleSurface()
    if src_surface.Empty():
        break
    #t2=time.time()
    #print(t2-t1)

    rgb_pln = to_rgb.run(src_surface)
    if rgb_pln.Empty():
        break

    frameRGB = np.ndarray(shape=(rgb_pln.HostSize()), dtype=np.uint8)
    success = nvDwn.DownloadSingleSurface(rgb_pln, frameRGB)

    #cv2.imwrite("{}.jpg".format(count), frameRGB.reshape((x_height, x_width, 3)))
    cv2.imshow("PyNvCodec",frameRGB.reshape((x_height, x_width, 3)))
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

[rnaskulwar]

you need flush the remaining frames from decoder, to get the latest frame or any frame for that matter, you could use Seek functionality from demuxer

[summm]

@rnaskulwar Thank you very much for your reply. I am testing streaming media for live streaming. So the decoding speed is always faster than obtaining new frame image data from the network. So it is very strange that when two programs are opened on the same computer at the same time, why is the video displayed after vpf decoding always slower than the other by tens of milliseconds. In this case, the seek function should not be used, right? I have always thought that there is a cache mechanism that causes such a delay of 3-6 frames. So I tested max_delay and bufsize, but it seems that this is irrelevant.

[rnaskulwar]

yes, due to B frames (inter frame dependency) there is latency of 2-3 frames, Could you elaborate on the two applications being run, If my understanding is correct, you are running VPF for streaming media and a video from disk. VPF decode for streaming is slower in your case?

[RomanArzumanyan]

Hi @summm

I recommend you to build VPF with NPP support and examine timeline like it's described here: VPF-Performance-analysis.

Also, take a closer look at the "Actual decoding latency" section.

[summm]

@rnaskulwar The media source for the test is live streaming media, such as links like https://www.testtest.test/test.flv. It is not reading files on the hard disk. At any time when I take a screenshot, they will have different degrees of time difference, but using PyNvCodec to decode is the slower one.The way they are displayed has been converted to rgb numpy arrays and displayed through opencv, which should be the same, but I measured the time for getframe part, VPF is much faster than the other, so I really feel very strange. Thank you @RomanArzumanyan, I am reading it, which is a bit difficult for beginners like me to understand, but I am trying my best to understand it.

[RomanArzumanyan]

@theHamsta @gedoensmax Hello guys. Please take a closer look at this PR: #332, it may be the case.