NVIDIA / VideoProcessingFramework

Set of Python bindings to C++ libraries which provides full HW acceleration for video decoding, encoding and GPU-accelerated color space and pixel format conversions
Apache License 2.0
1.32k stars 233 forks source link

Seeking by timestamp is limited to integer number of seconds. #490

Closed sotelo closed 1 year ago

sotelo commented 1 year ago

This is related to: https://github.com/NVIDIA/VideoProcessingFramework/issues/232

Describe the bug nvc.SeekContext expects an integer for the seek_frame parameter. However, when we want to search by timestamp (because of a variable frame rate), we are limited to seeking for the frames at the integer second positions and can't seek for the frame at intermediate locations (i.e. 1.5 seconds).

To Reproduce Steps to reproduce the behavior:

seek_criteria = nvc.SeekCriteria.BY_TIMESTAMP

# This works:
seek_ctx = nvc.SeekContext(seek_frame=1, seek_criteria=seek_criteria)

# This doesn't work:
seek_ctx = nvc.SeekContext(seek_frame=1.5, seek_criteria=seek_criteria)

nvDec.DecodeSingleSurface(seek_ctx)
...

Expected behavior We should be able to specify floats as the seek location when seeking by timestamp.

sotelo commented 1 year ago

Maybe a simpler to implement intermediate solution would be to change the interpretation from seconds to milliseconds? That way you could keep the variable as an integer while giving significantly more flexibility. What do you think @rarzumanyan?

RomanArzumanyan commented 1 year ago

Hi @sotelo

It's a VPF bug. Thanks for bringing this up! Actual seek function treats seek_frame variable as double despite it's uint64_t: https://github.com/NVIDIA/VideoProcessingFramework/blob/d8d5d1874c65ecfe6a82db2c282182e1b865452e/src/TC/src/FFmpegDemuxer.cpp#L295-L299

https://github.com/NVIDIA/VideoProcessingFramework/blob/d8d5d1874c65ecfe6a82db2c282182e1b865452e/src/TC/src/FFmpegDemuxer.cpp#L240-L251

SeekContext structure needs to be extended with double timestamp.

RomanArzumanyan commented 1 year ago

@sotelo I've created a PR which shall fix this issue, could you please test it on your side? #497

sotelo commented 1 year ago

@RomanArzumanyan not sure why I wasn't notified of your comment. Will check it out and report back! Thanks a lot for looking into this.