Crash and imprecise fps report

[cryonox]

Video is here https://drive.google.com/file/d/1xhRuh7S_2n04BLXoSdPeYNO5DuAtUk9E/view?usp=sharing . The video might not have been "properly" created, it's done by merging two videos using ffmpeg concat demuxer without re-encode and probably is variable fps. But the library should not crash and gives same result as decord.

code:

import video_reader as vr
filename = 'police_protestor.mp4'
info_dict = vr.get_info(filename)
print(info_dict)
frames = vr.get_batch(filename,[0,3,4975 ],with_fallback=True)
print('ok!')
frames = vr.get_batch(filename,[4760,4808,4856],with_fallback=True) #crash
print('ok!')
frames = vr.get_batch(filename,[4760,4808,4856]) #crash
print('ok!')

Crash with

A few things I would like to complain in addition to the crash:

• info_dict is just returning strings. you should return relevant types, such as intger for 'width' and floating point for fps etc.
• fps returns 34. the fps reported by ffprobe (and also by decord) is 33.57669.... Its important for us to get this one right as it's used to calculate actual frame indices from time duration.
• it seems to me that by requiring the filename argument as part of get_batch function, there might be a non-neglible cost that would have to be paid for every batch call. our use case requires many such batch calls for one video( possibly up to ~500) that might become problematic. I havent really done any benchmarking but i think maybe a class structure or some sort of reusable struct(similar to decord) that can save initialization info might be better?
• if any crashes happen, it should be catchable via python try catch, otherwise it's problematic to say the least.

Decord code for the same video works.

from decord import VideoReader, cpu
import decord
decord.bridge.set_bridge("native")
filename = 'police_protestor.mp4'
vr = VideoReader(filename, ctx=cpu(0))
print(vr.get_avg_fps())
frames = vr.get_batch([0,3,4975]).asnumpy()
frames = vr.get_batch([4760,4808,4856]).asnumpy()

[gcanat]

Hello, I will have a look, thank you.

[gcanat]

I've pushed some changes to address the issue with fps. It should now report the correct average fps. For now you need to wrap the result in float(), ie float(info_dict["fps"]). I also fixed the issue with frames not being correctly retrieved. It should not crash anymore.

As for having a class structure similar to decord.VideoReader, I will see if I can work it out, but it is not trivial to implement. Will require more work.

[cryonox]

Great! It works now. Will update once I run into more problems while trying to move away from decord.

[gcanat]

Well, turns out it was not as hard as expected. So you can test with latest version if you want.

from video_reader import PyVideoReader

vr = PyVideoReader(filename)
indices = [12, 50, 32]
frames = vr.get_batch(indices)

[cryonox]

hmm...this new one changes the behaviour of previous function style. if it's not intended, maybe need more fixing? although the panic should definitely be fixed.

from video_reader import PyVideoReader

filename = 'samples/vos/police_protestor.mp4'
vr = PyVideoReader(filename)

info_dict = vr.get_info()
print(info_dict)
frames = vr.get_batch([0,3,4975 ],with_fallback=True)
frames = vr.get_batch([0,3,4975 ])
#frames = vr.decode_fast(start_frame=4760,end_frame=4975) # panic!
print('ok!')
frames = vr.get_batch([4760,4808,4856],with_fallback=True) 
print('ok!')
try:
    frames = vr.get_batch([4760,4808,4856])# error, but catchable
except:
    pass
print('ok!')

[gcanat]

Interesting. Will look. :+1:

[gcanat]

Should be fixed in v0.2.1

[cryonox]

fixed.