search

danbider / lightning-pose

Accelerated pose estimation and tracking using semi-supervised convolutional networks.
MIT License
223 stars 32 forks source link

Fix for variable frame rate (vfr) videos #179

Closed hummuscience closed 5 days ago

hummuscience commented 5 days ago

I am recording videos with a Raspberry Pi camera. The camera has a “startup time” where the frame rate is high in the beginning and then stabilizes to the set frame rate.

This leads to files that don't have a technical “fixed” frame rate (of 30fps, for example) but to a slightly higher one (30.00000001 fps, for example).

Dali doesn't like variable frame rate (vfr) videos. Which makes sense, in cases where the frame rate is constantly fluctuating throughout the video.

This leads to the following error when analysing such videos:

Screenshot 2024-06-26 at 14 49 09

Turning off the vfr check allows the videos to still be analysed. The results look fine to me.

themattinthehatt commented 5 days ago

@hummuscience thanks for the PR! glad it was a simple fix. can you do me a favor and verify that everything passes when you run pytest from inside of your lightning-pose directory? we still don't have CI set up due to the need for GPUs (which is much more expensive than if we just needed to run unit tests on the CPU!)

hummuscience commented 5 days ago

All passed but with warnings. RTX A6000 with CUDA 12.1 (sorry for the screenshots, havent gotten tmux copying running yet :D)

Screenshot 2024-06-26 at 15 26 45 Screenshot 2024-06-26 at 15 26 29 Screenshot 2024-06-26 at 15 26 07

themattinthehatt commented 4 days ago

@hummuscience I wanted to check in on this again - I was looking at the code a second time and realized that this parameter never actually gets passed to anything inside of the video_pipe function. Can you verify that simply adding this as an argument to the pipeline function (with a default to true) does in fact change the behavior of the video reader with your videos?

hummuscience commented 4 days ago

Ah, something went wrong with my commit.

It needs to go in fn.readers.video instead (Initially had it in video pipe, but realized its not passed on and moved it there. And then staged the wrong fix >.<).

Ill make a new pull request. Sorry