Open H-Dempsey opened 1 year ago
Hi @H-Dempsey, thank you so much for contributing! I just today had to deal with a video that was very slow to read, so I was excited to try your PR. Unfortunately, on that 100MB video the difference is barely visible, and after reading the code behind napari-video
I realized that the implementation is pretty similar to what we had tried earlier on here.
It was faster, but cv2.VideoCapture().read() is not index-safe..., so on long videos, we'd often see a mismatch between the video frame and the corresponding annotations 😕 Is that something you observed too?
Hi @jeylau,
Thank you for your response!
Unfortunately, on that 100MB video the difference is barely visible
I can show that it works on my end with larger videos. The previous video I used was 25 MB, and here is another test with a 6.7 GB video (1920x1080, 49 mins, 30 fps).
Before:
After:
cv2.VideoCapture().read() is not index-safe..., so on long videos, we'd often see a mismatch between the video frame and the corresponding annotations 😕 Is that something you observed too?
I just played around with an analysed 12-hour (20 fps) video, and I haven't observed it so far. Could you explain why cap.read() is not index-safe, and could lead to a mismatch between the frame and annotations in long videos? Could that be due to cap.set(cv2.CAP_PROP_POS_FRAMES, frame_no) rather than cap.read()? I ran into this thread. Setting the frame number using this seems to be unreliable sometimes (e.g. when the frame rate is variable).
While my pull request seems to improve speed, if it leads to inaccurate frame setting, I think we should not merge it.
Harry
Hi again,
I spent some more time looking into the issue of CAP_PROP_POS_FRAMES
being inaccurate sometimes.
The people from this other project ran into the same issue as us.
They solved it by using a video player called decord. It splits up the video seeking into fast and accurate versions. By default, the video player uses accurate seek and it is also faster than the OpenCV and PyAV versions.
Hi napari-deeplabcut!
I am a big fan of both napari and deeplabcut, and I was so excited when I first found out about this collaboration. This is my first pull request and I apologise if I make some mistakes.
I noticed that when I scroll through a video during manual frame extraction, the interface lags.
https://github.com/DeepLabCut/napari-deeplabcut/assets/101311642/117bcceb-b611-44bd-8e79-bd7fab3b3b95
If I replace the current video reader class + dask lazy loads with the napari-video reader class, the scrolling is much smoother.
https://github.com/DeepLabCut/napari-deeplabcut/assets/101311642/cf3864fb-2027-44bb-b40c-e97a24d40882
For the non-opencv option, I tried to make my own version of the napari-video class and replace all cv2 functions with imageio functions. But unfortunately, the speed looked similar, so I decided to keep the PyAV and dask lazy loading for that. I have included it below for interest.
Thank you for making this really useful tool.
Harry