search

v-iashin / video_features

Extract video features from raw videos using multiple GPUs. We support RAFT flow frames as well as S3D, I3D, R(2+1)D, VGGish, CLIP, and TIMM models.
https://v-iashin.github.io/video_features
MIT License
532 stars 97 forks source link

When the extraction_total parameter is set to 1, the problem of missing frames is encountered #86

Closed ChenyuLy closed 2 years ago

ChenyuLy commented 2 years ago

Hi,I encountered a problem when I tried to extract a global feature for whole video. I set extraction_total parameter to 1. But there is a problem When executing the ffmpeg instruction. Is there a way to extract a global feature without video slice? image

v-iashin commented 2 years ago

Hi, thanks a lot for making an issue.

While I am trying to figure out how we could fix it, try to use a higher value of extract total (eg 5) and then pick the middle item from the saved list

v-iashin commented 2 years ago

@Kamino666 what do you think? Should we warn users about using a single frame (assert).

this is of course an option but it seems like a nice feature to have. The problem again with that the cv2.

I will take a closer look later

v-iashin commented 2 years ago

@ChenyuLy By the way, you could try to average a few extracted features. I am not sure if it will work for your application but you can run an experiment comparing averaging of 5 features vs 1 middle feature. Luckily, you just need to extract features once.

v-iashin commented 2 years ago

Hm, @ChenyuLy, do you still get some features on those failed examples?

ChenyuLy commented 2 years ago

I do get some features and not all videos will encounter this problem. And ,by the way, It seems that this result is not caused by the extraction_total parameter being 1.

image

ChenyuLy commented 2 years ago

Hm, @ChenyuLy, do you still get some features on those failed examples?

I do get some features on those failed examples.

ChenyuLy commented 2 years ago

To be more specific, when I set the parameter to 1, I cannot get the failed examples feature. When I set the parameter to other values, I can get the failed examples feature.

v-iashin commented 2 years ago

Could you make a minimal (not) working example to reproduce the issue?

From what I understood, you get features on the failed examples and I think it is some sort of a middle frame. So, there is nothing to fix, is it?

Actually, this is not an error but a warning. Since the code does not fail with any error, I would just ignore it.

ChenyuLy commented 2 years ago

Yes, it is just a warning. I'm just a little worried about the effect of the feature .😂 Thanks for the reply and This is really a great repository

v-iashin commented 2 years ago

I'm just a little worried about the effect of the feature

As I see it: if a video has 250 frames and the first one is skipped that middle frame will be at around the same position anyway.