jayleicn / singularity

[ACL 2023] Official PyTorch code for Singularity model in "Revealing Single Frame Bias for Video-and-Language Learning"
https://arxiv.org/abs/2206.03428
MIT License
129 stars 13 forks source link

Corruption Issue with ssv2 videos #31

Closed Hritikbansal closed 1 year ago

Hritikbansal commented 1 year ago

Hi @jayleicn ,

I wanted to evaluate a video-text alignment model on the ssv2-template and ssv2-label dataset. I downloaded the videos from qualcomm website and extracted all the videos from their zip files.

Following this, i was running my code on the extracted webm videos and found that 1989 videos out of 2088 videos are corrupted. The code to load video is here:

from decord import VideoReader
import numpy as np
from PIL import Image

def get_index(num_frames, num_segments):
    seg_size = float(num_frames - 1) / num_segments
    start = int(seg_size / 2)
    offsets = np.array([
        start + int(np.round(seg_size * idx)) for idx in range(num_segments)
    ])
    return offsets

def load_video(path, num_frames=4):
    vr = VideoReader(path, height=224, width=224)
    total_frames = len(vr)
    frame_indices = get_index(total_frames, num_frames)
    images_group = list()
    for frame_index in frame_indices:
        img = Image.fromarray(vr[frame_index].asnumpy()).convert('RGB')
        images_group.append(img)
    return images_group

Have you seen this corruption issue before? If possible, can you share your 2088 val small videos for SSv2 with me on my email?

Hritikbansal commented 1 year ago

Okay, I found a way to fix these. Closing the issue!