frame_number exceeding video length

[vbelagali]

Hi authors,

I am currently trying to set up the ego4d dataset for HOI-ref. But I find that some entries in the dataset have frame number higher than the length of the video. I have attached the code I am using to extract frames.

` def save_ref_frame(frame_path=None, ego4d_root="/data/"): video_id = frame_path.split("/")[-2] frame_str = frame_path.split("/")[-1].rstrip(".jpg") frame_number = int(framestr.split("")[-1]) video_path = f'{ego4d_root}/Ego4D/v2/clips/{video_id}.mp4' target_frame_path = f'{ego4d_root}/{frame_path}' frame_count = frame_number video = cv2.VideoCapture(video_path) if not video.isOpened(): print("Error: Could not open video.", video_path) exit() video.set(cv2.CAP_PROP_POS_FRAMES, frame_count) success, frame = video.read() print(f"video id: {video_id}, frame: {frame_str}") print(f"video len : {int(video.get(cv2.CAP_PROP_FRAME_COUNT))}") assert success == True parent_path = Path(target_frame_path).parent Path(parent_path).mkdir(parents=True, exist_ok=True) cv2.imwrite(target_frame_path, frame)

def process(meta_file="hoiqa_dataset/ego4d-train.json"): failed_metas = [] with open(meta_file, 'r') as file: metas = json.load(file) success = 0 for i, meta in tqdm(enumerate(metas[700000:]), total=len(metas)): save_ref_frame(meta["image_path"]) success = success + 1

if name == "main": process() `

I get the following error and the video len is less than the frame number

I used the ego4d cli tool as follows to download the videos. ego4d --output_directory="/data/Ego4D/" \ --datasets clips \ --version v2 \ --video_uid_file ego4d-clip-ids.txt \ --aws_profile_name ego4d

Can you please let me know if I am doing something wrong or can you please provide scripts to extract frames from ego4d videos ?

Thank you, Varun

[Sid2697]

Hello @vbelagali,

You'll have to use the videos from clip folder in Ego4D/v2 and account for the offset of the video by using this video['parent_start_frame']. Let me know if there is any confusion.

Regards, Siddhant Bansal

[vbelagali]

Hi Siddhant,

Thanks for the guidance. Can you please confirm if folder name is clip or clips ?

I can locate the parent_frame_start in manifest.csv file

Do I need to subtract the parent_frame_start from the frame_number to get the correct offset ?