can't find youcook_val.pkl

[EmreOzkose]

Hi,

I am trying to reproduce MMPT (videoclip). However, I can't find how to download 'data/youcook/youcook_val.pkl'. Can you provide a link to reach that data?

[EmreOzkose]

It is in here.

[DoubleSsh]

Hi. I have the same problem.I found 'data/youcook/youcook_val.pkl' in https://github.com/antoine77340/howto100m#downloading-meta-data-for-evaluation-csv-pre-extracted-features-for-evaluation-word2vec. But When I enter ‘python locallaunch.py projects/retri/videoclip/test_youcook_zs.yaml --jobtype local_predict # zero-shot evaluation’ I'm still missing 'data/feat/feat_youcook_s3d'.Do you know where I can find it? @EmreOzkose

[EmreOzkose]

@DoubleSsh You should extract them by yourself.

cd here

python scripts/video_feature_extractor/extract.py --vdir /mnt/mot/datasets/youcook2/raw_videos/training \
                                                  --fdir data/feat/feat_youcook_s3d_training \
                                                  --type=s3d \
                                                  --num_decoding_thread=0 \
                                                  --batch_size 8 \
                                                  --half_precision 1

python scripts/video_feature_extractor/extract.py --vdir /mnt/mot/datasets/youcook2/raw_videos/validation \
                                                  --fdir data/feat/feat_youcook_s3d_validation \
                                                  --type=s3d \
                                                  --num_decoding_thread=0 \
                                                  --batch_size 8 \
                                                  --half_precision 1

mkdir data/feat/feat_youcook_s3d/
cp data/feat/feat_youcook_s3d_validation/*.npy data/feat/feat_youcook_s3d_training/*.npy data/feat/feat_youcook_s3d/

This commands are also written in here. As far as I remember, validation and training took less than 12 hours on RTX2060S 8GB. I think you can combine 2 commands as

python scripts/video_feature_extractor/extract.py --vdir /mnt/mot/datasets/youcook2/raw_videos \
                                                  --fdir data/feat/feat_youcook_s3d \
                                                  --type=s3d \
                                                  --num_decoding_thread=0 \
                                                  --batch_size 8 \
                                                  --half_precision 1

[DoubleSsh]

Hi. Thank you for your help. If I want to reproduce the results on the YouCook2 dataset in the paper, I think I need the video files of the dataset, but I couldn't find them in the official download link. Do you know where I can find them? @EmreOzkose

[EmreOzkose]

You should download raw videos via this official script and these official splits*.

Official script uses 'youtube-dl', but I strongly recommend to use yt-dlp since 'youtube-dl' downloads very slow.

*http://youcook2.eecs.umich.edu/download

[tiesanguaixia]

Hi. Thank you for your help. If I want to reproduce the results on the YouCook2 dataset in the paper, I think I need the video files of the dataset, but I couldn't find them in the official download link. Do you know where I can find them? @EmreOzkose

Hi! Have you finished the download of YouCook2 dataset? I am facing the same problem with you. And may I ask do you know how to split the YouCook2 video in to several clips as I found that each long video consists of some short clips and each clip has only 1 caption.

[tiesanguaixia]

You should download raw videos via this official script and these official splits*.

Official script uses 'youtube-dl', but I strongly recommend to use yt-dlp since 'youtube-dl' downloads very slow.

*http://youcook2.eecs.umich.edu/download

Hi! Thank you for your guidance! May I ask do you know how to split the YouCook2 video in to several clips as I found that each long video consists of some short clips and each clip has only 1 caption.

[EmreOzkose]

Hi @tiesanguaixia , sorry for late reply, you can use ffmpeg:

import subprocess

def cut_video(video_path, start_time, end_time, save_path):
    assert video_path != save_path, "video_path and save_path must be different"
    assert start_time < end_time, "start_time must be smaller than end_time"
    assert Path(video_path).exists(), "video_path does not exist"

    """command = ['ffmpeg',
        '-y',
        '-i', '"%s"' % video_path,
        '-ss', str(start_time),
        '-t', str(end_time - start_time),
        '-c:v', 'libx264', '-c:a', 'copy',
        '-threads', '1',
        '-loglevel', 'panic',
        '"%s"' % save_path]
    command = ' '.join(command)"""
    command = f"ffmpeg -i {video_path} -vcodec copy -acodec copy -ss {seconds_to_hms(start_time)} -t {seconds_to_hms(end_time-start_time)} -loglevel panic {save_path}"

    # status = subprocess.check_output(command, shell=True, stderr=subprocess.STDOUT)
    status = subprocess.run(command.split(), capture_output=True)
    print(status)