search

google-deepmind / kinetics-i3d

Convolutional neural network model for video classification trained on the Kinetics dataset.
Apache License 2.0
1.75k stars 462 forks source link

which one is the checkpoint? #116

Open tangyuelm opened 3 years ago

tangyuelm commented 3 years ago

Thanks for your codes and model. I want to download the i3d model pre-trained on the Kinetics dataset but feel confused about the checkpoint. I followed the path in evaluate_sample.py _CHECKPOINT_PATHS = { 'rgb': 'data/checkpoints/rgb_scratch/model.ckpt', 'rgb600': 'data/checkpoints/rgb_scratch_kin600/model.ckpt', 'flow': 'data/checkpoints/flow_scratch/model.ckpt', 'rgb_imagenet': 'data/checkpoints/rgb_imagenet/model.ckpt', 'flow_imagenet': 'data/checkpoints/flow_imagenet/model.ckpt', } I copied the checkpoints folder to a new path and tried to load it. However, is seems that no such file or directory: 'checkpoints/rgb_scratch_kin600/model.ckpt' I found 4 files under the folder rgb_scratch_kin600 but cannot find the model.ckpt. Could anyone help me? Besides, can the checkpoint be loaded to model writing in Pytorch?

tangyuelm commented 3 years ago

To my understand, the rgb_scratch_600 means the checkpoints for RGB frames pre-trained on Kinetics-600. What do the folder rgb_scratch and flow_scratch mean? Besides, in each folder, there are four files called "checkpoints", "model.ckpt.data-00000-of-00001", "model.ckpt.index", "model.ckpt.meta", but there is no file called model.ckpt. Which one should I use? I am not familiar with Tensorflow, so I am wondering how to load these checkpoints in Pytorch. Do you have the .pth version for the checkpoint pre-trained on Kinetics-400? I can only find the checkpoints pre-trained on ImageNet and Charades in your Pytorch codes from https://github.com/piergiaj/pytorch-i3d, but I believe Kinetics-400 is more commonly used for pre-training.

Tortoise17 commented 3 years ago

I am also looking for same answer. Or any way to convert them to .h5 format?

joaoluiscarreira commented 3 years ago

Sorry for the long delay. rgb_scratch means model trained from scratch on pixels (RGB values). From scratch as opposed, to from ImageNet-pretrained weights. Flow_scratch means models trained from scratch on optical flow.

I'm not familiar with Pytorch either... i'm sure there are Pytorch checkpoints on the web, though.

Best,

Joao

On Tue, Jul 6, 2021 at 2:06 PM tangyuelm @.***> wrote:

To my understand, the rgb_scratch_600 means the checkpoints for RGB frames pre-trained on Kinetics-600. What do the folder rgb_scratch and flow_scratch mean? Besides, in each folder, there are four files called "checkpoints", "model.ckpt.data-00000-of-00001", "model.ckpt.index", "model.ckpt.meta", but there is no file called model.ckpt. Which one should I use? I am not familiar with Tensorflow, so I am wondering how to load these checkpoints in Pytorch. Do you have the .pt version for the checkpoint pre-trained on Kinetics-400? I can only find the checkpoints pre-trained on ImageNet and Charades in your Pytorch codes from https://github.com/piergiaj/pytorch-i3d, but I believe Kinetics-400 is more commonly used for pre-training.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/deepmind/kinetics-i3d/issues/116#issuecomment-874742552, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADXKU2TZ3NYG2UKHIT74SBLTWL5T7ANCNFSM473GENAA .

joaoluiscarreira commented 3 years ago

Sorry, I'm not entirely sure about PyTorch. Do look online for Pytorch checkpoints, there should be some available.

Best,

Joao

On Tue, Sep 28, 2021 at 4:45 PM Tortoise17 @.***> wrote:

I am also looking for same answer. Or any way to convert them to .h5 format?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/deepmind/kinetics-i3d/issues/116#issuecomment-929351955, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADXKU2SXZ634CM4SGIPAPRDUEHPIPANCNFSM473GENAA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Tortoise17 commented 3 years ago

@joaoluiscarreira thank you so much. Is there any command line with which I can input the mp4 and get back the logits and features? or do I have to export flow and frames for the input in model to get these.