Fine-tuning pre-trained models on AVA dataset?

[TheKursta]

Hi!

I would like to fine-tune pre-trained model using AVA dataset format. How to achieve this using pytorchvideo? Current tutorial shows on how to run inference on already fine-tuned models.

Thank you!

[kalyanvasudev]

Hi @TheKursta, I will add an example for the same soon.

[TheKursta]

Hi! Is there any estimate when it could be available in documentation?

[JayLee15]

Hi @kalyanvasudev Is there an example for video detection training?

[lorepieri8]

Any news on this?

[dienhoa]

I succeed make transfer learning works on slowfast model. Please find in the snippet below:

import torch
model = torch.hub.load('facebookresearch/pytorchvideo', 'slowfast_r50', pretrained=True)
## Freeze model
for param in model.parameters():
    param.requires_grad = False
model.blocks[6] = create_res_basic_head(in_features=2304, out_features=101, pool=None)

Normally, it just loads the pretrained model, modify the head with your number of output features (you can freeze the body weights if you don't want to loose the pretrained weights)

[soumyadbanik]

I succeed make transfer learning works on slowfast model. Please find in the snippet below:

import torch
model = torch.hub.load('facebookresearch/pytorchvideo', 'slowfast_r50', pretrained=True)
## Freeze model
for param in model.parameters():
    param.requires_grad = False
model.blocks[6] = create_res_basic_head(in_features=2304, out_features=101, pool=None)

Normally, it just loads the pretrained model, modify the head with your number of output features (you can freeze the body weights if you don't want to loose the pretrained weights)

Did you train on the exact AVA formatted dataset? I mean how did you set out_features=101. Is that your custom data has 101 classes?

[dienhoa]

@soumyadbanik No I don't use the AVA dataset, I just want to share how I made transfer learning works. It's been a while but I used the Kinetics Dataset