search

piergiaj / pytorch-i3d

Apache License 2.0
971 stars 250 forks source link

Fail to finetune on ucf101 #31

Open ziqi-zhang opened 5 years ago

ziqi-zhang commented 5 years ago

Hi, I'm trying to finetune the pretrained model on ucf 101 but I only get 78% accuracy. I wonder have you tried similar settings?

piergiaj commented 5 years ago

I haven't tried running this code on ucf101, so I'm not sure what performance it should give. According to the paper, it should get to around 98% with two-stream finetuning. But the learning rate, batch size, and number of training iterations will all impact this, and I'm not sure what the right settings for ucf101 are.

ziqi-zhang commented 5 years ago

@piergiaj Thanks for replying! I'd like to double check training configuration with you. According to charades_dataset.py, the image is normalized to [-1,1] and the image size is 224x224, the clip length is 64. Is that right?

piergiaj commented 5 years ago

Yes, that is the correct setting for training. For evaluation, using 256x256 with all the frames is important for improved performance.

Sushant-aggarwal commented 5 years ago

Hi @ziqi-zhang , @piergiaj. I am trying to replicate the experiment of finetuning i3d trained on Kinetics, for ucf-101, as per the methods specified in the paper, but getting unsuccessful in achieving any significant accuracy (not even on training data) even after several iterations. Could you please give any guidance? To simplify, I just used 10 classes from ucf-101, extracted 64 contiguous frames from each video, from any (feasible) random begin point for each epoch, randomly spatially cropped 224x224 from video resized to 256 for each epoch, used Adam optimiser with initial learning rate(1e-4), and mean subtarction and division by 255 from data. I have a M2000 GPU. I am using only RGB pipeline.

is-cs commented 5 years ago

@piergiaj regarding your weights file rgb_imagenet.pt - is this i3d trained just on imagenet? or imagenet+kinetics? If it is the former, can you please let me know where can i find the weights for latter? Also, can you also please let me know what is the expected accuracy according to the i3d paper by using single stream i3d with your rgb_imagenet.pt weights only.

GayatriPurandharT commented 4 years ago

@ziqi-zhang could you finetune the model on ucf101 successfully? I want to know how the source of .json file for ucf101. Thank you.