Closed AKASH2907 closed 3 years ago
For Q1 and Q5: Because tasks are different in pretraining (train_predict.py) and finetuing (ft_classfy.py). For finetuing, we should remove the head of pretext tasks (speed prediction and frame reconstruction) and remain the backbone of c3d / r21d / r3d to initialize the model. The base network means the backbone of c3d / r21d / r3d. The length of string "base_network." is 13.
For Q2: Yes, you can also use multi-gpus when training. Just use "--gpu 0,1,2,3,..."
For Q3,4: It seems so. We only use the the backbone ("base_network") of c3d / r21d / r3d to initialize the model for finetuing.
Is the above one true for finetuning, I mean if only we search for base_network in parameters name and then do the fine-tuning and strict =False. Or that would be an incorrect approach? (like checkpoint weights would not be loaded)
For finetuing, apart from remaining the backbone ("base_network") of c3d / r21d / r3d, we also should add the head ("fc layer") of action classification task. So considering the added fc layer, we should set the strict=False when using the weight of pretrained model to initialize the finetuned model. If you set strict=Ture, you can find only the parameters of the added fc layer ("linear.weight", "linear.bias") missing.
I printed the name of model parameter keys in ft_classfy.py. So, in that, if you use the only base_network, you can find the checkpoint keys whose weights are updated. But you modify the name (+14:) so that the new keys whose weights would be used for testing has strict =True so that code doesn't show errors for last linea layers: linear.weight, linear.bias?
I have no problems in code just have some doubts about weight loading
Thanks.