How do I apply transfer learning to a pre-trained Vivit model?

[Fredbcx]

How do i train and test on a new dataset, a pre-trained model, after loading the model as it said in the demo notebook?

[mx-mark]

@Fredbcx it depends on your task. If you are handling a classification problem, maybe a liitle change on the current repo (build a dataset list to store the video path and its label) will finish it.

[Fredbcx]

@mx-mark Is there a way to get in touch with you? I need to ask you some questions urgently

[mx-mark]

@Fredbcx sorry for the slow reply, normally i do not log in the github during the weekdays. You can leave the message here in details, i will keep following your questions in the several comming days.

[Fredbcx]

@mx-mark Let's say I have a new dataset for binary classification, and have divided it in train and test. What do i need to load a Vivit model, already pre-trained, and then use it for my new dataset(train, test and evaluate)?

[mx-mark]

@Fredbcx If you just need the pretrained weights of vivit, you can simply build a finetune training script to your new dataset. And use the pretrain weights as the weight initiailization step to the vivit defination, or you can choose to moderately modify the notebook by replacing the data preprocessing to your own dataset loader along with a trainer to optimize the learning procedure.

[Anshu-Kumar-1]

@mx-mark Follow up question on above discussion, What is the procedure to create our own new dataset of video? Is there any specific input size required or any input size is fine?