how to transfer/fine tune on my dataset?

[dmumtaz]

Hi I want to:

1. Retrain your model for my video dataset
2. finetune/transfer learn on my dataset

Can you please Provide the minimum list and directory structure of the files/weights/dataset that I need to download for the above mentioned tasks?

You have provided the l" python3 split_train.py " for fine tuning. However I am facing the following error when I run it:

{'name': 'DiViDe-MRSSSL-DivideHead-NOUP', 'num_epochs': 30, 'l_num_epochs': 0, 'warmup_epochs': 2.5, 'ema': True, 'save_model': True, 'batch_size': 16, 'num_workers': 6, 'need_upsampled': False, 'need_feat': True, 'need_fused': False, 'wandb': {'project_name': 'VQA_Experiments_2022'}, 'data': {'train': {'type': 'FusionDataset', 'args': {'phase': 'train', 'random_crop': False, 'anno_file': './examplar_data_labels/train_labels.txt', 'data_prefix': '../datasets/LSVQ', 'sample_types': {'fragments': {'fragments_h': 8, 'fragments_w': 8, 'fsize_h': 16, 'fsize_w': 16, 'aligned': 32}, 'resize': {'size_h': 128, 'size_w': 128}}, 'clip_len': 32, 'frame_interval': 2, 'num_clips': 1}}, 'val': {'type': 'FusionDataset', 'args': {'phase': 'test', 'anno_file': './examplar_data_labels/LIVE_VQC/labels.txt', 'data_prefix': '../datasets/LIVE_VQC', 'sample_types': {'fragments': {'fragments_h': 8, 'fragments_w': 8, 'fsize_h': 16, 'fsize_w': 16, 'aligned': 32}, 'resize': {'size_h': 128, 'size_w': 128}}, 'clip_len': 32, 'frame_interval': 2, 'num_clips': 4}}}, 'model': {'type': 'DiViDeAddEvaluator', 'args': {'divide_head': True, 'vqa_head': {'in_channels': 768, 'hidden_channels': 64}}}, 'optimizer': {'lr': 0.001, 'backbone_lr_mult': 0.1, 'wd': 0.05}, 'load_path': '../model_baselines/NetArch/swin_tiny_patch244_window877_kinetics400_1k.pth', 'test_load_path': './pretrained_weights/DiViDe-MRSSSL-DivideHead-BiLearn_s_dev_v0.0.pth'}
Traceback (most recent call last):
  File "split_train.py", line 627, in <module>
    main()
  File "split_train.py", line 365, in main
    print(opt["split_seed"])
KeyError: 'split_seed'

[teowu]

Hi dmumtaz:

The split_train.py is correct for fine-tuning, but you need to switch to correct .yml files. Please refer to the options/finetune/fast/konvid.yml to see the template, and modify the path / label files according to your custom dataset.

[dmumtaz]

Thank you for the response, got it!