How training samples are used for unsupervised pretraining and fine tuning the model

[hungryGeek16]

@gzerveas

• So I have gone through the paper, and it mentions that the dataset of a relevant domain is split into 80%-20% for training and validation respectively.
• The issue is I'm trying to pretrain the mvts model on FaceDetection dataset by imputation/unsupervised method and finetune it further. But I'm not understanding how much data should I use to finetune the model, and validate it, if I'm training on the train split.
• Even the FaceDetection dataset has train-test splits, no seperate split for finetuning.
• I believe the commands shown below are supposed to be used in the given order:

# Pretraining
python src/main.py --output_dir experiments --comment "pretraining through imputation" --name FaceDetection_pretrained --records_file Imputation_records.xls --data_dir dataset --data_class tsra --pattern TRAIN --val_ratio 0.2 --epochs 700 --lr 0.001 --optimizer RAdam --batch_size 128 --pos_encoding learnable --d_model 128 --dim_feedforward 256 --num_head 8 --num_layers 3

# Fine-tuning
!python src/main.py --output_dir experiments --comment "finetune for classification" --name finetuned --records_file Classification_records.xls --data_dir dataset --data_class tsra --load_model experiments/FaceDetection_pretrained_2023-11-06_19-06-24_MVw/checkpoints/model_last.pth --pattern TRAIN --val_pattern TEST --batch_size 128 --epochs 100 --pos_encoding learnable --d_model 128 --dim_feedforward 256 --num_head 8 --num_layers 3 --task classification --change_output --key_metric accuracy

• Let me know if that's correct. Thanks