test_demo_finetune.py arguments - difference between finetune_shot and n_shot

[ssinha89]

The demo script test_demo.sh includes function calls to test_demo_ani.py and test_demo_finetune.py. It seems the test_demo_finetune.py finetunes the pre-trained model on the test video frames, while the test_demo_ani.py simply evaluates without any fine-tuning. Could you explain the difference between the purpose of arguments --finetune_shot and --n_shot in test_demo_finetune.py? If I want to do say 32-shot fine-tuning what should my arguments be? Also for argument n_shot = 8 should the arguments --ref_img_id be always 0 or it can be a list of 8 images?

[ssinha89]

The demo script test_demo.sh includes function calls to test_demo_ani.py and test_demo_finetune.py. It seems the test_demo_finetune.py finetunes the pre-trained model on the test video frames, while the test_demo_ani.py simply evaluates without any fine-tuning. Could you explain the difference between the purpose of arguments --finetune_shot and --n_shot in test_demo_finetune.py? If I want to do say 32-shot fine-tuning what should my arguments be? Also for argument n_shot = 8 should the arguments --ref_img_id be always 0 or it can be a list of 8 images?

I tried to run test_demo_finetune.py by setting both --finetune_shot and --n_shot to 16, the generated output on the sample data appears like this:

[rainingDesert]

The demo script test_demo.sh includes function calls to test_demo_ani.py and test_demo_finetune.py. It seems the test_demo_finetune.py finetunes the pre-trained model on the test video frames, while the test_demo_ani.py simply evaluates without any fine-tuning. Could you explain the difference between the purpose of arguments --finetune_shot and --n_shot in test_demo_finetune.py? If I want to do say 32-shot fine-tuning what should my arguments be? Also for argument n_shot = 8 should the arguments --ref_img_id be always 0 or it can be a list of 8 images?

I tried to run test_demo_finetune.py by setting both --finetune_shot and --n_shot to 16, the generated output on the sample data appears like this:

--n_shot refers to the number of reference images used during training. Normally you don't need to set --finetune_shot and the number of reference images will be the same as --n_shot during finetune. (If you want to change number of reference images, you can uncomment line 120-125 of test_demo_finetune.py and set value of --finetune_shot.)

As our model requires number of reference images to be the same during training and testing, please set value of --n_shot same as that of training, which is 8. If number of reference images in --ref_img_id is less than value of --n_shot, we will reuse specific images. If you want to use more images as reference during evaluation, you need to retrain the model.