More flexible fine-tuning and testing, more logs, zero-shot demo, minimal run examples

[GioFic95]

Fine-tuning

(FSC_finetune_cross.py)

1. You can enable (default) or disable data augmentation with the flags --do_aug and --no_do_aug;
2. You can resume a training with the flag --do_resume (also resume the W&B run by providing the --wandb_id);
3. More efficient computation of training metrics (MAE and RMSE);
4. Validation during training, with MAE, RMSE and NAE (Normalized Average Error, i.e., $\frac{MAE}{\text{gt count}}$, as in https://arxiv.org/abs/2308.10468);
5. Logging of train and val metrics on W&B, plus updated images of the predictions and exemplars used for each image (as you can see in this report https://api.wandb.ai/links/giovanni-ficarra/ilxvm4to and in the images below);
6. Model saving:
   1. save last model locally at each epoch, overwriting the previous one (finetuning_last);
   2. save locally AND on W&B every save_freq epochs or at the end of the training (finetuning_<epoch>);
   3. save locally AND on W&B if the validation MAE is the lowest (finetuning_minMAE);
7. More flexibility on the training dataset by using the data_path from the command line argument in FSC147.py as in FSC_finetune_cross.py.

Testing

(FSC_test_cross(few-shot).py)

1. More flexibility: you can test on the val split with the parameter --split;
2. Log NAE, loading time and inference time as well as MAE and RMSE;
3. Save better prediction images together with the exemplars used for each test image (particularly useful when using external exemplars), see examples here https://drive.google.com/file/d/134ac1x381PsaC3Y6RFXQWzpmlSzaoc3X/view?usp=sharing and in the images below;
4. Log and plot test results with util.misc.log_test_results and misc.plot_test_results.

Zero-shot demo

• demo_zero.py allows to test CounTR on your own data in an easy manner without providing any exemplar, just an input file or directory;
• For each input image, it will print the number of counted objects and the elapsed time;
• See examples of outputs here https://drive.google.com/file/d/134ac1x381PsaC3Y6RFXQWzpmlSzaoc3X/view?usp=sharing and in the image below;
• With util.misc.frames2vid, you can make a video from the output of the demo (particularly useful when dealing with video frames).

run_minimal.MD

In this file I summarised how to finetune and test CounTR with the new parameters, with examples of commands and input images and configurations.