Evaluate "contrastive loss"

[hominot]

• [ ] Decide on batch size.
  • K80 12GB gpu can run batch size of 32. Batch size of 48 gives OOM.
  • Try batch size of 64, 96, 128 etc on M40 24GB.
  • Let's use the same batch size for all experiments.
• [ ] Tune hyperparameters:
  • Learning rate: trainer.jinja2 learning_rate
  • Batch design: dataset.jinja2 group_size, num_groups (usually, num_groups = batch_size / group_size)
  • For tuning, perform a grid search. For each grid point, run one or two experiments and compare validation error to decide on the best hyperparameter setting.
  • Example configs: https://github.com/hominot/research/blob/master/metric_learning/experiments/contrastive_tuning.json
• [ ] Run cross validation on test dataset
  • Example script: python run.py --split=0/1/2/3/4
  • Examplt script: python evaluate.py --experiment={{ run_name }} --step={{ checkpoint_number }}
  • report average/stdev metrics

[hominot]

Quoting from https://arxiv.org/pdf/1706.07567.pdf

All models are trained using Adam [18] with a batch size of 200 for face verification, 80 for Stanford Online Products, and 128 for other experiments. The network architecture follows ResNet-50 (pre-activation) [13].

[hominot]

https://github.com/hominot/research/issues/35 should be resolved before proceeding with the experiments. I will work on the new cross validation split logic and perform some preliminary tests on it to make sure the overfitting issue is mitigated.