Advice on which loss to optimize

[sapan]

First of all, great blog and an amazing piece of code. The question is related to the loss that one should optimize for.

• In the code, the loss that is being minimized is : sum(triplet_loss) / #valid triplets However, this loss value many times stays to the margin value due to model collapsing. Other times, it keeps on fluctuating (you have mentioned this here, https://github.com/omoindrot/tensorflow-triplet-loss/issues/15#issuecomment-427763507 that An additional note: the loss will not decrease during training because it's the average over the positive triplets. So the loss will just oscillate during training. However the fraction of positive triplets should decrease.

So, it looks like it is difficult to assume that the loss as it is minimized currently would gradually decrease. In fact, I observed that after a few epochs, when only tough triplets are present in the valid set, the loss increases a bit. On the other hand, the fraction of valid triplets keeps decreasing (at least in my application).

So, how can we decide when (after how many epochs) to stop training? For instance, If I want to have an early stopping criteria, shall that be on the loss value or the fraction of valid triples?

[omoindrot]

Monitoring the fraction of valid triplets is more interesting because it will decrease during training as you say.

But the best metric to monitor (and base early stopping on) really depends on your application. If you are doing face recognition, then maybe the metric you want to monitor should be the ROC-AUC on a separate test set or something adapted.

[sapan]

Thanks for the suggestions, closing this.