It seems that there is a lack of shuffle the batch when training. Thus the loss will decrease very fast, and it learns nothing. It only learns the fixed similarity matrix output. After training, the model cannot work.
Each batch of data should be permuted and then unpermuted. There is a example of pytorch version.
Pytorch_Speaker_Verification:
It seems that there is a lack of shuffle the batch when training. Thus the loss will decrease very fast, and it learns nothing. It only learns the fixed similarity matrix output. After training, the model cannot work. Each batch of data should be permuted and then unpermuted. There is a example of pytorch version. Pytorch_Speaker_Verification: