The loss of uis-rnn model

[Naminwang]

When I trained the uis-rnn model, I found that the loss curve was abnormal. It shocked at -600 to -800 and never decline. I don't know how to solve it, can you give me some advice. And I used https://github.com/HarryVolek/PyTorch_Speaker_Verification this code to extract embeddings. Part of my loss records: Negative Log Likelihood: 62.7516 Sigma2 Prior: -702.1078 Regularization: 0.0011 Iter: 1250 Training Loss: -682.2758 Negative Log Likelihood: 67.6140 Sigma2 Prior: -749.8909 Regularization: 0.0011 Iter: 1260 Training Loss: -646.3773 Negative Log Likelihood: 75.9724 Sigma2 Prior: -722.3507 Regularization: 0.0011 Iter: 1270 Training Loss: -605.5015 Negative Log Likelihood: 80.3110 Sigma2 Prior: -685.8135 Regularization: 0.0011 Iter: 1280 Training Loss: -650.1578 Negative Log Likelihood: 93.3594 Sigma2 Prior: -743.5183 Regularization: 0.0011

[FlorentF9]

Your loss is increasing instead of decreasing... Have you used the trick to add a tiny value to the embeddings containing lots of zero values? And try to reduce the learning rate?

[Naminwang]

Your loss is increasing instead of decreasing... Have you used the trick to add a tiny value to the embeddings containing lots of zero values? And try to reduce the learning rate?

Sorry, I used E2E method to train diarization model, so I didn't use the trick.