Negative Loss

[mmbejani]

I train the deep speech on Mozilla Persian Dataset (https://voice.mozilla.org/en/datasets). In the training procedure, always the value of the loss becomes negative then the parameters don't update. I read the document of the CTCLoss of pytorch that the output of the model should be logarithmized probabilities. Then, I add log_softmax function and the loss function becomes positive and this problem is solved, but another problem appears. The loss of the model stuck in a specific value, also the WRE and CRE don't change. What is the problem? (I can share my codes!)

[shivmgg]

Check if you are using the NVIDIA apex for scaling loss values correctly. Also, you can directly use ctc_loss branch if you are using Pytorch based CTC loss function.

[mmbejani]

I'll check it.

[mmbejani]

Thanks for responding. I checked the Pytorch document again and put the log_softmax as the activation function of the last layer in the training phase and it works well (in 200 epochs with batch_size 64, the model reaches to 20% WRE)