Question on training loss

google / uis-rnn

This is the library for the Unbounded Interleaved-State Recurrent Neural Network (UIS-RNN) algorithm, corresponding to the paper Fully Supervised Speaker Diarization.

https://arxiv.org/abs/1810.04719

Apache License 2.0

1.55k stars 320 forks source link

Question on training loss #10

Closed 77281900000 closed 5 years ago

77281900000 commented 5 years ago

During the fit process,the loss has 3 parts.Could anyone tell me what's the meaning of loss2?And why counts the number of non-zero(the code is just above calculating loss2)?

AnzCol commented 5 years ago

Loss2 is the negative log probability of the prior p(\sigma2). If you choose to train \sigma2 this term will matter. The reason we count non-zeros is that since rnn_truth is a collection of different length sequences padded with zeros to fit them into a tensor. We are just counting the sum of 'real lengths' of all sequences.

wq2012 commented 5 years ago

Thanks Aonan for explaining. Closing this issue now. Please re-open it if you have more questions.