search

google / uis-rnn

This is the library for the Unbounded Interleaved-State Recurrent Neural Network (UIS-RNN) algorithm, corresponding to the paper Fully Supervised Speaker Diarization.
https://arxiv.org/abs/1810.04719
Apache License 2.0
1.56k stars 319 forks source link

Predicted labels doesn't match with Ground truth labels but the accuracy of test results is 0.8% #90

Closed SanaullahOfficial closed 2 years ago

SanaullahOfficial commented 2 years ago

I am just curious about the flow of "Ground truth labels" as that's not matched at all with the "Predicted labels:" but the accuracy of test results is around 0.8%. Any idea or possible suggestion to look at the UIS RNN api. 

Predicted labels:
[0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 2, 3, 3, 3, 3, 3, 3, 4, 4, 3, 3, 3, 3, 3, 3, 2, 4, 4, 4, 4, 2, 2, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 7, 7, 8, 8, 9, 9, 9, 9, 9, 10, 10, 10, 11, 11, 9, 9, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 11, 11, 9, 12, 13, 13, 13, 14, 14, 13, 13, 15, 15, 15, 15, 15, 15, 15, 15, 16, 16, 13, 13, 13, 13, 13, 13, 15, 15, 15, 17, 17, 17, 17, 17, 18, 18, 17, 18, 18, 13, 13, 19, 19, 13, 13, 13, 13, 13, 13, 19, 19, 19, 19, 19, 13, 13, 13, 13, 13, 20, 20, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 20, 20, 20, 20, 22, 22, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 24, 24, 24, 24, 24, 24, 25, 25, 25, 25, 25, 25, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 6, 26, 26, 18, 18, 27, 27, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 27, 27, 27, 19, 19, 19, 13, 29, 29, 29, 13, 13, 30, 30, 30, 30, 30, 19, 13, 13, 31, 31, 7, 7, 20, 20, 20, 20, 20, 20, 31, 20, 20, 20, 13, 20, 20, 20, 20, 20, 15, 30, 30, 20, 20, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 26, 26, 17, 17, 17, 2, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32]
--------------------------------------------------------------------------------
Ground truth labels:
[11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 13, 13, 13, 13, 13, 13, 13, 13, 13, 9, 9, 9, 9, 9, 9, 9, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 12, 12, 12, 12, 12, 12, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 23, 23, 23, 23, 23, 23, 23, 23, 23, 23, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3]
wq2012 commented 2 years ago

Diarization evaluations are supposed to be permutation invariant. Meaning [1,1,2,2,3] is equivalent to [3,3,2,2,1].