search

pyannote / pyannote-metrics

A toolkit for reproducible evaluation, diagnostic, and error analysis of speaker diarization systems
http://pyannote.github.io/pyannote-metrics
MIT License
186 stars 33 forks source link

Results are wrong if I change speaker label format in mdtm file. #26

Closed viannecc closed 5 years ago

viannecc commented 5 years ago

I installed the pyannote-audio package through conda virtual env commands according to your suggestions. Then, I arranged my own dataset according to your documents for pyannote usage. In tst.mdtm file, I denote label for speaker info is 'o' or 'c', the result is normal, which is: Confusion: 0.09935131116341267

However, after I change the speaker label into 'speaker1' or 'speaker2', the result is abnormal, which is: Confusion: 0.14562803946816988

Via debug, I found something goes wrong with optimal_mapping(reference, hypothesis) function that returns wrong mapping relationship.

Hope you mates can fix this bug.

hbredin commented 5 years ago

Can you please provide a detailed guide on a how to reproduce this error? (e.g. by providing your MDTM files and telling me which command you ran...)

hbredin commented 5 years ago

Closing. Please re-open and provide the requested information if the problem still exists.