search

pyannote / pyannote-metrics

A toolkit for reproducible evaluation, diagnostic, and error analysis of speaker diarization systems
http://pyannote.github.io/pyannote-metrics
MIT License
183 stars 30 forks source link

Add Decision Cost Function metric #46

Closed nryant closed 4 years ago

nryant commented 4 years ago

Adds Decision Cost Function (DCF) metric used by NIST for evaluating SAD system output in OpenSAT.

hbredin commented 4 years ago

Thanks @nryant.

Would you mind mentioning this new metric in the documentation as well?

https://github.com/pyannote/pyannote-metrics/blob/69a7140fff51722a4cc3d1b422ca1bb21d94c801/docs/source/reference.rst#L38-L47

nryant commented 4 years ago

@hbredin , I updated the docs with a blurb about decision cost function. Let me know if that is satisfactory.

hbredin commented 4 years ago

That’s great! Thanks.

Will release a new version unless you have other contributions in the pipeline.

hbredin commented 4 years ago

Oh. One last question. Is it Decision or Detection Cost Function?

nryant commented 4 years ago

Ack. Should be Detection Cost Function. Thanks for the catch...guess I've spent too much time dealing with ROC curves recently. I've updated the pull request to reflect this.

hbredin commented 4 years ago

FYI, I just released v3.0, that includes this PR:

http://pyannote.github.io/pyannote-metrics/reference.html?highlight=detectioncostfunction#pyannote.metrics.detection.DetectionCostFunction