Detection Error Rate Computation

[ssarfjoo]

Dear Herve Bredin,

For computing the detection error rate, DetER is defined as

DetER = \text{false alarm rate} + \text{miss detection rate} 

Where each of them are defined as:

\text{false alarm rate} = \frac{FP}{N}

\text{miss detection rate} = \frac{FN}{P}

However you compute the false alarm rate as:

\text{false alarm rate} = \frac{FP}{P},

where P and N are the duration of speech and silence in the reference file. Is there any speciffic reason behind this definition?

Best, Saeed Sarfjoo

[hbredin]

Thanks for your interest in pyannote.metrics, @ssarfjoo.

This is not how the detection error rate is defined in pyannote.metrics, which is consistent with the definition of diarization error rate.

pyannote.metrics does provide detection precision and recall metrics if that suits your needs.

[ssarfjoo]

Thanks for fast answering. Based on your documentation, the detection error rate is: (False alarm + Miss detection)/ Total or False alarm/Total + Miss detection/Total, where Total is the total duration of speech in the reference. My question is, is this valid method for computing the false alarm rate?

Best, Saeed

[hbredin]

Thanks for fast answering. Based on your documentation, the detection error rate is: (False alarm + Miss detection)/ Total

Yes. That's how it is computed in pyannote.metrics: the detection error rate is not defined as the sum of false alarm rate and miss detection rate.

or False alarm/Total + Miss detection/Total, where Total is the total duration of speech in the reference. My question is, is this valid method for computing the false alarm rate?

If you are looking to compute a false alarm rate (which, I repeat, is not used to compute the detection error rate), you should divide "duration of false positive" by "actual duration of positive", like you suggested.

[ssarfjoo]

Thanks for the clarification.