search

pyannote / pyannote-metrics

A toolkit for reproducible evaluation, diagnostic, and error analysis of speaker diarization systems
http://pyannote.github.io/pyannote-metrics
MIT License
183 stars 30 forks source link

Segmentation metrics fail with empty annotations #36

Open hbredin opened 4 years ago

hbredin commented 4 years ago

Description

Segmentation metrics fail with empty annotations.

Steps/Code to Reproduce

from pyannote.core import Annotation, Segment
reference = Annotation()
hypothesis = Annotation()

from pyannote.metrics.diarization import SegmentationPurity
purity = SegmentationPurity()
purity(reference, hypothesis)
ZeroDivisionError: float division by zero
from pyannote.metrics.diarization import SegmentationCoverage
coverage = SegmentationCoverage()
coverage(reference, hypothesis)
ValueError: zero-size array to reduction operation maximum which has no identity
from pyannote.metrics.diarization import SegmentationPurityCoverageFMeasure
both = SegmentationPurityCoverageFMeasure()
both(reference, hypothesis)
ValueError: zero-size array to reduction operation maximum which has no identity

Expected Results

It does not really make sense to compute segmentation purity and coverage on empty files but I guess a more explicite message should be raised.

Versions

pyannote.core==3.0
pyannote.database==2.3.1
pyannote.metrics==2.1