ArneBinder / pytorch-ie

PyTorch-IE: State-of-the-art Information Extraction in PyTorch
MIT License
75 stars 7 forks source link

add `annotation_processor` to `F1Metric` #404

Closed ArneBinder closed 9 months ago

ArneBinder commented 9 months ago

This allows to add special mapping logic before the annotations get compared for computing tp / fn / fp counts.

E.g. using the following method as annotation_processor when evaluating binary relations makes the comparison indifferent towards the argument labels:

from typing import Tuple

from pytorch_ie.annotations import BinaryRelation, Span
from pytorch_ie.core import Annotation

def to_binary_relation_without_argument_labels(ann: Annotation) -> Tuple[int, int, int, int, str]:
    if not isinstance(ann, BinaryRelation):
        raise ValueError("Annotation must be a BinaryRelation")
    if not isinstance(ann.head, Span) or not isinstance(ann.tail, Span):
        raise ValueError("BinaryRelation must have two Span arguments")
    return ann.head.start, ann.head.end, ann.tail.start, ann.tail.end, ann.label