Concerning restriction of CEAF metric

[panagiotidi]

A question concerning the evaluation metrics mentioned on the papers, and of course I might have misunderstood the way this works.

I have read that CEAF works under the restriction that "each key entity should be mapped to exactly one reference entity and vice versa". From what I understand, fast-coref might return overlapping chains that refer to different entities, therefore, not respecting the restriction.

Would I therefore assume correctly that CEAF would not give valid results on fast-coref?

[shtoshni]

So CEAFE essentially tries to find the best alignment between predicted clusters and ground truth clusters. The quoted statement is meant to say that this alignment should be one-to-one. See this paper which has illustrative examples regarding how computation is done - https://www.cs.cmu.edu/~hovy/papers/14ACL-coref-scoring-standard.pdf.

[panagiotidi]

Thank you for your reply, you are right. How about MUC and B-cubed? Would the same stand? Because I see that MUC relies on partitions created from links between elements, meaning that if one element belongs to two chains, then these two chains are also correferential. And also, B-cubed it seems that an entity can exist in one output chain. Maybe again I am mistaken, can you please confirm that these metrics would be valid?