When annotations are sets of values, there are two ways of calculating raw percentage agreement: you can consider two annotations to be equivalent when all their elements match, or when at least one of their elements match. Véronis [1] calls the former case "min" agreement, and the latter "max". (Véronis considers only pairwise agreement, though the idea is trivially extensible to annotations applied by more than one rater.)
DKPro Statistics can calculate "min" percentage agreement as-is with PercentageAgreement. However, I don't think it currently supports computing "max" percentage agreement. It would be nice if support for this could be added.
When annotations are sets of values, there are two ways of calculating raw percentage agreement: you can consider two annotations to be equivalent when all their elements match, or when at least one of their elements match. Véronis [1] calls the former case "min" agreement, and the latter "max". (Véronis considers only pairwise agreement, though the idea is trivially extensible to annotations applied by more than one rater.)
DKPro Statistics can calculate "min" percentage agreement as-is with
PercentageAgreement. However, I don't think it currently supports computing "max" percentage agreement. It would be nice if support for this could be added.[1] Jean Véronis. A study of polysemy judgements and inter-annotator agreement. In Proceedings of SENSEVAL-1, 1998.