search

catalpa-cl / inceptalytics

An easy-to-use API for analyzing INCEpTION annotation projects.
16 stars 7 forks source link

allow passing keyword arguments for IAA measures. #12

Open simulacrum6 opened 2 years ago

simulacrum6 commented 2 years ago

Some IAA measures allow some customisation (e.g. alpha and beta values in gamma agreement score). Users should be able to pass these through using keyword arguments:

gamma_score = view.agreement(measure='gamma', arguments={'alpha': 0.5, 'beta': 1.5, fast=False})

Unifying agreement measures under a shared interface and allowing to pass them would probably be a good addition as well, making an extension to multiple measures simpler.

gamma_score = view.agreement(measure=Gamma(alpha=0.5, beta=1.5, fast=False))
gamma_score, alpha_score = view.agreement(measure=[Gamma(), KrippendorffsAlpha()])