Implement different ways to calculate the inter-rater reliability

langtonhugh / asreview_irr

Code to automatically produce a report from ASReview on inter-rater reliability.

Apache License 2.0

6 stars 2 forks source link

Implement different ways to calculate the inter-rater reliability #13

Closed MvanSteenbergen closed 7 months ago

MvanSteenbergen commented 10 months ago

Currently, only the kappa statistic is calculated for the IRR. Implement different ways of calculating the IRR that are for example robust to missing data. Give options for comparing the different ways of calculating the inter-rater reliability, and statistics for comparing them.

langtonhugh commented 7 months ago

New IRR tests have been added and tested with #19