search

clbustos / statsample

A suite for basic and advanced statistics on Ruby.
http://github.com/clbustos/statsample
BSD 3-Clause "New" or "Revised" License
403 stars 96 forks source link

Add support for Krippendorff's alpha reliability coefficient #13

Open ludwigschubert opened 11 years ago

ludwigschubert commented 11 years ago

(First of all, thanks so much for this package! While it took me some time to get used to it, it's super handy even for simple statistics in business applications.)

I might lack the knowledge to combine the pieces of this framework, but I'm missing explicit support for measuring reliability between raters, e.g. Krippendorff's alpha.

R provides a nice interface for this here.

Thanks again for your efforts and time in supporting this project!

clbustos commented 11 years ago

Can you provide me a reference? A journal or book reference will be cool :)

ludwigschubert commented 11 years ago

Yes.

The Original paper by Prof. Krippendorf is available here. Together with a colleague he also provided a (to me unreadable) implementation in SAS here.

Instead, I'd recommend working of Matlab code and the wikipedia article on the subject. I'm currently using R's package "irr"'s implementation, which can be found here - I extracted the specific file for you in this GIST.

Let me know if there's anything else I can help with - unfortunately I had about one introductory class in statistics, so the finer points of Krippendorf's alpha are entirely lost on me.