search

OpenDataLiteracy / Feedback

Research project on feedback, question and answer, and user comments on open data portals
0 stars 1 forks source link

Calculate IRR for first and second round coding #8

Open nniiicc opened 7 years ago

nniiicc commented 7 years ago

https://en.wikipedia.org/wiki/Cohen%27s_kappa

Inter rater reliability measure for first and second round coding.

nniiicc commented 7 years ago

@annieyan Ok - I just finished second-level of coding.

We can probably calculate a Kappa score for:

  1. Our first-level codes
  2. Our second level codes
annieyan commented 7 years ago

https://docs.google.com/a/uw.edu/spreadsheets/d/1M8WX-ldeiOUjJsITkIlI49vAD0nXOshhvpgB9eMhcJQ/edit?usp=sharing see here for first round IRR.

annieyan commented 7 years ago

updated IRR for second round : https://docs.google.com/a/uw.edu/spreadsheets/d/13SAMGONDgXbVVI0Wu5hi-9NW7WkaPxBhskzauiROglk/edit?usp=sharing

nniiicc commented 7 years ago

:+1: