Add MTurk labeling agreement statistics to draft

bellecarrell / twitter_brand

In developing a brand on Twitter (and social media in general), how does what you say and how you say it correspond to positive results (more followers, for example)?

0 stars 1 forks source link

Add MTurk labeling agreement statistics to draft #84

Closed abenton closed 5 years ago

abenton commented 5 years ago

Add descriptive statistics for the MTurk labeling task (Krippendorf alpha, % agreement with gold label, confusion matrix) to the current draft.

abenton commented 5 years ago

Added the following to section 7.3 "Labeling Users"

Description of MTurk labeling, categories
# self-promoting, distribution over main specialization
Krippendorff's alpha inter-annotator agreement for each label type
Confusion matrices and agreement for labeling account type and main specialization compared to gold-labeled 200 users

Descriptive statistics will likely need to be cleaned up.