bin "other" users - Githubissues

bellecarrell / twitter_brand

In developing a brand on Twitter (and social media in general), how does what you say and how you say it correspond to positive results (more followers, for example)?

0 stars 1 forks source link

bin "other" users #71

Closed bellecarrell closed 5 years ago

bellecarrell commented 5 years ago

Run through all user annotations and bin \emph{Other} users appropriately, or construct new specialization bins if we missed a popular category.

bellecarrell commented 5 years ago

I see this as the first task for this (and it was listed first in the document). @abenton questions:

Do you see the process as
- manually bin users currently labeled as "other" into current bins
- any users who don't fit cluster and add new specializations if necessary
If the process above is valid, how do we reconcile our manual annotations with the fact that the others are done by paid annotators?

abenton commented 5 years ago

I would reconcile as follows:

If we assign a label to a user, the user gets that label
if not, use MACE to choose the most likely label

bellecarrell commented 5 years ago

started by pulling out other rows. need to collect ids from original input, as ids in results files are truncated

bellecarrell commented 5 years ago

labeled first 50 and started an uncertain file. 21/50 uncertain across prestudy, first batch and second, 707 assignments to annotate. across 3 days that's 236 annotations per day. going to spread out with coding tasks to avoid decline in annotation quality

bellecarrell commented 5 years ago

100 labels/hour, if not a little faster. through first 265 or so. going to get through around 100 or so more tonight