search

mainlp / awesome-human-label-variation

A curated list of awesome datasets with human label variation (un-aggregated labels) in Natural Language Processing and Computer Vision, accompanying The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation (EMNLP 2022)
76 stars 8 forks source link

Suggestion for Relevant Work on Your Repository #5

Closed RuyuanWan closed 1 year ago

RuyuanWan commented 1 year ago

Hello,

I'm reaching out to express my appreciation for the valuable open-source repository you've organized!

In line with your project's theme, I'd like to suggest adding my recent AAAI paper, "Everyone's Voice Matters: Quantifying Annotation Disagreement Using Demographic Information" to the key references of your GitHub repository.

This paper takes a unique look at label variation by studying the impact of annotators' demographics on their disagreements in NLP tasks. Our findings reveal that demographic information, such as gender, ethnicity, and education level, is instrumental in predicting annotator disagreements.

I look forward to hearing from you.

Best regards, Ruyuan

bplank commented 1 year ago

hi @RuyuanWan Many thanks for this reference and the list of datasets.