search

jerbarnes / semeval22_structured_sentiment

SemEval-2022 Shared Task 10: Structured Sentiment Analysis
75 stars 42 forks source link

MPQA dev split has no opinions #3

Closed amith-ananthram closed 3 years ago

amith-ananthram commented 3 years ago

Hey there! Sorry to spam you -- just wanted to call out that the dev split of MPQA that process_mpqa.sh produces contains no opinions. Not sure if that's expected -- I can just use part of the training split as my dev set so not a big issue but just wanted to draw your attention to it. Thanks!

>>> with open('dev.json', 'r') as f:
...   dev = json.load(f)
... 
>>> c = Counter()
>>> for sentence in dev:
...   c[len(sentence['opinions'])] += 1
... 
>>> c
Counter({0: 2024})
jerbarnes commented 3 years ago

Hey, no problem. You're correct. As the shared task is currently in the development phase, we have not released the dev annotations yet. When we start the evaluation phase, we'll release all the annotations.

amith-ananthram commented 3 years ago

Ah understood, thank you!