jerbarnes
commented
2 years ago Dear Rob,
You're right that the labels for intensity have not been normalized. This is because we decided not to evaluate these labels in the current iteration of the shared task and so we left them as they were originally labeled in the various datasets. I agree it would be useful to normalize them at some point in time.
To answer your question, None, Standard, and Average would all need to be normalized to a single label (perhaps Average). It would also be worth double checking the None labels to make sure that these are truly Average and not simply annotation errors where the intensity was not labeled.
Best, Jeremy
Dear organizers,
The vocabulary for the intensity label is confusing, there seems to be some cases with None, can we consider those to be standard/average (for example: ula/Article247_500-9) ? Also is there a difference between the label standard and average?, the neutral class seems to contain no instances labeled with standard.
Best, Rob