search

jerbarnes / semeval22_structured_sentiment

SemEval-2022 Shared Task 10: Structured Sentiment Analysis
75 stars 42 forks source link

Miss sentences in test #26

Closed DrJZhou closed 2 years ago

DrJZhou commented 2 years ago

Hi, we submit a result to the evaluation and got the error of missing sentences as follows. However, the missing ids are not included in the test data. Could you please help us to deal with it.

assert g.issubset(p), "missing some sentences: {}".format(g.difference(p)) AssertionError: missing some sentences: set([u'000778-54-01', u'000335-01-01', u'108702-07-01', u'700798-09-01', u'108702-07-03', u'108702-07-02', u'701679-14-01', u'000227-14-01', u'500718-05-02', u'500718-05-03', u'500718-05-01', u'301571-18-01', u'000778-11-02', u'700798-09-02', u'000778-11-03', u'700798-09-03', u'200102-06-01', u'200102-06-02', u'104926-03-05', u'500718-09-04', u'500718-09-05', u'500718-09-02', u'500718-09-03', u'700798-15-01', u'500718-09-01', u'104926-03-06', u'202263-10-01', u'200102-02-01', u'200102-02-02', u'002907-08-02', u'002907-08-03', u'002907-08-01', u'000227-17-01', u'000286-10-02', u'000286-10-01', u'000227-17-02', u'200942-10-02', u'200942-10-01', u'304685-06-02', u'304685-06-03', u'304685-06-01', u'000735-05-02', u'000735-05-01', u'105373-03-02', u'000778-42-03', u'105373-03-01', u'000778-42-06', u'000778-42-04', u'004567-08-01', u'400295-09-01', u'300002-02-01', u'202263-05-01', u'300433-07-04', u'300433-07-01', u'300433-07-02', u'300433-07-03', u'304825-01-02', u'304825-01-01', u'000227-11-02', u'000778-04-01', u'000778-04-02', u'000778-04-03', u'700798-15-09', u'200102-20-01', u'200102-20-02', u'600614-02-01', u'000227-06-03', u'000227-06-02', u'000227-06-01', u'000778-21-01', u'000778-21-03', u'000778-21-02', u'104987-18-01', u'102329-24-01', u'102329-24-03', u'102329-24-02', u'102329-24-05', u'102329-24-04', u'102329-24-07', u'102329-24-06', u'104987-14-06', u'104987-14-04',

jerbarnes commented 2 years ago

Hi,

It seems we had left the dev data for Norec. Apologies. It should be fixed now.

DrJZhou commented 2 years ago

@jerbarnes Could we have one more choice to submit the results since we lose one?

DrJZhou commented 2 years ago

We submit the results, it seems the test norec data is an empty file without ground truth.

Hi,

It seems we had left the dev data for Norec. Apologies. It should be fixed now.

jerbarnes commented 2 years ago

I just checked and the data seems fine. I also checked your submission and the output on codalab, and everything seems fine there. Is there a reason you think the test norec data is empty?

jerbarnes commented 2 years ago

@jerbarnes Could we have one more choice to submit the results since we lose one?

Sure, I deleted the failed submission, so hopefully that will give you an extra opportunity today.