Open keanepotato opened 2 weeks ago
Additionally, I believe this doesn't count the exact matches of aspects, but rather only if a string is contained within another string.
So, in the case where the model matches "of", but the ground truth is "bowl of sushi", this is marked as a true positive as well.
Hi Kevin,
Thanks for your contribution to the ABSA task. I just wanted to bring your attention to the following code block in your utils.py file within the InstructABSA folder. Seems that because each matched prediction isn't removed from the pred_val list, in the case where gt_val contains repeated instances ['food', 'food'], but pred_val contains only one instance ['food'], the model is considered to predict all instances correctly, despite missing out the second 'food'?