search

HSLCY / ABSA-BERT-pair

Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence (NAACL 2019)
https://www.aclweb.org/anthology/N19-1035
MIT License
493 stars 138 forks source link

Questions about Aspect accuracy #22

Open ayanesakura opened 4 years ago

ayanesakura commented 4 years ago

Hello,I have read your paper,excellent work! But in the paper you may did not mention how the accuracy is calculated, now I know the sentiment ACC is ACC of the classification task of sentence-pairs,but how about aspect acc?

sunny678 commented 4 years ago

Hi, aspect acc is the strict acc defined in [1]: image

[1] Yukun Ma, Haiyun Peng, and Erik Cambria. 2018. Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive lstm. In Proceedings of AAAI.

ayanesakura commented 4 years ago

Thank you for your reply， I calculated the apsect acc in that way, result is as below,the red is my result and the black is result in paper. image The sentiment acc is close to result in paper but the apsect accrate is too bad, maybe the reason is that the Bert model I used is different from the paper? I used the uncased_L-12_H-768_A-12 bert model. Or other reasons?

sunny678 commented 4 years ago

Did you use the "evaluation.py" file in the evaluation step? If not, you may use it: python evaluation.py --task_name sentihood_NLI_M --pred_data_dir results/sentihood/NLI_M/test_ep_4.txt

ayanesakura commented 4 years ago

Oh! I'm such an idiot, thank you!