search

Cartus / AGGCN

Attention Guided Graph Convolutional Networks for Relation Extraction (authors' PyTorch implementation for the ACL19 paper)
MIT License
432 stars 88 forks source link

Questions about the updated results #12

Closed zzysay closed 5 years ago

zzysay commented 5 years ago

Hi, In the issue 7, you said that "For the mean and std of F1 score in my experiments, the stats is 68.2% +- 0.5%. Also, we will update this score on our paper, for a fair an concrete comparison to other methods."

However, In the final version of ACL19 and the latest version in arxiv, the F1 score on TACRED are both 69.0. May I ask where the latest results were released?

imnujf commented 5 years ago

Hi, In the issue 7, you said that "For the mean and std of F1 score in my experiments, the stats is 68.2% +- 0.5%. Also, we will update this score on our paper, for a fair an concrete comparison to other methods."

However, In the final version of ACL19 and the latest version in arxiv, the F1 score on TACRED are both 69.0. May I ask where the latest results were released?

How do you get 68.2% +- 0.5% ??????? Only 67.5% in my experiment!!!!!! Could you help me? My email address is imnujf@163.com. Thanks a lot.

marchbnr commented 5 years ago

Hi, In the issue 7, you said that "For the mean and std of F1 score in my experiments, the stats is 68.2% +- 0.5%. Also, we will update this score on our paper, for a fair an concrete comparison to other methods." However, In the final version of ACL19 and the latest version in arxiv, the F1 score on TACRED are both 69.0. May I ask where the latest results were released?

How do you get 68.2% +- 0.5% ??????? Only 67.5% in my experiment!!!!!! Could you help me? My email address is imnujf@163.com. Thanks a lot.

I get similar results: P: 71.64 +-0.74, R: 63.55 +-0.53, F1: 67.35 +-0.44

Cartus commented 5 years ago

Hi @zzysay,

We updated the mean result from 5 independent runs in the latest version on arxiv and the ACL oral. The latest version on arxiv should be available soon. For the ACL version, we report the result of the pretrained model we released here. We cannot modify that version since the camera ready period passed.

Cartus commented 5 years ago

Hi @imnujf

I guess you have the same issue here: https://github.com/Cartus/AGGCN_TACRED/issues/7

It would be appreciated if you can kindly refer to the discussion there. If you have further questions, feel free to drop me an email as stated in the paper.