search

Cartus / AGGCN

Attention Guided Graph Convolutional Networks for Relation Extraction (authors' PyTorch implementation for the ACL19 paper)
MIT License
433 stars 88 forks source link

micro F1 and macro F1 #8

Closed AaronWhite95 closed 5 years ago

AaronWhite95 commented 5 years ago

The F1 score in your paper is macro F1, but in your code you calculate the micro score. Can you share us with the code to calculate macro F1 score? Thank you so much!

Cartus commented 5 years ago

The current code of this repo is for TACRED dataset. We report micro-averaged F1 scores on this dataset as the SOTA system: https://nlp.stanford.edu/pubs/zhang2018graph.pdf

The evaluation script is borrowed from this repo: https://github.com/qipeng/gcn-over-pruned-trees, which is the official implementation of the paper mentioned above.

Cartus commented 5 years ago

I do have a script for evaluating macro F1 score, you can give me your email address or others. Then I can just send it to you.