search

bjascob / amr_coref

A python library / model for creating co-references between AMR graph nodes.
MIT License
8 stars 1 forks source link

Comparison to other AMR Coref Systems #1

Open bjascob opened 2 years ago

bjascob commented 2 years ago

The only other system that I'm aware of is https://github.com/Sean-Blank/AMRcoref as described in the paper "End-to-End AMR Coreference Resolution". The scores in the paper are relatively similar to this project's scores.

bjascob commented 2 years ago

There is also... https://github.com/IreneZihuiLi/VG-AMRCoref which has noticeably better scores.

The following is a quick comparison of the 3 different projects available on GitHub (including this one) AMR Coref Comparisons 2022-07-02.odt

plandes commented 10 months ago

Re Fu et al. and Li et al work: the scores you mention are the scores they report in their work. However, I have tried to reproduce their work and failed. I contacted both authors, and while I had more exchange with Li, they both stopped responding to my correspondence.

This is very unfortunate given Fu et al. reports a F1 ave from 34 to 62.4 (28.4 improvement over the baseline) and Li et al. reports a F1ave from 62.4 to 73.5 (11.1 improvement over the baseline), which is now the new baseline.

For all practical purposes, amr_coref is the SOTA for others using AMR coref in their own research.