Spico197 / DocEE

🕹️ A toolkit for document-level event extraction, containing some SOTA model implementations.
https://doc-ee.readthedocs.io/
MIT License
232 stars 36 forks source link

DueeFin结果讨论 #43

Closed KgLearner closed 2 years ago

KgLearner commented 2 years ago

Idea sharing While sharing what you want to do, make sure to protect your ideas.

Problems 嗨,我跟着您的repo 训练完了'PTPCG_P1-DuEE_fin-woTgg-wOtherType'任务(即|R|=1, Tgg=×).对于Results文件夹的结果有些疑惑,特来请教。 total_results: [ { "ModelType": "TriggerAwarePrunedCompleteGraph", "Total": { "precision": "68.8", "recall": "62.4", "f1": "65.5" }] 请问total_results的f1值是按照下图(图1)的方式计算的吗 捕获

其次 "m2m": { "classification": { "precision": "94.696", "recall": "93.718", "f1": "94.204" }, "entity": { "precision": "80.362", "recall": "85.863", "f1": "83.022" }, "combination": { "precision": "22.823", "recall": "24.050", "f1": "23.421" }, "rawCombination": { "precision": "22.448", "recall": "21.756", "f1": "22.097" }, "overall": { "precision": "68.838", "recall": "62.408", "f1": "65.465" }, "instance": { "precision": "21.141", "recall": "22.977", "f1": "22.021" } } 这里的combination的分数是论元的提取分数吗,如果是,则22的f1值是否说明该模型对论元提取任务的参考欠佳。

Others Other things you may want to share or discuss.

Spico197 commented 2 years ago

您好,

  1. DuEE-fin的最终结果需要提交至百度的线上系统之后才可以获取最终结果。线下的评价方式和ChFinAnn数据集的评价方法保持一致;
  2. 文档所示,combination指组合的评价指标,您提到的“论元”更贴近“entity”(实体)的指标。