the computing methods of Edit succ, Portability, Locality, Fluency

[kailinjiang]

I'm really sorry to bother you again by writing I would like to confirm the relationship between the four indicators you show in your paper and the json result run by run_knowedit_llama2.py. Edit succ = post rewrite_acc Portability = average (Relation_Specificity_acc, Forgetfulness_acc) Locality = average (Subject_Aliasing_acc, reasoning_acc, Logical_Generalization_acc) Fluency = ngram_entropy Hope to get your reply! Thank you very much!

Or can you give an example of the calculation in the data below

"post": { "rewrite_acc": [ 0.4166666666666667 ]. "locality": { "Relation_Specificity_acc": [ 1.0, 1.0, 0.25, 0.8, 1.0, 1.0 ]. "Forgetfulness_acc": [ 0.5 ] }, "portability": { "Subject_Aliasing_acc": [ 0.4166666666666667 ]. "Logical_Generalization_acc": [ 0.0 ] }, "fluency": { "ngram_entropy": 6.271604245343871 } } },

[littlefive5]

We provide the code here: https://github.com/zjunlp/EasyEdit/blob/main/easyeditor/editors/utils.py#L11

[kailinjiang]

"Relation_Specificity_acc": [ 1.0, 1.0, 0.25, 0.8, 1.0, 1.0 ]. 如果是这样的情况，是算这里面的所有值的均值，但是数量怎么定义，数量是+1还是+6呢

[pengzju]

均值

1

[kailinjiang]

非常感谢你们的耐心回答！ Edit succ = average（post rewrite_acc） Portability = average (Relation_Specificity_acc, Forgetfulness_acc) Locality = average (Subject_Aliasing_acc, reasoning_acc, Logical_Generalization_acc) Fluency = average（ngram_entropy）

关于这个问题，我还想再问最后一个问题，希望能够得到准确回答 核心就是Portability和Locality，例如Portability，以一条数据为例，先算Relation_Specificity_acc和 Forgetfulness_acc内部的均值，然后再求average (Relation_Specificity_acc, Forgetfulness_acc)，作为这一条数据的Portability，然后再算整个数据集的average （Portability），作为整个数据集的Portability 请问是这样的吗

[pengzju]

是的，计算过程没问题。但是有个东西弄反了

Locality = average (Relation_Specificity_acc, Forgetfulness_acc) Portability = average (Subject_Aliasing_acc, reasoning_acc, Logical_Generalization_acc)

你可以参考这里的数据格式？https://huggingface.co/datasets/zjunlp/KnowEdit?row=0

[pengzju]

不客气，有其他问题随时沟通～ 如果解决了您的疑虑，请帮我关闭此issue。