How to calculate locality of convsent

[Blank141]

how to calculate locality of convsent i add my solution to compute_sent_metric in easyeditor/evaluate/evaluate.py ` I don't know that is right or not

related issue is https://github.com/zjunlp/EasyEdit/issues/147

evaluate.py中的compute_sent_metric

# Convsent的locality计算
def kl_divergence_from_logits(base_logits, edit_logits):
    # 计算概率分布
    base_distribution = torch.nn.functional.softmax(base_logits, dim=-1)
    edit_distribution = torch.nn.functional.softmax(edit_logits, dim=-1)

    # 计算KL散度
    kl_div = torch.nn.functional.kl_div(torch.log(base_distribution), edit_distribution, reduction='batchmean')

    return kl_div

# 计算内部KL散度
inner_kl_div = kl_divergence_from_logits(inner_base_logits, inner_edit_logits)
print(inner_kl_div)
# 计算外部KL散度
outer_kl_div = kl_divergence_from_logits(outer_base_logits, outer_edit_logits)
print(outer_kl_div)
result['locality'] = (inner_kl_div.item() + outer_kl_div.item())/2

'

[shengyumao]

Hello~ As illustrated in paper SERAC, the dd metric of convsent is implemented by the KL_DIV of outer topics, and we simply consider dd as the locality metric for ConvSent in KnowEdit. You can find the implementation in EasyEdit.

[Blank141]

Thank you for your reply! great job