Evaluation Question

[SXxinxiaosong]

您好，代码中的评估应该是token 级别的准确率计算，请问怎么改成 exact match 的 0/1 评估呢？

[XeeKee]

If you need string-level matching, you should make modifications here: https://github.com/zjunlp/EasyEdit/blob/main/easyeditor/evaluate/evaluate_utils.py.

If it's just 0/1 evaluation, you can set all non-1 values to 0.

[SXxinxiaosong]

请问一下，你们使用token level的accuracy 是出于什么考虑呢？~

[XeeKee]

we follow rome

[SXxinxiaosong]

Got it, thanks!