Maybe a little bug in evaluate_utils.py

zjunlp / EasyEdit

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.

https://zjunlp.github.io/project/KnowEdit

MIT License

1.74k stars 210 forks source link

Maybe a little bug in evaluate_utils.py #253

Closed Luoyang144 closed 3 months ago

Luoyang144 commented 3 months ago

Hi, I find this line: https://github.com/zjunlp/EasyEdit/blob/73a1efbd3d92c270fed4044e62a3b269ffcabe2f/easyeditor/evaluate/evaluate_utils.py#L90 In this loop, I think it should be: target_new_tokens = tok.encode(target_new). By the way, this is a bug that occurred when running the editor, so I discovered this problem. If there is any problem with my modification, please point it out.

XeeKee commented 3 months ago

Thank you very much for your correction; we have updated the code. Previously, EasyEdit mainly handled single edits, which is why we had not noticed this bug. Wishing you a pleasant life!

Luoyang144 commented 3 months ago

Thanks for reply, I will close this issue. Thank you again for your contribution!