search

zjunlp / EasyEdit

[知识编辑] [ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.
https://zjunlp.github.io/project/KnowEdit
MIT License
1.63k stars 200 forks source link

About KE and KN in edit.py #301

Closed yaohui120 closed 1 month ago

yaohui120 commented 1 month ago

I found that there is no KEHyperParams class, but there is config file of KE method. Would you provide that? Additionlly, in test_KN function in edit.py, is ground_truth the expected true answer? Besides, I thought target_new is the wrong prediction of the original model.

yaohui120 commented 1 month ago

Besides, I also found that function compute_multimodal_edit_quality_demo in /EasyEdit/easyeditor/evaluate/multimodal_evaluate.py doesn't consider the variable exact_match.

tbozhong commented 1 month ago

Besides, I also found that function compute_multimodal_edit_quality_demo in /EasyEdit/easyeditor/evaluate/multimodal_evaluate.py doesn't consider the variable exact_match.

Thanks for pointing this out. Please use compute_multimodal_edit_quality. compute_multimodal_edit_quality_demo is only for debugging. I will optimize it soon.

Additionlly, in test_KN function in edit.py, is ground_truth the expected true answer? Besides, I thought target_new is the wrong prediction of the original model.

ground_truth is the prediction of the original model, while target_new is the true answer.

yaohui120 commented 1 month ago

Thank you for your answer!