search

zjunlp / EasyEdit

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.
https://zjunlp.github.io/project/KnowEdit
MIT License
1.74k stars 210 forks source link

Reproduces knowedit data #272

Closed kailinjiang closed 3 months ago

kailinjiang commented 3 months ago

图片1 Could you please provide a tutorial for the reproduction of this data? run_knowedit_llama2.py run_convsent_llama2.py run_Sanitation_llama2.py. And in run_knowedit_llama2.py, the seq_pre. json file is missing. Thank you very much.

littlefive5 commented 3 months ago

Hello, We have provided the tutorial before in Knowedit.md. If you have any problem please leave the error information. The seq_pre.json is to store the pre-edit performance of the model before editing in order to avoid repeat-computing when testing the same model with different methods. This would be computed when you first edit the dataset on the specific model and stored at the given place: https://github.com/zjunlp/EasyEdit/blob/main/examples/run_knowedit_llama2.py#L194 https://github.com/zjunlp/EasyEdit/blob/main/easyeditor/editors/editor.py#L256 https://github.com/zjunlp/EasyEdit/blob/main/easyeditor/editors/editor.py#L265

So, this file is based on your experiments. You can just skip this argument and the model would compute the pre-edit results automatically.

littlefive5 commented 3 months ago

I have updated some code, please try again.