search

zjunlp / EasyEdit

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.
https://zjunlp.github.io/project/KnowEdit
MIT License
1.77k stars 213 forks source link

About the implements of SERAC #216

Closed Luoyang144 closed 5 months ago

Luoyang144 commented 5 months ago

Hi, I'm using SERAC to edit llama2. And I'm using model provided in this link, here is issue link. Here is some question:

  1. On which data set was llama-160m trained?
  2. I used the counterfact and zsre data sets to test the output of the replacement model in the checkpoint, but neither of them output the correct answer. Why is this?
  3. I checked the code of SERAC and found that new_model.train(self.training) is used in it. Doesn’t SERAC need training during training? Why does it need to call this function?
XeeKee commented 5 months ago
  1. Sorry, due to the passage of time, I am unable to provide related information.
  2. It might be because the model was underfit during previous training sessions or the model was damaged.
  3. In SERAC, the large model does not require training; it is the smaller model that needs training, which in your case is llama-160m. For the training code of SERAC, you can refer to this section.EasyEdit/easyeditor/trainer/EditTrainer.py
zxlzr commented 5 months ago

hi, do you have any further questions?

zxlzr commented 5 months ago

I will close this issue.

If you have any inquiries or need assistance, please don't hesitate to reach out to us. We are committed to providing ongoing support and maintenance for EasyEdit.