zjunlp / EasyEdit

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.
https://zjunlp.github.io/project/KnowEdit
MIT License
1.85k stars 225 forks source link

Mismatch of the input variable of editor.batch_edit() #393

Open Syon-Li opened 4 days ago

Syon-Li commented 4 days ago

Dear EasyEdit Team,

I think there is a mismatch for the batch edit function (under the editor class, see here). Should the "locality_ground_truth" be "portability_inputs" and "locality_prompts" be "locality_inputs" just like the edit function? (They are using the same prepare requests function.)

It is possible that I missed something or got something wrong.

Regards, Syon

zxlzr commented 4 days ago

Dear Syon,

Thanks for pointing out this issue, we will handle it as soon as possible.

EasyEdit Team

pengzju commented 4 days ago

Thank you for your question. I have updated the parameter names, and the function should now work correctly https://github.com/zjunlp/EasyEdit/commit/529b4410e8d2f88a8e40c195bd15652ff86b5856. However, please note that not all editing methods support batch editing. For further details, you can refer to the batch_editor.py file.

Let me know if you have any other questions!

zxlzr commented 3 days ago

hi, do you have any further questions?

Syon-Li commented 2 days ago

Dear EasyEdit Team,

I'm sorry for not getting back to you sooner, but I currently do not have any questions for this one. However, I do encounter an issue with implementing GRACE. The error message leads me to this function. I have solved it by changing it to:

def split_epsilons_in_half(self, nearest_key, smallest_distance):
    # print(smallest_distance, self.epsilons[nearest_key])
    # self.epsilons[nearest_key] = self.epsilons[nearest_key].to(torch.float)
    self.epsilons[nearest_key] = (smallest_distance.item() / 2) - 1e-5 # Cut nearest epsilon in half
    self.epsilons[-1] = smallest_distance.item() / 2 # Cut new epsilon in half  

I am not sure whether this is caused by my running environment, but I just inform you about this. Thank you very much for your responses and patience.

Regards, Syon

pengzju commented 2 days ago

This applies to the case where batch size (bs) is 1. If you want to edit multiple instances simultaneously, GRACE is likely not supported (we refer to the official implementation of GRACE for this: https://github.com/thartvigsen/grace).

pengzju commented 2 days ago

https://github.com/zjunlp/EasyEdit/blob/main/easyeditor/models/grace/grace_main.py#L21

Syon-Li commented 2 days ago

yeah, I did not use batch edit for grace, I used the editor.edit()