search

zjunlp / EasyEdit

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.
https://zjunlp.github.io/project/KnowEdit
MIT License
1.74k stars 210 forks source link

About selecting demonstration when testing IKE #244

Closed yaohui120 closed 3 months ago

yaohui120 commented 4 months ago

In function apply_ike_to_multimodal_model in EasyEdit-main/easyeditor/models/ike/ike_main.py, the last line is

icl_examples.append(f'New Fact: {new_fact}\nPrompt: {new_fact}\n\n')

I'm a little confused. Why adding the question and answer of a testing sample when applying ike? My understanding is that the input of model should not contain the true answer of the testing question.

tbozhong commented 4 months ago

Thanks for your attention to our work.

It seems that you have mentioned this question before in #215, we just follow the setting of IKE. I'd suggest direct contact with the authors to seek further clarification. They may be better poised to assist you in your inquiry.

yaohui120 commented 4 months ago

Thank you for your help. I'll do experiments to explore the influence.