search

zjunlp / EasyEdit

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.
https://zjunlp.github.io/project/KnowEdit
MIT License
1.88k stars 232 forks source link

WISE CONTEXT_TEMPLATES_CACHE #329

Closed SXxinxiaosong closed 3 months ago

SXxinxiaosong commented 3 months ago

Hello, could you please explain the role of CONTEXT_TEMPLATES_CACHE in the WISE method? Thank you very much!

pengzju commented 3 months ago

We sample 10 texts from LMs to construct the prefix: five of length 5 and five of length 10. The intention is to ensure that the editor(WISE) accounts for the different contexts. In the paper we also provide this prefix for ROME, MEMIT, and GRACE

pengzju commented 3 months ago

We sample 10 texts from LMs to construct the prefix: five of length 5 and five of length 10. The intention is to ensure that the editor(WISE) accounts for the different contexts. In the paper we also provide this prefix for ROME, MEMIT, and GRACE

https://github.com/zjunlp/EasyEdit/blob/09f322ea65265569a992e96b5237bb6329e69c61/easyeditor/models/rome/rome_main.py#L161

https://github.com/zjunlp/EasyEdit/blob/09f322ea65265569a992e96b5237bb6329e69c61/easyeditor/models/memit/memit_main.py#L302

SXxinxiaosong commented 3 months ago

谢谢~ 实验过程中，print(context_templates)得到的prefix如下，这种没有意义的前缀对于WISE的Rel./Gen.有帮助吗？ ['{}', 'I\n\n\n\n {}', 'You Are You Ready {}', 'Because\n\n\n {}', 'Yes\n\n\n {}', 'Q: What is the difference between {}', 'I\n\n\n\n\n\n\n\n\n {}', 'You Are You Ready to Take Your Business to {}', 'Because\n\n\nimport numpy as np\n {}', 'Yes\n\n\nimport numpy as np\n {}', 'Q: What is the difference between a "salesperson {}']

pengzju commented 3 months ago

很确定的说有，后续我会补充paper内容，目前wise还属于preprint版本。你也可以看看rome论文的附录Key Selection有相关讨论。

SXxinxiaosong commented 3 months ago

请问是随机选择的前缀还是实验得到的这几个效果比较好?

pengzju commented 3 months ago

没有探索过，根据上下文我应该只尝试过你说的“实验得到的”，实验结果来看有比没有好，但什么形式/规则的前缀好并不清楚，It’s the future work.

SXxinxiaosong commented 3 months ago

好的，谢谢您的解答~