Closed 1010648766 closed 6 months ago
Hello, thanks for your attention: We want to first clarify that our toolkit aims to support these editing methods for better use, but for some method-specific questions, we can just provide some experimental suggestions. We are not the authors of these methods, so for the specific model parameter selections, we recommend you refer to the original paper and experiment by yourself. In fact, we also need to conduct experiments and do hyper-parameter searching when we want to employ the new methods in a new model, and we're unable to make a universal conclusion for every model.
Back to the other questions, here are some suggestions:
Thanks again for your attention, and as for some bugs and errors in our toolkit, we will fix them as soon as possible.
Thank you for your interest in EasyEdit, every question you ask is very thoughtful. At present, Neurips DDL is coming, and I may not be able to reply to you one by one and solve your confusion. If there is something urgent in your use, you can send your contact information to my email: peng2001@zju.edu.cn. I will answer and discuss your questions, and the summary will be recorded in this issue in the future
Looking forward to further communication with you.
Sincerely, Peng
hi, do you have any further questions?
While the easyedit framework already supports editing for a multitude of models, I'm interested in how to add support for more models for specific knowledge editing methods.
In detail:
How can I add new supported models for the GRACE method? The
inner_params
setting in the yaml file typically looks likemodel.layers[27].mlp.down_proj.weight
. My question is, for new models, how do we choose the number of layers in this parameter?For the MEND and SERAC methods, pre-training is required before editing, which demands a high GPU memory and often results in out-of-memory (OOM) issues. Are there any memory-saving techniques available? Could you provide a reference for the amount of GPU memory required for pre-training?
How should I choose the number of layers for the
inner_params
in the yaml settings file for the MEND method?The ROME, MEMIT, PMET methods require determining through causal tracing. After obtaining the results (as shown in the figure below), how can I identify the layer(s) that need editing?
In the MELLO model, how can I determine the
target_modules
andgrace_layer
in the yaml file?For the KE method, how can I determine the number of layers in the
inner_params
?Does KE method need pre-training? What's the
archive
in the yaml file ?These does not exist KE subfolder in the
EasyEdit/model
folderFor the MALMEN method, how can I determine the number of layers in the
inner_params
?I'm looking forward to your guidance on these issues. Thank you in advance for your support.