Question about MEMIT and PMET in editing Llama.

[Lut-hub]

Hi,

First of all, thank you very much for providing an excellent toolkit for knowledge editing.

However I have a problem with editing Llama using MEMIT and PMET.

I use the default parameters. When I pick batchsize=1, after about 450 edits, the probability of the output becomes very small, no matter how much I optimize the model:

PMET request sample: [What is the cause of death of Crowfoot?] -> [ tetanus]
Computing right vector (v)
Lookup index found: 9 | Sentence: What is the cause of death of Crowfoot?<s> tet | Token: foot
Rewrite layer is 8
Tying optimization objective to 31
Recording initial value of v* in attn
Recording initial value of v* in mlp
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + -0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08
loss 20.572 = 20.572 + 0.0 + 0.0 avg prob of [ tetanus] 4.66106939711608e-08

I tried hard to find out the reason, but I failed. So I would like to ask if you have encountered this during the editing process? Thanks!

[pengzju]

Thank you for your interest. 😊

• Hello, have you turned on continuous editing: keep_original_weight=False? It may be that MEMIT itself performs poorly in this setting.

• Also, which model are you editing?

[Lut-hub]

Thank you for your prompt response!

Yes, I use keep_original_weight=False to edit Llama2-7b.

It may be that MEMIT itself performs poorly in this setting.

Does this mean that MEMIT may not work properly with batch size = 1 when applied to sequential editing? When I use the same settings with ROME, even after 1k edits the final probability still could be around 0.98. This phenomenon confuses me.

Thank you once again for your assistance and for sharing your knowledge. 🙂

[pengzju]

I'm equally confused about your phenomenon, but I have some superficial understanding for your reference.

• Probability is not a measure of editing success. The most important thing is that the model can generate the correct token without catastrophic forgetting.

• In my many repeated experiments, ROME's continuous editing is very poor. The probability of 0.9+ you observed is probably due to overfitting. The final evaluation performance is still very poor. I guess the expression of MEMIT may just be impossible to fit (low probability).

• My suggestion is that you can use MEMIT-MASS, that is, set the bs (batch_size) of MEMIT to the number you want to edit (for example, 1K), and adopt the strategy of batch editing but single testing. The experimental results are expected to be considerable. (Don't Worry, This will not cause OOM)

Hope this helps, thank you~

[Lut-hub]

Thank you very much! 😃