What is the position that the diff vector is added to the pretrained model?

dguo98 / DiffPruning

Parameter Efficient Transfer Learning with Diff Pruning

Apache License 2.0

72 stars 9 forks source link

What is the position that the diff vector is added to the pretrained model? #5

Open CaffreyR opened 2 years ago

CaffreyR commented 2 years ago

Hi @dguo98 , I have a small question that in your paper you add a diff vector to the pertained model to finetune the model. May I ask that what position is the vector added to? Many thanks to your time and consideration.