search

Con6924 / SPM

Official implementation of paper "One-dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applications".
https://lyumengyao.github.io/projects/spm
Apache License 2.0
124 stars 12 forks source link

About 1-dim Lightweight Adapter and Facilitated Transport #6

Closed lyufan closed 5 months ago

lyufan commented 5 months ago

Hi, thanks for this wonderful work!

I have some questions about the structure of SPM. From the paper and codes, is an SPM adapter essentially a Lora model?

And when erasing multiple concepts, is Facilitated Transport similar to Lora merge (weighted sum of Lora model parameters)? When loading SPM, is only one weighted SPM parameter loaded, not multiple SPM parameters?

Is it just that Facilitated Transport calculates an adaptive weight based on the similarity of the prompt and the erased concepts?

Thank you very much!

Con6924 commented 5 months ago
  1. About structure: SPM, in fact, can borrow any adapter design to achieve its performance. LoKr, LoHa, etc., with our proposed losses should also perform well.
  2. About Facilitated Transport (FT): it's similar to a dynamic lora merge, with each weight of loras being adjusted online. All the parameters of SPMs should be loaded for erasing multiple concepts.
  3. About FT weights: your understanding is correct.
lyufan commented 5 months ago

Alright, I understand now, thank you for your answer!