open-mmlab / PowerPaint

[ECCV 2024] PowerPaint, a versatile image inpainting model that supports text-guided object inpainting, object removal, image outpainting and shape-guided object inpainting with only a single model. 一个高质量多功能的图像修补模型,可以同时支持插入物体、移除物体、图像扩展、形状可控的物体生成,只需要一个模型
https://powerpaint.github.io/
MIT License
364 stars 18 forks source link

textual inverison ? #47

Closed viperyl closed 1 week ago

viperyl commented 3 weeks ago

Are P_ctxt, P_obj, P_shape these three are actually textual inversion?

the textual inversion binding "ID" to text embedding, and power paint binding "task" to text embedding. power paint trained a new textual inversion and finetune the unet. am i misunderstanding?

zengyh1900 commented 1 week ago

Hi @viperyl,

Thanks for your interests! You're right! And, the only different part compared with textual inversion is that, textual inversion is often used for concept learning (e.g., pet dog, cat, toy, etc.) and we use it for task learning in PowerPaint, which correlates different task patterns with specific text embedding.