search

open-mmlab / PowerPaint

[ECCV 2024] PowerPaint, a versatile image inpainting model that supports text-guided object inpainting, object removal, image outpainting and shape-guided object inpainting with only a single model. 一个高质量多功能的图像修补模型，可以同时支持插入物体、移除物体、图像扩展、形状可控的物体生成，只需要一个模型
https://powerpaint.github.io/
MIT License
364 stars 18 forks source link

Questions about training and data prepare #49

Open majiajiong opened 2 weeks ago

majiajiong commented 2 weeks ago

I want to train a Brushnet similar to PowerPaintV2. If i use the text_encoder from PowerPaintV2, what should i prepare data ?

  1. When using random mask, the prompt input for Brushnet is "P_ctxt"?
  2. When using segmentation mask, the prompt input for Brushnet is "P_obj"?
  3. The Unet part of basemodel is frozen and the prompt input is the whole image caption?

Should i put the image caption to Brushnet?