Thank you very much for your work. Except segmentation labels, how to extract BLIP captions for image segmentation region?

open-mmlab / PowerPaint

[ECCV 2024] PowerPaint, a versatile image inpainting model that supports text-guided object inpainting, object removal, image outpainting and shape-guided object inpainting with only a single model. 一个高质量多功能的图像修补模型，可以同时支持插入物体、移除物体、图像扩展、形状可控的物体生成，只需要一个模型

https://powerpaint.github.io/

MIT License

642 stars 41 forks source link

Thank you very much for your work. Except segmentation labels, how to extract BLIP captions for image segmentation region? #82

Open carlfu127 opened 1 month ago

carlfu127 commented 1 month ago

We use the semantic segmentation subset of OpenImage v6[15] as the main dataset for multi-task prompt tuning. In addition, following Smartbrush [32], we use segmentation labels and BLIP captions[16] as local text descriptions.

zengyh1900 commented 1 month ago

hi @carlfu127 , we cropped the region according to the segmentation mask and use the cropped region for blip caption.