search

sail-sg / EditAnything

Edit anything in images powered by segment-anything, ControlNet, StableDiffusion, etc. (ACM MM)
Apache License 2.0
3.24k stars 184 forks source link

Encode Mask for ControlNet #10

Closed SnowdenLee closed 1 year ago

SnowdenLee commented 1 year ago

Hi, I'm wondering how the masks are encoded as the input of ControlNet. If I understood correctly, the masks predicted by SAM are binary masks without class information. So did you randomly sample colors for each mask or?

Thanks a lot in advance!

gasvn commented 1 year ago

The color is used for visualization only. We use the encoded mask id as the input to the ControlNet. You can check this func for details. https://github.com/sail-sg/EditAnything/blob/8d978b5f73586ed1ac504bb72b613293ae47ad78/utils/sam_dataset.py#L34

SnowdenLee commented 1 year ago

Thank you!