How to train semantic-segmentation-controlnet with my data

lllyasviel / ControlNet-v1-1-nightly

Nightly release of ControlNet 1.1

4.72k stars 374 forks source link

How to train semantic-segmentation-controlnet with my data #105

Open fangchuan opened 1 year ago

fangchuan commented 1 year ago

Hi, thanks for your awesome work! I am wondering how to train segmentation controlnet with my customed dataset. I noticed that the Predictor of OneFormer must use BGR image in pre-trained seg-controlnet, but how about the requirement for my segmentation image if I want to refine the pre-trained seg-contronet on my dataset? Do I need to format my segmentation result in BGR order? or RGB order in my Dataset class?

I am looking forward to your reply, appreciate it! @lllyasviel

PerlMonker303 commented 8 months ago

Have you found anything? Dealing with the same scenario and found this resource to be useful https://www.reddit.com/r/StableDiffusion/comments/11e4vxl/paint_by_color_numbers_with_controlnet/.