search

sail-sg / EditAnything

Edit anything in images powered by segment-anything, ControlNet, StableDiffusion, etc. (ACM MM)
Apache License 2.0
3.3k stars 188 forks source link

Train ControlNet - custom resolution? #4

Open alelordelo opened 1 year ago

alelordelo commented 1 year ago

Hi, thanks for sharing this amazing repo! I was thinking of doing something similar, super impressive you related this just a couple days after SAM! 🚀

My dataset is 1024 x 768. Is it possible to train with a a custom?

Thanks!

gasvn commented 1 year ago

Currently, we are using SD 2.1 as the base model, so we are using 512x512 for training. Do you mean to train a model with resolution 1024 x 768? I think that's achievable.

alelordelo commented 1 year ago

Yup, train the model with 1024 x 768, instead of 512x512. Do you think that is possible?

I tried something like this in ControlNet original repo, but didn't work:

accelerate launch train_controlnet.py --pretrained_model_name_or_path=$MODEL_DIR --output_dir=$OUTPUT_DIR --dataset_name=fusing/fill50k --resolution=1024x768 --learning_rate=1e-5 --validation_image "./conditioning_image_1.png" "./conditioning_image_2.png" --validation_prompt "red circle with blue background" "cyan circle with brown floral background" --train_batch_size=4

gasvn commented 1 year ago

It should be able to work with 1024x768. Maybe there are some settings that make this resolution not working. I will check it and let you know.

alelordelo commented 1 year ago

Thanks @gasvn , if we can have custom aspect ratio that would open a lot of possibilities!