search

LeslieZhoa / Simple-Lora

diffusion lora chinese tutorial，虚拟idol训练中文教程
MIT License
95 stars 14 forks source link

how to use: convert_original_stable_diffusion_to_difdusers.py??? #2

Open jiachen0212 opened 2 months ago

jiachen0212 commented 2 months ago

Thanks for the great work of open source. I want to use LoRa structure to fine-tune based on ControlNet. It seems that the code does not support it yet?

LeslieZhoa commented 1 month ago

Hi @jiachen0212 ,what error did you encounter? Is it because the version of your diffuser is incorrect, or is your network restricted？If your diffusers is latest, you can try here

jiachen0212 commented 1 month ago

Hi @jiachen0212 ,what error did you encounter? Is it because the version of your diffuser is incorrect, or is your network restricted？If your diffusers is latest, you can try here

enen thanks very much，I solved this problem~:

git clone https://github.com/huggingface/diffusers.git  (and install the env~: conda install -c conda-forge diffusers)
cd diffusers/scripts 
python convert_original_stable_diffusion_to_diffusers.py --checkpoint_path the/ControlNet/models/epoch=30-step=22226.ckpt --dump_path the/pretrained_models/my_controlnet_seg --image_size 512 --prediction_type epsilon

then models/config.py change: self.basemodel = './pretrained_models/my_controlnet_seg' I can use lora to fine-tune my controlnet_seg model~