kohya-ss / sd-scripts

Apache License 2.0
4.51k stars 762 forks source link

diffusers inference #320

Open anomySz opened 1 year ago

anomySz commented 1 year ago

I'm having trouble inference with diffusers.

When I convert fine-tuned ckpt to diffusers and then inference, I'm getting a completely different image than ckpt. The inference code is taken from diffusers.

Do you have any inference code that you use after converting to diffusers? The convert code I referenced is below. https://note.com/kohya_ss/n/n374f316fe4ad

kohya-ss commented 1 year ago

The latest version of conversion script is convert_diffusers20_original_sd.py in this repo. Please convert with this, if you didn't use this.

How are images different? For example, even if the seed is the same, Web UI and Diffusers will have different images. Is the quality of the image different?

anomySz commented 1 year ago

yeah. quality of the image is different.

i used version4, is this different from file you attached?

I fine-tuned it to a 3D image. after converting(ckpt2onnx), the result looks like 2d image(It is almost identical to the pretrained model.), but after reconverting(onnx2ckpt) the result is 3d image. So, I thought it was a problem with the inference code, so I asked if there was a separate inference code.

I used scripts directly using diffusers to infer the onnx model like this.(https://huggingface.co/docs/diffusers/optimization/onnx) and when i infer the ckpt, put it in the webUI.

kohya-ss commented 1 year ago

The conversion script is fundamentally same.

Diffusers has a pipeline to inference. The example is here: https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/text2img

Also this repo has a script to inference called gen_img_diffusers.py. Unfortunately the document for the script is in Japanese and outdated, but I think it will help (please ignore the installation process in the document).

https://note.com/kohya_ss/n/n2693183a798e