search

ssube / onnx-web

web UI for GPU-accelerated ONNX pipelines like Stable Diffusion, even on Windows and AMD
https://discord.gg/7CdQmutGuw
MIT License
201 stars 26 forks source link

Different image result between stable diffusion webui and onnx-web when using lora #413

Closed newgrit1004 closed 8 months ago

newgrit1004 commented 10 months ago

Hi,

I got different result between stable-diffusion-webui and onnx-web when I gave the same input.

It doesn't mean applying lora is not working on onnx web. It looks like applying LoRA is working but the generated image is different.

prompt: nikon RAW photo,8 k,Fujifilm XT3,masterpiece, best quality, 1girl,solo,realistic, photorealistic,ultra detailed, diamond stud earrings, long straight black hair, hazel eyes, serious expression, slender figure, wearing a black blazer and white blouse, standing against a city skyline at night iu1, <> Negative prompt: (worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ((grayscale)), skin spots, acnes, skin blemishes, age spot, glans,extra fingers,fewer fingers,strange fingers,bad hand, Steps: 40 Sampler: Euler CFG scale: 8 Seed: 1492306006 Size: 512x512 Model hash: fc2511737a Model: chilloutmix_NiPrunedFp32Fix Lora hashes: "iu_v35: 1818d7f2c01a"

I didn't use upscale and correction method on onnx-web as well.

How can I generate the same image when I use onnx-web using lora?

HoopyFreud commented 10 months ago

It looks like your image size is significantly smaller for onnx-web. Please post a screenshot of the webUI.

ssube commented 10 months ago

Using the same size and other parameters should help, but images may never be totally reproducible between tools, due to differences outside of our control. Images should be fairly reproducible within onnx-web, except for a few schedulers that make no attempt to be deterministic (DDIM with eta > 0 and Euler Ancestral, iirc).

The default settings for the ONNX runtime are not perfectly deterministic: https://github.com/microsoft/onnxruntime/issues/4611, https://github.com/microsoft/onnxruntime/issues/7642#issuecomment-842583213. I think there might be a flag to run ORT in a more deterministic mode, but I'm not able to find it in their docs at the moment. This is a sufficiently well-known issue that both the HuggingFace and Civitai docs mention it as well: https://huggingface.co/docs/diffusers/using-diffusers/reproducibility, https://github.com/civitai/civitai/wiki/Image-Reproduction.

All of that said, LoRAs for ONNX use a completely custom implementation, so if you think there are issues with that math, please let me know. It's entirely possible that I have not implemented some operator correctly.

ssube commented 9 months ago

Were you able to produce a similar image when using the same parameters, or is this still an issue?

newgrit1004 commented 8 months ago

I think the issue came from the different prompt weighting between diffusers and stable-difusion webui. I got the similar issue on https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT