[Feature Request]: SDXL refinement stage interrupt

[ghost]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

Would like the interrupt button to by-pass the refinement model stage of the SDXL generation. It always seem to run it. Would like the refinement stage to be interruptible.

Proposed workflow

1. Go to ....
2. Press ....
3. ...

Additional information

No response

[w-e-w]

but interrupt dose work

https://github.com/AUTOMATIC1111/stable-diffusion-webui/assets/40751091/0596e77b-4cc8-4861-a318-53d2d4753c61

I set refine to srop at 50% and test if I can interrupt before and after refine starts and it does work

[dhwz]

@tridasha maybe I misunderstood it but refiner is only applied if you've opened the refiner accordion

[ghost]

Hi, If I generate 4 1024x1024 images against a simple prompt like "Woman in park" with an SD_XL model. It starts processing, with progress ticking over at 1%... 18% etc (this takes 5 seconds or so to get to 18% - 3080 i7-12700K, 32GB M.2s ) . I interrupt it at 18%, it stops generating and no more progress reported, but I then have to wait 34 seconds before the interrupt button turns back to a generate button. I had assumed this was the refinement process used by SDXL (I understand it does a generating with one model, then a second stage using the refinement model to clean the mess up). Because I get that same delay at the end of every picture, with no progress being reported, and no ability to interrupt when it does this.

[w-e-w]

I assume that you're on the current dev branch?

[ghost]

Me, I'm just using what ever git pull got about a day ago. So master.

[w-e-w]

but we don't support refine on master branch yet, only in dev.......................................

[dhwz]

Hi, If I generate 4 1024x1024 images against a simple prompt like "Woman in park" with an SD_XL model. It starts processing, with progress ticking over at 1%... 18% etc (this takes 5 seconds or so to get to 18% - 3080 i7-12700K, 32GB M.2s ) . I interrupt it at 18%, it stops generating and no more progress reported, but I then have to wait 34 seconds before the interrupt button turns back to a generate button. I had assumed this was the refinement process used by SDXL (I understand it does a generating with one model, then a second stage using the refinement model to clean the mess up). Because I get that same delay at the end of every picture, with no progress being reported, and no ability to interrupt when it does this.

No what you're talking about is just the image decoding final step. That has nothing todo with refiner. On dev branch there is an option to speedup that if you press interrupt.

[ghost]

Ah okay, cool. some times its 34 seconds some times its 10. SD 1.5 doing the same 1024x1024 doesn't have the same lag when interrupt is pressed, which is why I thought it might be the refiner.

[dhwz]

Are you already using the fp16 VAE? If not you should probably try as the default embedded one is fp32. Which is slower, needs more VRAM

https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/tree/main

[w-e-w]

wait... I thought we cast the VAE down to fp16

[dhwz]

wait... I thought we cast the VAE down to fp16

Nope, I think you mean "Automatically revert VAE to 32-bit floats

(triggers when a tensor with NaNs is produced in VAE; disabling the option in this case will result in a black square image)"

But thats still slower than the fp16 fixed VAE