search

lllyasviel / stable-diffusion-webui-forge

GNU Affero General Public License v3.0
8.01k stars 781 forks source link

Hiresfix checkpoint only works on same architecture #2107

Open jordanjay29 opened 1 day ago

jordanjay29 commented 1 day ago

The recent hiresfix #2100 will only work across same architecture models (e.g. SD1.5 to 1.5, XL/Pony to XL/Pony, Flux to Flux) due to the change in requirements for VAE and possibly text encoders.

Could we have an additional field to select the TE/VAE like on top?

altoiddealer commented 20 hours ago

hr_forge_additional_modules sounds like a good idea

DenOfEquity commented 19 hours ago

You won't get good results with latent upscale between architectures, because the latents are not compatible. You can hiresfix any->any with a suitable upscale method, like Lanczos. I've tested 1.5 <-> XL (both ways) and 1.5 -> Flux. There is an issue if there is incompatibility between selected VAE/TE and the two models - i.e. trying to use a Flux model that doesn't include text encoders for the highres pass; because the necessary additional models are not included (unless also included in the first pass). Even 1.5 -> 1.5 can fail here, if the second pass model doesn't include a VAE.

I'm looking at adding highres additional modules. Currently working, but still some issues around infotext loading.

jordanjay29 commented 19 hours ago

I expected that the output would be trickier to get right, but even having the ability to do so in Forge could at least make for some good experiments. Sometimes even bad output is good output, if you know what I mean.

A hires additional modules field would resolve it well!

andy8992 commented 10 hours ago

Possible suggestion related to this, it may be nice if it could render the batch as one then hiresfix them as another. So render all 5 as SD 1.5 and then do them all as a hires batch with xl, to cut down on back and forth model loading for those of us with less resources.

I will say i'm happy to see this feature return