search

LagPixelLOL / cog-sdxl

Inference SDXL with cog including multiple models in 1 instance support.
MIT License
5 stars 4 forks source link

[FR] Ability to use non-diffuser VAE files (possibly nonviable) #7

Closed Succubyss closed 2 weeks ago

Succubyss commented 2 weeks ago

In regards to this for example: https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/tree/main

Right now we have to use a diffusion_pytorch_model with config.json instead of just using the sdxl.vae.safetensors file.

Would it be possible to make it so we can use the sdxl.vae.safetensors file? Some models, such as the base model of Pony XL v6, only have a "normal" VAE file available on the CivitAI website. (I'm sure I can find a diffusion conversion somewhere, but it's just an example.)

This might be an upstream limitation you can't work around, but I just thought I'd ask.

LagPixelLOL commented 2 weeks ago

HuggingFace Diffusers doesn't have .from_single_file() for AutoencoderKL, it would be doable by converting when loading but it would need a rather large refactor of the current VAE loading logic, so I'm not planning to add this feature.

Succubyss commented 2 weeks ago

HuggingFace Diffusers doesn't have .from_single_file() for AutoencoderKL, it would be doable by converting when loading but it would need a rather large refactor of the current VAE loading logic, so I'm not planning to add this feature.

Understood. If you'd happen to have a convenient way to do this conversion from a single file to a diffuser in Python so that I can incorporate it into my deployment workflow, I'd appreciate it a lot if you shared it. If not, that's fine, just thought I'd ask.

LagPixelLOL commented 2 weeks ago

Welp I never had the need to use single file VAE, so I don't have a way of doing it, but I can try to see if it's easy to do.

LagPixelLOL commented 2 weeks ago

Some models, such as the base model of Pony XL v6, only have a "normal" VAE file available on the CivitAI website.

I just probed the weights and it's exactly the same weight as the SDXL VAE FP16 Fix one. The file structure is the same and it's only lacking the config file and the weight names are different. It shouldn't be too hard to make a script for it, it would just be very annoying to do since the names need to be renamed. And consider it's literally the same as the FP16 fix one, and from my experience basically all the models which provide "custom VAE" is just the same file as the FP16 fix, I'd say there's no point making a script for it.