search

MooreThreads / Moore-AnimateAnyone

Character Animation (AnimateAnyone, Face Reenactment)
Apache License 2.0
3.14k stars 245 forks source link

SDXL support #88

Open gsrujana opened 8 months ago

gsrujana commented 8 months ago

Hello, Is it possible to use this with SDXL instead of SD1.5? Also, for training this model with SDXL base model.

matbee-eth commented 5 months ago

From some research, the reason its not supported on SDXL is due to the "sd-image-variations" model was only finetuned on sd 1.5. I believe there's been an SDXL (or SD 2.1) initiative to achieve something similar. Will take a look and check back.

Stability AI themselves released Unclip for SD 2.1 (https://huggingface.co/stabilityai/stable-diffusion-2-1-unclip), this is effectively the same as sd-image-variations

SDXL currently has an open issue requesting Unclip for SDXL (https://github.com/Stability-AI/generative-models/issues/22)

It looks like SDXL-refiner is already an image-to-image variation model, so it may very well work