search

Doubiiu / DynamiCrafter

[ECCV 2024] DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors
Apache License 2.0
2.09k stars 165 forks source link

Can I change the SDT2I base model for different style #39

Closed qinziqiao closed 3 months ago

qinziqiao commented 3 months ago

Great work! I wonder if I can switch the base model during inference, such as some finetune models from SD1.5. Is it compatible?

Doubiiu commented 3 months ago

Thanks for your interest! I think it cannot be directly switched since other layers in this video model (e.g. temporal layers, image context projection module) are trained based on SDv2.1. It might require a lot of effort to switch that, which may involve further fine-tuning, thus I don't recommend that. You can fine-tune the spatial layers of this model based on your needs instead.