search

nateraw / stable-diffusion-videos

Create 🔥 videos with Stable Diffusion by exploring the latent space and morphing between text prompts
Apache License 2.0
4.4k stars 420 forks source link

Allow users to pass in a specified StableDiffusionPipeline #25

Closed minimaxir closed 1 year ago

minimaxir commented 2 years ago

In stable_diffusion_walk.py, the StableDiffusionPipeline is hardcoded at v1.4 and fp16.

https://github.com/nateraw/stable-diffusion-videos/blob/8a7b3b0533aaf1bcb4eb89714bf606746f60535b/stable_diffusion_videos/stable_diffusion_walk.py#L13-L18

The walk() function should take in a parametric StableDiffusionPipeline because a) there may be other/better StableDiffusion models in the future and b) if the text tokenizer of the Pipeline needs to be augmeneted in order to generate from textual inversion-trained embeddings.

nateraw commented 2 years ago

Yea this is probably wise. Honestly I think the best way is to have a separate pipeline class. So we'd init the class with the other pipeline and then the call function would be walk.

I heard there are new community pipelines for diffusers, and we might end up wanting to distribute the tools here as one of those. will look into it and update here

nateraw commented 2 years ago

28 proposes the pipeline solution.

btw, not against just doing what you suggest here and throwing the pipeline idea away, so feel free to weigh in :) I appreciate your contribution here.

nateraw commented 1 year ago

I ended up going with the pipeline option. So this is closed now, as this feature is effectively done.