nateraw / stable-diffusion-videos

Create 🔥 videos with Stable Diffusion by exploring the latent space and morphing between text prompts
Apache License 2.0
4.36k stars 417 forks source link

Allow use of a single prompt and seed. #101

Open ZeroCool940711 opened 1 year ago

ZeroCool940711 commented 1 year ago

As of now it seems like you need to use a list of prompts and seeds in order to be able to run stable diffusion videos, it would be nice to be able to use a single prompt and a single seed instead of multiple of them, if it's really needed to have more than one for the walk to work then a simple solution would be to loop the prompt and seed internally during walk.

nateraw commented 1 year ago

There's nothing to walk over if you have just one prompt with one seed. I think you would want 1 prompt with 2 seeds in this case.

you would do that like this:

prompt = 'some prompt'
seeds = [42, 1337]
prompts = [prompt] * len(seeds)

pipeline.walk(prompts, seeds, ...)
ZeroCool940711 commented 1 year ago

Oh, that makes sense, thanks for the reply, this is enough for me to use it that way but for others it would be nice if it's done automatically internally by the walk function, if a single prompt and a single seed is provided then a second random seed is added to the seed list with something like random.randint(0, 2**32 - 1) and the prompt is repeated the same amount of time as the length of the seeds list as in your example. This should increase the uses for the library as most people will probably try to use a single prompt and seed if they make a CLI or a normal GUI for stable diffusion and then try to add video generation with your code, making it so it accepts a list of prompts and seeds is not the first thought of some users using this on their code. Again, thanks for your time and thanks for your work on this code man, really appreciate it. Have a good day/night.

Atomic-Germ commented 1 year ago

If I'm understanding properly, you'd like it so that you can provide just a single prompt, and up to two seeds. It would then use a random seed if one is not provided at all, or use the seed provided plus a second seed calculated in some manner, or use two seeds as provided? Rather than wanting it to actually work directly with only a single seed and prompt, which would just give the same image over and over.