Open nateraw opened 1 year ago
would love to see common numbers to use as well, I ran a couple of experiments and so far the videos don't come out as good as the examples.
Hi is it possible to show some examples on how to match the seeds.
I've managed to start the image part, but got errors when trying to create the video. So I have several folders on G drive with images, but no idea where to add those seeds to restart the video generation from those images.
This is the error I get when trying to generate the movie. Maybe there is something else I'm doing wrong.
`Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/gradio/routes.py", line 374, in run_predict
output = await app.get_blocks().process_api(
File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 1017, in process_api
result = await self.call_function(
File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 835, in call_function
prediction = await anyio.to_thread.run_sync(
File "/usr/local/lib/python3.8/dist-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "/usr/local/lib/python3.8/dist-packages/stable_diffusion_videos/app.py", line 91, in fn_videos
return self.pipeline.walk(**kwargs)
File "/usr/local/lib/python3.8/dist-packages/stable_diffusion_videos/stable_diffusion_pipeline.py", line 878, in walk
return make_video_pyav(
File "/usr/local/lib/python3.8/dist-packages/stable_diffusion_videos/stable_diffusion_pipeline.py", line 123, in make_video_pyav
frames = frames.permute(0, 2, 3, 1)
AttributeError: 'NoneType' object has no attribute 'permute'`
Examples. note to myself. add these to readme with accordion
from stable_diffusion_videos import StableDiffusionWalkPipeline
import torch
device = "mps" if torch.backends.mps.is_available() else "cuda" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if device == "cuda" else torch.float32
pipe = StableDiffusionWalkPipeline.from_pretrained(
"CompVis/stable-diffusion-v1-4",
torch_dtype=torch_dtype,
safety_checker=None
).to(device)
pipe.walk(
prompts=['a cat', 'a dog'],
seeds=[1234, 4321],
num_interpolation_steps=5,
num_inference_steps=30,
fps=5
)
from stable_diffusion_videos import StableDiffusionWalkPipeline
import torch
device = "mps" if torch.backends.mps.is_available() else "cuda" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if device == "cuda" else torch.float32
pipe = StableDiffusionWalkPipeline.from_pretrained(
"runwayml/stable-diffusion-v1-5",
torch_dtype=torch_dtype,
safety_checker=None,
).to(device)
pipe.walk(
prompts=['a cat', 'a dog'],
seeds=[1234, 4321],
num_interpolation_steps=5,
num_inference_steps=30,
fps=5
)
import torch
from stable_diffusion_videos import StableDiffusionWalkPipeline
from diffusers import DPMSolverMultistepScheduler
device = "mps" if torch.backends.mps.is_available() else "cuda" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if device == "cuda" else torch.float32
pipe = StableDiffusionWalkPipeline.from_pretrained(
"stabilityai/stable-diffusion-2-1",
torch_dtype=torch_dtype,
).to(device)
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)
pipe.walk(
prompts=['a cat', 'a dog'],
seeds=[1234, 4321],
num_interpolation_steps=5,
num_inference_steps=50,
fps=5,
)
Keep getting same questions about things, should just put these in readme:
Feel free to add/suggest more here if you are creeping on this issue and its still open. I'll make this PR asap