kabachuha / sd-webui-text2video

Auto1111 extension implementing text2video diffusion models (like ModelScope or VideoCrafter) using only Auto1111 webui dependencies
Other
1.28k stars 107 forks source link

Add inbuilt prompt weights support + hotkeys #54

Closed kabachuha closed 1 year ago

kabachuha commented 1 year ago

bacon and eggs https://user-images.githubusercontent.com/14872007/227776291-d98c06c0-9bdb-4297-b635-23a682502bf0.mp4

bacond and (eggs:1.5) https://user-images.githubusercontent.com/14872007/227776304-33658b93-847e-4142-9cc0-d57a60698b21.mp4

hithereai commented 1 year ago

Actual +/- hotkeys work, but getting an error upon generation.

Changed prompt field (rest is default): image

Error trace:


latents torch.Size([1, 4, 24, 32, 32]) tensor(-0.0034, device='cuda:0') tensor(1.0030, device='cuda:0')
Traceback (most recent call last):
  File "D:\D-SD\AUTO\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope-text2vid.py", line 148, in process
    samples, _ = pipe.infer(prompt, n_prompt, steps, frames, seed, cfg_scale,
  File "D:\D-SD\AUTO\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\t2v_pipeline.py", line 253, in infer
    x0 = self.diffusion.ddim_sample_loop(
  File "D:\D-SD\AUTO\stable-diffusion-webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
TypeError: GaussianDiffusion.ddim_sample_loop() got an unexpected keyword argument 'c'
Exception occurred: GaussianDiffusion.ddim_sample_loop() got an unexpected keyword argument 'c'```
kabachuha commented 1 year ago

should have fixed the problem now, test it again

Please, :)

hithereai commented 1 year ago

EDIT: Fixed. Testing again.

Traceback (most recent call last):
  File "D:\D-SD\AUTO\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope-text2vid.py", line 148, in process
    samples, _ = pipe.infer(prompt, n_prompt, steps, frames, seed, cfg_scale,
  File "D:\D-SD\AUTO\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\t2v_pipeline.py", line 253, in infer
    x0 = self.diffusion.ddim_sample_loop(
  File "D:\D-SD\AUTO\stable-diffusion-webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "D:\D-SD\AUTO\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\t2v_model.py", line 1518, in ddim_sample_loop
    conds_list, tensor = prompt_parser.reconstruct_multicond_batch(c, i)
NameError: name 'prompt_parser' is not defined
Exception occurred: name 'prompt_parser' is not defined
hithereai commented 1 year ago

Now getting this:


latents torch.Size([1, 4, 24, 32, 32]) tensor(-0.0005, device='cuda:0') tensor(0.9996, device='cuda:0')
DDIM sampling:   0%|                                                                            | 0/31 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "D:\D-SD\AUTO\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope-text2vid.py", line 148, in process
    samples, _ = pipe.infer(prompt, n_prompt, steps, frames, seed, cfg_scale,
  File "D:\D-SD\AUTO\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\t2v_pipeline.py", line 253, in infer
    x0 = self.diffusion.ddim_sample_loop(
  File "D:\D-SD\AUTO\stable-diffusion-webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "D:\D-SD\AUTO\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\t2v_model.py", line 1527, in ddim_sample_loop
    uc.unsqueeze(0).repeat(num_sample, 1, 1)
RuntimeError: Number of dimensions of repeat dims can not be smaller than number of dimensions of tensor
Exception occurred: Number of dimensions of repeat dims can not be smaller than number of dimensions of tensor```
hithereai commented 1 year ago
Traceback (most recent call last):
  File "D:\D-SD\AUTO\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope-text2vid.py", line 148, in process
    samples, _ = pipe.infer(prompt, n_prompt, steps, frames, seed, cfg_scale,
  File "D:\D-SD\AUTO\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\t2v_pipeline.py", line 253, in infer
    x0 = self.diffusion.ddim_sample_loop(
  File "D:\D-SD\AUTO\stable-diffusion-webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "D:\D-SD\AUTO\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\t2v_model.py", line 1518, in ddim_sample_loop
    conds_list, tensor = reconstruct_multicond_batch(num_sample*[c], i)
  File "D:\D-SD\AUTO\stable-diffusion-webui\modules\prompt_parser.py", line 229, in reconstruct_multicond_batch
    param = c.batch[0][0].schedules[0].cond
AttributeError: 'list' object has no attribute 'batch'
Exception occurred: 'list' object has no attribute 'batch'