search

continue-revolution / sd-webui-animatediff

AnimateDiff for AUTOMATIC1111 Stable Diffusion WebUI
Other
3.08k stars 255 forks source link

[Bug]: Closed loop option not working properly #210

Closed AugmentedRealityCat closed 1 year ago

AugmentedRealityCat commented 1 year ago

Is there an existing issue for this?

Have you read FAQ on README?

What happened?

The results obtained are identical with or without activating the closed loop option.

My feeling is that the closed loop option is always ON but I am not sure.

Steps to reproduce the problem

  1. Select the base 1.5 model - in my case I loaded v1-5-pruned-emaonly.safetensors [6ce0161689]
  2. Add a simple prompt - for example "red ball rolling down the stairs"
  3. Activate the AnimateDiff extension with the enable AnimateDiff checkbox
  4. Select the mm_sd_v15_v2.ckpt AnimateDiff model
  5. Press Generate and wait for the process to complete.
  6. Press the "green recycling" icon next to the seed# parameter - this will apply the same seed you just used to the next animation. - for this test I used seed number 2798457700 for both animations.
  7. Activate the closed loop option using the corresponding checkbox.
  8. Press Generate again and wait for the process to complete.
  9. Go to the folder where the generated GIFs are located and compare them. In my tests, they are identical.

What should have happened?

The results should be different depending on wether or not you activate the closed loop option.

Commit where the problem happens

webui:
version: v1.6.0-127-g102b6617  •  python: 3.10.6  •  torch: 2.0.1+cu118  •  xformers: 0.0.20  •  gradio: 3.41.2  •  checkpoint: 6ce0161689

extension: 552f5059

What browsers do you use to access the UI ?

Mozilla Firefox

Command Line Arguments

--xformers --no-half-vae

Console logs

venv "C:\stable-diffusion-webui\venvxformers\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.6.0-127-g102b6617
Commit hash: 102b6617dacffdcc89c56badcaae6c5e83c3ff21
Launching Web UI with arguments: --xformers --no-half-vae
Civitai Helper: Get Custom Model Folder
Civitai Helper: Load setting from: C:\stable-diffusion-webui\extensions\Stable-Diffusion-Webui-Civitai-Helper\setting.json
Civitai Helper: No setting file, use default
2023-10-11 22:20:37,008 - ControlNet - INFO - ControlNet v1.1.410
ControlNet preprocessor location: C:\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads
2023-10-11 22:20:37,073 - ControlNet - INFO - ControlNet v1.1.410
[sd-webui-freeu] Controlnet support: *enabled*
[Vec. CC] Style Sheet Loaded...
Loading weights [6ce0161689] from C:\stable-diffusion-webui\models\Stable-diffusion\S_models_base\v1-5-pruned-emaonly.safetensors
Creating model from config: C:\stable-diffusion-webui\configs\v1-inference.yaml
*Deforum ControlNet support: enabled*
Loading VAE weights specified in settings: C:\stable-diffusion-webui\models\VAE\vae-ft-mse-840000-ema-pruned.ckpt
Applying attention optimization: xformers... done.
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 9.9s (prepare environment: 1.4s, import torch: 1.6s, import gradio: 0.5s, setup paths: 0.3s, initialize shared: 0.1s, other imports: 0.3s, list SD models: 0.6s, load scripts: 1.4s, create ui: 3.1s, gradio launch: 0.4s).
Model loaded in 5.6s (load weights from disk: 0.3s, create model: 0.3s, apply weights to model: 1.4s, apply half(): 0.6s, load VAE: 0.2s, load textual inversion embeddings: 2.1s, calculate empty prompt: 0.6s).
2023-10-11 22:33:41,460 - AnimateDiff - INFO - AnimateDiff process start.
2023-10-11 22:33:41,461 - AnimateDiff - INFO - You are using tested mm mm_sd_v15_v2.ckpt.
2023-10-11 22:33:41,461 - AnimateDiff - INFO - Loading motion module mm_sd_v15_v2.ckpt from C:\stable-diffusion-webui\extensions\sd-webui-animatediff\model\mm_sd_v15_v2.ckpt
2023-10-11 22:33:43,865 - AnimateDiff - WARNING - Missing keys <All keys matched successfully>
2023-10-11 22:33:44,381 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.ckpt into SD1.5 UNet middle block.
2023-10-11 22:33:44,381 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.ckpt into SD1.5 UNet input blocks.
2023-10-11 22:33:44,383 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.ckpt into SD1.5 UNet output blocks.
2023-10-11 22:33:44,383 - AnimateDiff - INFO - Setting DDIM alpha.
2023-10-11 22:33:44,387 - AnimateDiff - INFO - Injection finished.
2023-10-11 22:33:44,387 - AnimateDiff - INFO - Hacking lora to support motion lora
2023-10-11 22:33:44,387 - AnimateDiff - INFO - Hacking CFGDenoiser forward function.
2023-10-11 22:33:44,388 - AnimateDiff - INFO - Hacking ControlNet.
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:11<00:00,  1.71it/s]
2023-10-11 22:34:00,449 - AnimateDiff - INFO - Restoring ControlNet.███████████████████| 20/20 [00:10<00:00,  1.78it/s]
2023-10-11 22:34:00,449 - AnimateDiff - INFO - Restoring CFGDenoiser forward function.
2023-10-11 22:34:00,450 - AnimateDiff - INFO - Restoring hacked lora
2023-10-11 22:34:00,450 - AnimateDiff - INFO - Restoring DDIM alpha.
2023-10-11 22:34:00,450 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet input blocks.
2023-10-11 22:34:00,451 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet output blocks.
2023-10-11 22:34:00,452 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet middle block.
2023-10-11 22:34:00,452 - AnimateDiff - INFO - Removal finished.
2023-10-11 22:34:00,452 - AnimateDiff - INFO - Merging images into GIF.
2023-10-11 22:34:00,961 - AnimateDiff - INFO - AnimateDiff process end.
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:15<00:00,  1.28it/s]
2023-10-11 22:34:15,644 - AnimateDiff - INFO - AnimateDiff process start.██████████████| 20/20 [00:15<00:00,  1.78it/s]
2023-10-11 22:34:15,645 - AnimateDiff - INFO - You are using tested mm mm_sd_v15_v2.ckpt.
2023-10-11 22:34:15,656 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.ckpt into SD1.5 UNet middle block.
2023-10-11 22:34:15,657 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.ckpt into SD1.5 UNet input blocks.
2023-10-11 22:34:15,657 - AnimateDiff - INFO - Injecting motion module mm_sd_v15_v2.ckpt into SD1.5 UNet output blocks.
2023-10-11 22:34:15,657 - AnimateDiff - INFO - Setting DDIM alpha.
2023-10-11 22:34:15,660 - AnimateDiff - INFO - Injection finished.
2023-10-11 22:34:15,660 - AnimateDiff - INFO - Hacking lora to support motion lora
2023-10-11 22:34:15,661 - AnimateDiff - INFO - Hacking CFGDenoiser forward function.
2023-10-11 22:34:15,661 - AnimateDiff - INFO - Hacking ControlNet.
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:11<00:00,  1.78it/s]
2023-10-11 22:34:31,203 - AnimateDiff - INFO - Restoring ControlNet.███████████████████| 20/20 [00:10<00:00,  1.79it/s]
2023-10-11 22:34:31,203 - AnimateDiff - INFO - Restoring CFGDenoiser forward function.
2023-10-11 22:34:31,204 - AnimateDiff - INFO - Restoring hacked lora
2023-10-11 22:34:31,204 - AnimateDiff - INFO - Restoring DDIM alpha.
2023-10-11 22:34:31,205 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet input blocks.
2023-10-11 22:34:31,205 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet output blocks.
2023-10-11 22:34:31,206 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet middle block.
2023-10-11 22:34:31,206 - AnimateDiff - INFO - Removal finished.
2023-10-11 22:34:31,206 - AnimateDiff - INFO - Merging images into GIF.
2023-10-11 22:34:31,700 - AnimateDiff - INFO - AnimateDiff process end.
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:15<00:00,  1.30it/s]
T

Additional information

Let me know if there is anything else I could test and document to help you with this.

continue-revolution commented 1 year ago

That's true, it's always on. It's just a more-or-less problem. If you did check that checkbox, it will aggressively apply close loop. If you really do not want closed loop, you should set number of frame to be 0, and use batch size as your frame number. Close loop is not effective when you use batch size as your frame number.

I tried completely removing any close loop but the generated content is garbage. I have another idea to do this - I will implement another context generator, but this takes time.

AugmentedRealityCat commented 1 year ago

I tried completely removing any close loop but the generated content is garbage. I have another idea to do this - I will implement another context generator, but this takes time.

The looping behaviour by default is not a problem for me by the way, I was just trying to report what I thought was a bug to help you fix it. Take your time, I am sure there are much more important and interesting features you can work on !

I will try the alternative you propose with frame set to 0 and batch size set to the number of frames I want. EDIT: It did work perfectly. 👍

Thanks a lot for the detailed information ! I will close this for now as I do not consider this an issue.