search

deforum-art / sd-webui-deforum

Deforum extension for AUTOMATIC1111's Stable Diffusion webui
https://deforum.github.io
Other
2.69k stars 389 forks source link

[Bug]: Setting Strength to 0 and using same seed produces different outputs #532

Closed neural-loop closed 1 year ago

neural-loop commented 1 year ago

Have you read the latest version of the FAQ?

Is there an existing issue for this?

Are you using the latest version of the Deforum extension?

What happened?

Strength 0 with Static seed doesn't generate same output

Steps to reproduce the problem

  1. Set strength to 0
  2. Set Seed Iter N to 10
  3. Set Frames to 10
  4. Set FPS to 1
  5. Set starting seed to 100
  6. Give a single prompt
  7. Generate

What should have happened?

Given strength 0, the whole previous frame should be irrelevant.

So the question is to try to understand what is the reason that it is not generating consistent output given the same seed with strength at 0.

WebUI and Deforum extension Commit IDs

webui commit id - commit 22bcc7be428c94e9408f589966c2040187245d81 (HEAD -> master, origin/master, origin/HEAD) deforum exten commit id - fbe6eabf (Thu Apr 6 03:46:20 2023)

On which platform are you launching the webui with the extension?

Local PC setup (Linux)

Deforum settings

https://gist.github.com/neural-loop/28e2f9bbe2b9aa9fb3a9e739aa21cc36

Webui core settings

https://gist.github.com/neural-loop/7fe3995f5c7bb431651e2210962975c8

Console logs

################################################################
Install script for stable-diffusion + Web UI
Tested on Debian 11 (Bullseye)
################################################################

################################################################
Running on user
################################################################

################################################################
Repo already cloned, using it as install directory
################################################################

################################################################
Create and activate python venv
################################################################

################################################################
Launching launch.py...
################################################################
Python 3.10.7 (main, Mar 10 2023, 10:47:39) [GCC 12.2.0]
Commit hash: 22bcc7be428c94e9408f589966c2040187245d81
Installing requirements for Web UI

Launching Web UI with arguments: 
No module 'xformers'. Proceeding without it.
Loading weights [ad2a33c361] from /home/iguana/projects/python/stable-diffusion-webui/models/Stable-diffusion/v2-1_768-ema-pruned.ckpt
Creating model from config: /home/iguana/projects/python/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/configs/stable-diffusion/v2-inference-v.yaml
LatentDiffusion: Running in v-prediction mode
DiffusionWrapper has 865.91 M params.
Applying cross attention optimization (Doggettx).
Textual inversion embeddings loaded(0): 
Model loaded in 5.1s (load weights from disk: 1.8s, find config: 0.8s, create model: 0.1s, apply weights to model: 0.6s, apply half(): 0.5s, load VAE: 0.9s, move model to device: 0.4s).
*Deforum ControlNet support: enabled*
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 9.4s (import torch: 1.3s, import gradio: 0.8s, import ldm: 0.3s, other imports: 0.7s, load scripts: 0.7s, load SD checkpoint: 5.2s, create ui: 0.4s).
Deforum extension for auto1111 webui, v2.3b
Git commit: fbe6eabf (Thu Apr  6 03:46:20 2023)
Saving animation frames to:
/stable-diffusion-webui/outputs/img2img-images/Deforum_20230406192411
Animation frame: 0/10  
Seed: 100
Prompt: tiny cute swamp bunny, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus, by tomasz alen kopera
Not using an init image (doing pure txt2img)
╭─────┬───┬───────┬─────┬────┬────┬────╮
│Steps│CFG│Denoise│Angle│Zoom│Tr X│Tr Y│
├─────┼───┼───────┼─────┼────┼────┼────┤
│ 25  │7.0│ None  │  0  │ 1  │ 0  │ 0  │
╰─────┴───┴───────┴─────┴────┴────┴────╯
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:06<00:00,  3.65it/s]
Animation frame: 1/10  ███████████                                                                                                    | 25/250 [00:04<00:40,  5.50it/s]
Seed: 101
Prompt: tiny cute swamp bunny, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus, by tomasz alen kopera
╭─────┬───┬───────┬─────┬────┬────┬────╮
│Steps│CFG│Denoise│Angle│Zoom│Tr X│Tr Y│
├─────┼───┼───────┼─────┼────┼────┼────┤
│ 25  │7.0│   1   │  0  │ 1  │ 0  │ 0  │
╰─────┴───┴───────┴─────┴────┴────┴────╯
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:03<00:00,  6.89it/s]
Animation frame: 2/10  ██████████████████████▏                                                                                        | 50/250 [00:11<00:24,  8.13it/s]
Seed: 101
Prompt: tiny cute swamp bunny, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus, by tomasz alen kopera
╭─────┬───┬───────┬─────┬────┬────┬────╮
│Steps│CFG│Denoise│Angle│Zoom│Tr X│Tr Y│
├─────┼───┼───────┼─────┼────┼────┼────┤
│ 25  │7.0│   1   │  0  │ 1  │ 0  │ 0  │
╰─────┴───┴───────┴─────┴────┴────┴────╯
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:03<00:00,  8.13it/s]
Animation frame: 3/10  █████████████████████████████████▎                                                                             | 75/250 [00:14<00:20,  8.46it/s]
Seed: 101
Prompt: tiny cute swamp bunny, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus, by tomasz alen kopera
╭─────┬───┬───────┬─────┬────┬────┬────╮
│Steps│CFG│Denoise│Angle│Zoom│Tr X│Tr Y│
├─────┼───┼───────┼─────┼────┼────┼────┤
│ 25  │7.0│   1   │  0  │ 1  │ 0  │ 0  │
╰─────┴───┴───────┴─────┴────┴────┴────╯
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:02<00:00,  8.63it/s]
Animation frame: 4/10  ████████████████████████████████████████████                                                                  | 100/250 [00:18<00:17,  8.65it/s]
Seed: 101
Prompt: tiny cute swamp bunny, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus, by tomasz alen kopera
╭─────┬───┬───────┬─────┬────┬────┬────╮
│Steps│CFG│Denoise│Angle│Zoom│Tr X│Tr Y│
├─────┼───┼───────┼─────┼────┼────┼────┤
│ 25  │7.0│   1   │  0  │ 1  │ 0  │ 0  │
╰─────┴───┴───────┴─────┴────┴────┴────╯
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:04<00:00,  5.42it/s]
Animation frame: 5/10  ███████████████████████████████████████████████████████                                                       | 125/250 [00:23<00:22,  5.46it/s]
Seed: 101
Prompt: tiny cute swamp bunny, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus, by tomasz alen kopera
╭─────┬───┬───────┬─────┬────┬────┬────╮
│Steps│CFG│Denoise│Angle│Zoom│Tr X│Tr Y│
├─────┼───┼───────┼─────┼────┼────┼────┤
│ 25  │7.0│   1   │  0  │ 1  │ 0  │ 0  │
╰─────┴───┴───────┴─────┴────┴────┴────╯
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:04<00:00,  5.24it/s]
Animation frame: 6/10  ██████████████████████████████████████████████████████████████████                                            | 150/250 [00:28<00:19,  5.19it/s]
Seed: 101
Prompt: tiny cute swamp bunny, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus, by tomasz alen kopera
╭─────┬───┬───────┬─────┬────┬────┬────╮
│Steps│CFG│Denoise│Angle│Zoom│Tr X│Tr Y│
├─────┼───┼───────┼─────┼────┼────┼────┤
│ 25  │7.0│   1   │  0  │ 1  │ 0  │ 0  │
╰─────┴───┴───────┴─────┴────┴────┴────╯
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:04<00:00,  5.55it/s]
Animation frame: 7/10  █████████████████████████████████████████████████████████████████████████████                                 | 175/250 [00:33<00:12,  6.02it/s]
Seed: 101
Prompt: tiny cute swamp bunny, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus, by tomasz alen kopera
╭─────┬───┬───────┬─────┬────┬────┬────╮
│Steps│CFG│Denoise│Angle│Zoom│Tr X│Tr Y│
├─────┼───┼───────┼─────┼────┼────┼────┤
│ 25  │7.0│   1   │  0  │ 1  │ 0  │ 0  │
╰─────┴───┴───────┴─────┴────┴────┴────╯
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:04<00:00,  5.48it/s]
Animation frame: 8/10  ████████████████████████████████████████████████████████████████████████████████████████                      | 200/250 [00:38<00:09,  5.37it/s]
Seed: 101
Prompt: tiny cute swamp bunny, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus, by tomasz alen kopera
╭─────┬───┬───────┬─────┬────┬────┬────╮
│Steps│CFG│Denoise│Angle│Zoom│Tr X│Tr Y│
├─────┼───┼───────┼─────┼────┼────┼────┤
│ 25  │7.0│   1   │  0  │ 1  │ 0  │ 0  │
╰─────┴───┴───────┴─────┴────┴────┴────╯
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:04<00:00,  5.43it/s]
Animation frame: 9/10  ███████████████████████████████████████████████████████████████████████████████████████████████████           | 225/250 [00:43<00:04,  5.39it/s]
Seed: 101
Prompt: tiny cute swamp bunny, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus, by tomasz alen kopera
╭─────┬───┬───────┬─────┬────┬────┬────╮
│Steps│CFG│Denoise│Angle│Zoom│Tr X│Tr Y│
├─────┼───┼───────┼─────┼────┼────┼────┤
│ 25  │7.0│   1   │  0  │ 1  │ 0  │ 0  │
╰─────┴───┴───────┴─────┴────┴────┴────╯
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:04<00:00,  5.39it/s]
Deforum progress: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 250/250 [00:48<00:00,  5.15it/s]
Got a request to stitch frames to video using FFmpeg.
Frames:
/stable-diffusion-webui/outputs/img2img-images/Deforum_20230406192411/20230406192411_%09d.png
To Video:
/stable-diffusion-webui/outputs/img2img-images/Deforum_20230406192411/20230406192411.mp4
Stitching *video*...
Video stitching done in 0.20 seconds!

Additional information

image

It seems that each iteration is coming with a loss in quality, or an interesting correlation in that the file size is decreasing.

Denoise shows 1 in output but in my UI I think I've disabled it - though I don't think it should have an effect

https://user-images.githubusercontent.com/654993/230517191-a2ca5039-3ced-4d7d-985a-72fff50cdb87.mp4

neural-loop commented 1 year ago

Fixed seed produces same issue (but stays on 100 seed)

hithereai commented 1 year ago

First, what are the asterisks around the seed in your setting file? I hope that only to get our attention, as it's not a valid JSON like that.

And this is not a bug in deform, it's just how you setup your run. 🔢

You set seed to 100, but seed behaviour to Iter, which means that for each frame seed will be seed+1. If you want a static seed for the whole animation you'd have to set seed behaviour to Fixed.

hithereai commented 1 year ago

Fixed seed produces same issue (but stays on 100 seed)

What do you want to achieve? Let's look at this that way. You want to generate the exact same frame the entire video? what's the point?