[Bug]: generates only two different image in a gif , no animation

neozbr commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

Have you read FAQ on README?

[X] I have updated WebUI and this extension to the latest version

What happened?

theres no errors I think..except this warning:

2023-08-03 14:29:26,522 - AnimateDiff - WARNING - Missing keys 2023-08-03 14:29:27,045 - AnimateDiff - INFO - Hacking GroupNorm32 forward function. 2023-08-03 14:29:27,045 - AnimateDiff - INFO - Injecting motion module mm_sd_v15.ckpt into SD1.5 UNet input blocks. 2023-08-03 14:29:27,046 - AnimateDiff - INFO - Injecting motion module mm_sd_v15.ckpt into SD1.5 UNet output blocks. 2023-08-03 14:29:27,046 - AnimateDiff - INFO - Injection finished. 100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [03:02<00:00, 9.15s/it] 2023-08-03 14:32:47,567 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet input blocks.51<00:00, 9.00s/it] 2023-08-03 14:32:47,568 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet output blocks. 2023-08-03 14:32:47,568 - AnimateDiff - INFO - Restoring GroupNorm32 forward function. 2023-08-03 14:32:47,569 - AnimateDiff - INFO - Removal finished. 2023-08-03 14:32:47,569 - AnimateDiff - INFO - Merging images into GIF. 2023-08-03 14:32:49,315 - AnimateDiff - INFO - AnimateDiff process end.

but when the process ends it ends with two different still gifs in the results are in the link , i've tried two times closed and reopone web-ui, nothing

link to the gifs: https://imgur.com/a/cZJd4Ll

prompt used in the second image:

masterpiece, best quality, 1girl, solo, cherry blossoms, hanami, pink flower, white flower, spring season, wisteria, petals, flower, plum blossoms, outdoors, falling petals, white hair, black eyes, Negative prompt: canvas frame, (high contrast:1.2), (over saturated:1.2), (glossy:1.1), cartoon, 3d, ((disfigured)), ((bad art)), ((b&w)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))), extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck))), Photoshop, video game, ugly, tiling, poorly drawn hands, 3d render Steps: 10, Sampler: Euler a, CFG scale: 7, Seed: 3156615935, Size: 512x512, Model hash: f0407eaf51, Model: colorful_v21, Version: 1.5.1

another details if needed:

python: 3.10.6 • torch: 2.0.0+cu118 • xformers: N/A • gradio: 3.32.0 • checkpoint: [f0407eaf51]

Steps to reproduce the problem

masterpiece, best quality, 1girl, solo, cherry blossoms, hanami, pink flower, white flower, spring season, wisteria, petals, flower, plum blossoms, outdoors, falling petals, white hair, black eyes, Negative prompt: canvas frame, (high contrast:1.2), (over saturated:1.2), (glossy:1.1), cartoon, 3d, ((disfigured)), ((bad art)), ((b&w)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))), extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck))), Photoshop, video game, ugly, tiling, poorly drawn hands, 3d render Steps: 10, Sampler: Euler a, CFG scale: 7, Seed: 3156615935, Size: 512x512, Model hash: f0407eaf51, Model: colorful_v21, Version: 1.5.1

What should have happened?

I have no ideia, the model are ok is 1.5, theres no errors but its not generating a animation but 2 still images in the end as a gif.

Commit where the problem happens

webui: 1.5.1 extension: 48fc19d1

What browsers do you use to access the UI ?

No response

Command Line Arguments

@echo off

set COMMANDLINE_ARGS= --skip-install  --opt-sdp-attention --opt-sdp-no-mem-attention --no-half-vae --medvram
set PYTHON=S:\\Python310\\python.exe
set GIT=
set VENV_DIR=
set --ui-debug-mode= true

call webui.bat

Console logs

2023-08-03 14:29:26,522 - AnimateDiff - WARNING - Missing keys <All keys matched successfully>
2023-08-03 14:29:27,045 - AnimateDiff - INFO - Hacking GroupNorm32 forward function.
2023-08-03 14:29:27,045 - AnimateDiff - INFO - Injecting motion module mm_sd_v15.ckpt into SD1.5 UNet input blocks.
2023-08-03 14:29:27,046 - AnimateDiff - INFO - Injecting motion module mm_sd_v15.ckpt into SD1.5 UNet output blocks.
2023-08-03 14:29:27,046 - AnimateDiff - INFO - Injection finished.
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [03:02<00:00,  9.15s/it]
2023-08-03 14:32:47,567 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet input blocks.51<00:00,  9.00s/it]
2023-08-03 14:32:47,568 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet output blocks.
2023-08-03 14:32:47,568 - AnimateDiff - INFO - Restoring GroupNorm32 forward function.
2023-08-03 14:32:47,569 - AnimateDiff - INFO - Removal finished.
2023-08-03 14:32:47,569 - AnimateDiff - INFO - Merging images into GIF.
2023-08-03 14:32:49,315 - AnimateDiff - INFO - AnimateDiff process end.
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [03:10<00:00,  9.51s/it]
2023-08-03 14:35:41,962 - AnimateDiff - INFO - AnimateDiff process start with video Max frames 16, FPS 8, duration 2.0,  motion module mm_sd_v15.ckpt.
2023-08-03 14:35:42,850 - AnimateDiff - INFO - Hacking GroupNorm32 forward function.
2023-08-03 14:35:42,851 - AnimateDiff - INFO - Injecting motion module mm_sd_v15.ckpt into SD1.5 UNet input blocks.
2023-08-03 14:35:42,851 - AnimateDiff - INFO - Injecting motion module mm_sd_v15.ckpt into SD1.5 UNet output blocks.
2023-08-03 14:35:42,851 - AnimateDiff - INFO - Injection finished.

Additional information

No response

bhushanap commented 1 year ago

Your workflow is generating all the 16 frames. It is just that there is not a lot of motion in the gif for some reason. (I am having same issue). But it's indeed working correctly (kinda).

The 2 gifs are generated due to different handling of the token limit by A1111 and AnimateDiff. A1111 allows for more than 75 token limit. AnimateDiff considers splits anything over 75. So reduce your prompt size to be under 75 tokens and you will see a single gif of 16 images (it might not have more motion though)

zappityzap commented 1 year ago

I've seen mm_sd_v15.ckpt results often have very little motion. Try mm_sd_v14.ckpt

Yaiol commented 12 months ago

I have exact same issue, whatever prompt, number of frames, model v14 v15 I use, it creates a gif with 2 images only. Could it be a memory limit ? I have 8G VRAM

zappityzap commented 12 months ago

Yes, it could be a memory limit. It must render all images in one batch. Try with fewer frames or lower resolution.

imbrium201 commented 11 months ago

Any idea if this is a to-be-fixed item? Works great when keeping below 75 token limit but after that i get 2 unrelated and poor quality images. Not a huge limitation for most of the models I use but can be an issue in models that require very specific prompts.

zappityzap commented 11 months ago

It has to do with the way A1111 handles longer token prompts. It'll take someone who understands it well enough, assuming it is even possible to fix.

continue-revolution / sd-webui-animatediff