search

continue-revolution / sd-webui-animatediff

AnimateDiff for AUTOMATIC1111 Stable Diffusion WebUI
Other
3.11k stars 258 forks source link

[Bug]: #340

Closed designrans closed 11 months ago

designrans commented 11 months ago

Is there an existing issue for this?

Have you read FAQ on README?

What happened?

After using the lcm sampler and setting up the sampling deployment and cfg, the AnimateDiff plug-in is enabled to produce a gray animation picture.

1700722116290

Steps to reproduce the problem

  1. Go to ....
  2. Press ....

  3. 00000-3325973771 ...

What should have happened?

Normal color

Commit where the problem happens

webui: 1.61 extension:

What browsers do you use to access the UI ?

No response

Command Line Arguments

@echo off

set PYTHON=
set GIT=
set VENV_DIR=
set COMMANDLINE_ARGS=

call webui.bat

Console logs

Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
Version: v1.6.0-2-g4afaaf8a0
Commit hash: 4afaaf8a020c1df457bcf7250cb1c7f609699fa7
current transparent-background 1.2.9
is_installed check for tensorflow-cpu failed as 'spec is None'
Installing requirements for easyphoto-webui
Installing requirements for tensorflow
WARNING: Ignoring invalid distribution -illow (c:\sd-webui-aki-v4.4\python\lib\site-packages)
Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Looking in links: https://mirror.sjtu.edu.cn/pytorch-wheels/torch_stable.html
Requirement already satisfied: tensorflow-cpu in c:\sd-webui-aki-v4.4\python\lib\site-packages (2.14.0)
Requirement already satisfied: tensorflow-intel==2.14.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-cpu) (2.14.0)
Requirement already satisfied: absl-py>=1.0.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (1.4.0)
Requirement already satisfied: astunparse>=1.6.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (1.6.3)
Requirement already satisfied: flatbuffers>=23.5.26 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (23.5.26)
Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (0.5.4)
Requirement already satisfied: google-pasta>=0.1.1 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (0.2.0)
Requirement already satisfied: h5py>=2.9.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (3.10.0)
Requirement already satisfied: libclang>=13.0.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (16.0.6)
Requirement already satisfied: ml-dtypes==0.2.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (0.2.0)
Requirement already satisfied: numpy>=1.23.5 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (1.23.5)
Requirement already satisfied: opt-einsum>=2.3.2 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (3.3.0)
Requirement already satisfied: packaging in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (23.1)
Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (3.20.3)
Requirement already satisfied: setuptools in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (65.5.0)
Requirement already satisfied: six>=1.12.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (1.16.0)
Requirement already satisfied: termcolor>=1.1.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (2.3.0)
Requirement already satisfied: typing-extensions>=3.6.6 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (4.8.0)
Requirement already satisfied: wrapt<1.15,>=1.11.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (1.14.1)
Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (0.31.0)
Requirement already satisfied: grpcio<2.0,>=1.24.3 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (1.54.0)
Requirement already satisfied: tensorboard<2.15,>=2.14 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (2.14.1)
Requirement already satisfied: tensorflow-estimator<2.15,>=2.14.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (2.14.0)
Requirement already satisfied: keras<2.15,>=2.14.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorflow-intel==2.14.0->tensorflow-cpu) (2.14.0)
Requirement already satisfied: wheel<1.0,>=0.23.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from astunparse>=1.6.0->tensorflow-intel==2.14.0->tensorflow-cpu) (0.40.0)
Requirement already satisfied: google-auth<3,>=1.6.3 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (2.17.3)
Requirement already satisfied: google-auth-oauthlib<1.1,>=0.5 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (1.0.0)
Requirement already satisfied: markdown>=2.6.8 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (3.4.3)
Requirement already satisfied: requests<3,>=2.21.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (2.25.1)
Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (0.7.0)
Requirement already satisfied: werkzeug>=1.0.1 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (2.3.3)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (5.3.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (0.3.0)
Requirement already satisfied: rsa<5,>=3.1.4 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (4.9)
Requirement already satisfied: requests-oauthlib>=0.7.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from google-auth-oauthlib<1.1,>=0.5->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (1.3.1)
Requirement already satisfied: chardet<5,>=3.0.2 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from requests<3,>=2.21.0->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (4.0.0)
Requirement already satisfied: idna<3,>=2.5 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from requests<3,>=2.21.0->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (2.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from requests<3,>=2.21.0->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (1.26.18)
Requirement already satisfied: certifi>=2017.4.17 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from requests<3,>=2.21.0->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (2023.5.7)
Requirement already satisfied: MarkupSafe>=2.1.1 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from werkzeug>=1.0.1->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (2.1.2)
Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (0.5.0)
Requirement already satisfied: oauthlib>=3.0.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<1.1,>=0.5->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.0->tensorflow-cpu) (3.2.2)
WARNING: Ignoring invalid distribution -illow (c:\sd-webui-aki-v4.4\python\lib\site-packages)
Installing requirements for easyphoto-webui
Installing requirements for invisible-watermark
WARNING: Ignoring invalid distribution -illow (c:\sd-webui-aki-v4.4\python\lib\site-packages)
Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Looking in links: https://mirror.sjtu.edu.cn/pytorch-wheels/torch_stable.html
Requirement already satisfied: invisible-watermark in c:\sd-webui-aki-v4.4\python\lib\site-packages (0.2.0)
Requirement already satisfied: Pillow>=6.0.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from invisible-watermark) (9.5.0)
Requirement already satisfied: PyWavelets>=1.1.1 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from invisible-watermark) (1.4.1)
Requirement already satisfied: numpy>=1.17.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from invisible-watermark) (1.23.5)
Requirement already satisfied: opencv-python>=4.1.0.25 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from invisible-watermark) (4.8.0.76)
Requirement already satisfied: torch in c:\sd-webui-aki-v4.4\python\lib\site-packages (from invisible-watermark) (2.1.0+cu121)
Requirement already satisfied: filelock in c:\sd-webui-aki-v4.4\python\lib\site-packages (from torch->invisible-watermark) (3.12.0)
Requirement already satisfied: typing-extensions in c:\sd-webui-aki-v4.4\python\lib\site-packages (from torch->invisible-watermark) (4.8.0)
Requirement already satisfied: sympy in c:\sd-webui-aki-v4.4\python\lib\site-packages (from torch->invisible-watermark) (1.12)
Requirement already satisfied: networkx in c:\sd-webui-aki-v4.4\python\lib\site-packages (from torch->invisible-watermark) (3.1)
Requirement already satisfied: jinja2 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from torch->invisible-watermark) (3.1.2)
Requirement already satisfied: fsspec in c:\sd-webui-aki-v4.4\python\lib\site-packages (from torch->invisible-watermark) (2023.9.2)
Requirement already satisfied: MarkupSafe>=2.0 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from jinja2->torch->invisible-watermark) (2.1.2)
Requirement already satisfied: mpmath>=0.19 in c:\sd-webui-aki-v4.4\python\lib\site-packages (from sympy->torch->invisible-watermark) (1.3.0)
WARNING: Ignoring invalid distribution -illow (c:\sd-webui-aki-v4.4\python\lib\site-packages)
ReActor preheating... Device: CUDA
loading WD14-tagger reqs from C:\sd-webui-aki-v4.4\extensions\stable-diffusion-webui-wd14-tagger\requirements.txt
Checking WD14-tagger requirements.
WARNING: Ignoring invalid distribution -illow (c:\sd-webui-aki-v4.4\python\lib\site-packages)
WARNING: Ignoring invalid distribution -illow (c:\sd-webui-aki-v4.4\python\lib\site-packages)
Launching Web UI with arguments: --theme dark --xformers --xformers-flash-attention --autolaunch --disable-safe-unpickle
Civitai Helper: Get Custom Model Folder
Tag Autocomplete: Could not locate model-keyword extension, Lora trigger word completion will be limited to those added through the extra networks menu.
[-] ADetailer initialized. version: 23.11.1, num models: 12
2023-11-23 14:40:00,503 - modelscope - INFO - PyTorch version 2.1.0+cu121 Found.
2023-11-23 14:40:00,505 - modelscope - INFO - TensorFlow version 2.14.0 Found.
2023-11-23 14:40:00,505 - modelscope - INFO - Loading ast index from C:\sd-webui-aki-v4.4\.cache\modelscope\hub\ast_indexer
2023-11-23 14:40:00,640 - modelscope - INFO - Loading done! Current index file version is 1.9.3, with md5 a333b31aa69943719a3c3d689d114042 and a total number of 943 components indexed
2023-11-23 14:40:02,205 - ControlNet - INFO - ControlNet v1.1.419
ControlNet preprocessor location: C:\sd-webui-aki-v4.4\extensions\sd-webui-controlnet\annotator\downloads
2023-11-23 14:40:02,330 - ControlNet - INFO - ControlNet v1.1.419
[sd-webui-freeu] Controlnet support: *enabled*
sd-webui-prompt-all-in-one background API service started successfully.
14:40:03 - ReActor - STATUS - Running v0.5.0-b2
== WD14 tagger /gpu:0, uname_result(system='Windows', node='AT15269-HP', release='10', version='10.0.22621', machine='AMD64') ==
Loading weights [4199bcdd14] from C:\sd-webui-aki-v4.4\models\Stable-diffusion\2.5D\revAnimated_v122.safetensors
2023-11-23 14:40:03,925 - AnimateDiff - INFO - Injecting LCM to UI.
AnimateDiffScript init
AnimateDiffScript init
2023-11-23 14:40:04,129 - AnimateDiff - INFO - Hacking i2i-batch.
LightDiffusionFlow 绑定完成
Civitai Helper: Settings:
Civitai Helper: max_size_preview: True
Civitai Helper: skip_nsfw_preview: False
Civitai Helper: open_url_with_js: True
Civitai Helper: proxy: 
Creating model from config: C:\sd-webui-aki-v4.4\configs\v1-inference.yaml
*Deforum ControlNet support: enabled*
Running on local URL:  http://127.0.0.1:7860
Loading VAE weights specified in settings: C:\sd-webui-aki-v4.4\models\VAE\vae-ft-mse-840000-ema-pruned.safetensors

To create a public link, set `share=True` in `launch()`.
----------------- light_diffusion_flow api start------------------
[Lobe]: Initializing Lobe
Startup time: 37.6s (prepare environment: 18.9s, import torch: 6.2s, import gradio: 1.1s, setup paths: 0.7s, initialize shared: 0.2s, other imports: 0.6s, load scripts: 5.2s, create ui: 1.0s, gradio launch: 2.8s, app_started_callback: 0.6s).
Applying attention optimization: xformers... done.
Model loaded in 4.9s (load weights from disk: 0.7s, create model: 0.3s, apply weights to model: 2.8s, load VAE: 0.7s, load textual inversion embeddings: 0.2s, calculate empty prompt: 0.2s).
refresh_ui
Loading model SDXL\bluePencilXL_v050.safetensors [8c462a80f4] (2 out of 2)
Loading weights [8c462a80f4] from C:\sd-webui-aki-v4.4\models\Stable-diffusion\SDXL\bluePencilXL_v050.safetensors
Creating model from config: C:\sd-webui-aki-v4.4\repositories\generative-models\configs\inference\sd_xl_base.yaml
Applying attention optimization: xformers... done.
Model loaded in 10.8s (load weights from disk: 0.8s, create model: 0.6s, apply weights to model: 9.2s, calculate empty prompt: 0.1s).
==========================================================================================
A tensor with all NaNs was produced in VAE.
Web UI will now convert VAE into 32-bit float and retry.
To disable this behavior, disable the 'Automatically revert VAE to 32-bit floats' setting.
To always start with 32-bit VAE, use --no-half-vae commandline flag.
==========================================================================================
2023-11-23 14:42:58,435 - AnimateDiff - INFO - AnimateDiff process start.
2023-11-23 14:42:58,451 - AnimateDiff - INFO - Loading motion module mm_sdxl_v10_beta.ckpt from C:\sd-webui-aki-v4.4\extensions\sd-webui-animatediff\model\mm_sdxl_v10_beta.ckpt
2023-11-23 14:43:00,428 - AnimateDiff - INFO - Guessed mm_sdxl_v10_beta.ckpt architecture: MotionModuleType.AnimateDiffXL
2023-11-23 14:43:04,164 - AnimateDiff - WARNING - Missing keys <All keys matched successfully>
2023-11-23 14:43:04,591 - AnimateDiff - INFO - Injecting motion module mm_sdxl_v10_beta.ckpt into SDXL UNet input blocks.
2023-11-23 14:43:04,592 - AnimateDiff - INFO - Injecting motion module mm_sdxl_v10_beta.ckpt into SDXL UNet output blocks.
2023-11-23 14:43:04,592 - AnimateDiff - INFO - Setting DDIM alpha.
2023-11-23 14:43:04,661 - AnimateDiff - INFO - Injection finished.
2023-11-23 14:43:04,661 - AnimateDiff - INFO - Hacking CFGDenoiser forward function.
2023-11-23 14:43:04,661 - AnimateDiff - INFO - Hacking ControlNet.
2023-11-23 14:43:34,861 - AnimateDiff - INFO - Restoring ControlNet.
2023-11-23 14:43:34,861 - AnimateDiff - INFO - Restoring CFGDenoiser forward function.
2023-11-23 14:43:34,861 - AnimateDiff - INFO - Restoring DDIM alpha.
2023-11-23 14:43:34,861 - AnimateDiff - INFO - Removing motion module from SDXL UNet input blocks.
2023-11-23 14:43:34,861 - AnimateDiff - INFO - Removing motion module from SDXL UNet output blocks.
2023-11-23 14:43:34,861 - AnimateDiff - INFO - Removal finished.
2023-11-23 14:43:34,861 - AnimateDiff - INFO - Merging images into GIF.
2023-11-23 14:43:37,016 - AnimateDiff - WARNING - exiftool not found, required for infotext with optimized GIF palette, try: apt install libimage-exiftool-perl or https://exiftool.org/
2023-11-23 14:43:37,017 - AnimateDiff - INFO - Saving outputs\txt2img-images\AnimateDiff\2023-11-23\00000-3325973771.mp4
2023-11-23 14:43:37,150 - AnimateDiff - INFO - AnimateDiff process end.
Reusing loaded model 2.5D\revAnimated_v122.safetensors [4199bcdd14] to load SDXL\sd_xl_base_1.0_0.9vae.safetensors [31e35c80fc]
Loading weights [31e35c80fc] from C:\sd-webui-aki-v4.4\models\Stable-diffusion\SDXL\sd_xl_base_1.0_0.9vae.safetensors
Creating model from config: C:\sd-webui-aki-v4.4\repositories\generative-models\configs\inference\sd_xl_base.yaml
Applying attention optimization: xformers... done.
Model loaded in 11.2s (create model: 0.7s, apply weights to model: 10.0s, move model to device: 0.3s).
2023-11-23 14:45:39,940 - AnimateDiff - INFO - AnimateDiff process start.
2023-11-23 14:45:39,950 - AnimateDiff - INFO - Injecting motion module mm_sdxl_v10_beta.ckpt into SDXL UNet input blocks.
2023-11-23 14:45:39,951 - AnimateDiff - INFO - Injecting motion module mm_sdxl_v10_beta.ckpt into SDXL UNet output blocks.
2023-11-23 14:45:39,951 - AnimateDiff - INFO - Setting DDIM alpha.
2023-11-23 14:45:39,953 - AnimateDiff - INFO - Injection finished.
2023-11-23 14:45:39,953 - AnimateDiff - INFO - Hacking CFGDenoiser forward function.
2023-11-23 14:45:39,953 - AnimateDiff - INFO - Hacking ControlNet.
2023-11-23 14:46:16,888 - AnimateDiff - INFO - Restoring ControlNet.
2023-11-23 14:46:16,888 - AnimateDiff - INFO - Restoring CFGDenoiser forward function.
2023-11-23 14:46:16,888 - AnimateDiff - INFO - Restoring DDIM alpha.
2023-11-23 14:46:16,888 - AnimateDiff - INFO - Removing motion module from SDXL UNet input blocks.
2023-11-23 14:46:16,889 - AnimateDiff - INFO - Removing motion module from SDXL UNet output blocks.
2023-11-23 14:46:16,889 - AnimateDiff - INFO - Removal finished.
2023-11-23 14:46:16,889 - AnimateDiff - INFO - Merging images into GIF.
2023-11-23 14:46:19,369 - AnimateDiff - WARNING - exiftool not found, required for infotext with optimized GIF palette, try: apt install libimage-exiftool-perl or https://exiftool.org/
2023-11-23 14:46:19,371 - AnimateDiff - INFO - Saving outputs\txt2img-images\AnimateDiff\2023-11-23\00001-1032861924.mp4
2023-11-23 14:46:19,592 - AnimateDiff - INFO - AnimateDiff process end.

Additional information

No response

continue-revolution commented 11 months ago

unfortunately, xl model is not having a good performance for the time being. please wait patiently when researchers from Shanghai AI Lab are training a better model.