[Bug]: Impossible to run the extension with API

Sadrahel commented 5 months ago

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

Have you read FAQ on README?

[X] I have updated WebUI and this extension to the latest version

What happened?

Hi Here is the error message I have when i want to run the animatediff extension using the API

Thanx for your help !!!

Steps to reproduce the problem & What should have happened?

1-lanch sd webui 2- execute API code 3- generate a video

Commit where the problem happens

webui: stable diffusion version: v1.9.3 • python: 3.10.11 • torch: 2.1.2+cu121 • xformers: 0.0.23.post1 • gradio: 3.41.2 • checkpoint: 6ce0161689 extension: animatediff

What browsers do you use to access the UI ?

Mozilla Firefox

Command Line Arguments

from datetime import datetime
import urllib.request
import base64
import json
import time
import os

webui_server_url = 'http://127.0.0.1:7861'

out_dir = 'api_out'
out_dir_t2i = os.path.join(out_dir, 'txt2img')
out_dir_i2i = os.path.join(out_dir, 'img2img')
os.makedirs(out_dir_t2i, exist_ok=True)
os.makedirs(out_dir_i2i, exist_ok=True)

def timestamp():
    return datetime.fromtimestamp(time.time()).strftime("%Y%m%d-%H%M%S")

def encode_file_to_base64(path):
    with open(path, 'rb') as file:
        return base64.b64encode(file.read()).decode('utf-8')

def decode_and_save_base64(base64_str, save_path):
    with open(save_path, "wb") as file:
        file.write(base64.b64decode(base64_str))

def call_api(api_endpoint, **payload):
    data = json.dumps(payload).encode('utf-8')
    request = urllib.request.Request(
        f'{webui_server_url}/{api_endpoint}',
        headers={'Content-Type': 'application/json'},
        data=data,
    )
    response = urllib.request.urlopen(request)
    return json.loads(response.read().decode('utf-8'))

def call_txt2img_api(**payload):
    response = call_api('sdapi/v1/txt2img', **payload)
    for index, image in enumerate(response.get('images')):
        save_path = os.path.join(out_dir_t2i, f'txt2img-{timestamp()}-{index}.png')
        decode_and_save_base64(image, save_path)

# def call_img2img_api(**payload):
#     response = call_api('sdapi/v1/img2img', **payload)
#     for index, image in enumerate(response.get('images')):
#         save_path = os.path.join(out_dir_i2i, f'img2img-{timestamp()}-{index}.png')
#         decode_and_save_base64(image, save_path)

if __name__ == '__main__':
    payload = {
        "prompt": "1girl",  # extra networks also in prompts
        "negative_prompt": "",
        "seed": -1,
        "steps": 20,
        "width": 512,
        "height": 512,
        "cfg_scale": 7,
        "sampler_name": "DPM++ 2M",
        "n_iter": 1,
        "batch_size": 1,
        'alwayson_scripts': {
            'AnimateDiff': {
                'args': [{
                    'model': 'mm_sd15_v3.safetensors',  # Motion module
                    'format': ['MP4'],  # Save format, 'GIF' | 'MP4' | 'PNG' | 'WEBP' | 'WEBM' | 'TXT' | 'Frame'
                    'enable': True,  # Enable AnimateDiff
                    'video_length': 24,  # Number of frames
                    'fps': 12,  # FPS
                    'loop_number': 0,  # Display loop number
                    'closed_loop': 'R+P',  # Closed loop, 'N' | 'R-P' | 'R+P' | 'A'
                    'batch_size': 16,  # Context batch size
                    'stride': 1,  # Stride
                    'overlap': -1,  # Overlap
                    'interp': 'Off',  # Frame interpolation, 'Off' | 'FILM'
                    'interp_x': 2,  # Interp X
                    'latent_power': 1,  # Latent power
                    'latent_scale': 32,  # Latent scale
                    'last_frame': None,  # Optional last frame
                    'latent_power_last': 1,  # Optional latent power for last frame
                    'latent_scale_last': 32,  # Optional latent scale for last frame
                    'request_id': ''
                    # Optional request id. If provided, outputs will have request id as filename suffix
                }
                ]
            }
        },

    }
    call_txt2img_api(**payload)

Console logs

*** Error running before_process: /home/guillaume/Téléchargements/Data/Packages/stable-diffusion-webui/extensions/sd-webui-animatediff/scripts/animatediff.py
    Traceback (most recent call last):
      File "/home/guillaume/Téléchargements/Data/Packages/stable-diffusion-webui/modules/scripts.py", line 817, in before_process
        script.before_process(p, *script_args)
      File "/home/guillaume/Téléchargements/Data/Packages/stable-diffusion-webui/extensions/sd-webui-animatediff/scripts/animatediff.py", line 65, in before_process
        params.set_p(p)
      File "/home/guillaume/Téléchargements/Data/Packages/stable-diffusion-webui/extensions/sd-webui-animatediff/scripts/animatediff_ui.py", line 180, in set_p
        extract_frames_from_video(self)
      File "/home/guillaume/Téléchargements/Data/Packages/stable-diffusion-webui/extensions/sd-webui-animatediff/scripts/animatediff_utils.py", line 108, in extract_frames_from_video
        assert params.video_source, "You need to specify cond hint for ControlNet."
    AssertionError: You need to specify cond hint for ControlNet.

---
  0%|          | 0/20 [00:00<?, ?it/s]*** Error executing callback cfg_denoiser_callback for /home/guillaume/Téléchargements/Data/Packages/stable-diffusion-webui/extensions/sd-webui-animatediff/scripts/animatediff.py
    Traceback (most recent call last):
      File "/home/guillaume/Téléchargements/Data/Packages/stable-diffusion-webui/modules/script_callbacks.py", line 333, in cfg_denoiser_callback
        c.callback(params)
      File "/home/guillaume/Téléchargements/Data/Packages/stable-diffusion-webui/extensions/sd-webui-animatediff/scripts/animatediff_infv2v.py", line 102, in animatediff_on_cfg_denoiser
        ad_params.text_cond = ad_params.prompt_scheduler.multi_cond(cfg_params.text_cond, prompt_closed_loop)
    AttributeError: 'NoneType' object has no attribute 'multi_cond'

---

  5%|▌         | 1/20 [00:00<00:14,  1.30it/s]*** Error executing callback cfg_denoiser_callback for /home/guillaume/Téléchargements/Data/Packages/stable-diffusion-webui/extensions/sd-webui-animatediff/scripts/animatediff.py
    Traceback (most recent call last):
      File "/home/guillaume/Téléchargements/Data/Packages/stable-diffusion-webui/modules/script_callbacks.py", line 333, in cfg_denoiser_callback
        c.callback(params)
      File "/home/guillaume/Téléchargements/Data/Packages/stable-diffusion-webui/extensions/sd-webui-animatediff/scripts/animatediff_infv2v.py", line 175, in animatediff_on_cfg_denoiser
        cfg_params.text_cond = ad_params.text_cond
    AttributeError: 'AnimateDiffProcess' object has no attribute 'text_cond'

Additional information

No response

Sadrahel commented 5 months ago

Hi, Help ;)

zappityzap commented 5 months ago

Thanks for the details. This is a bug, PR has been submitted. It's a fairly simple change to the code if you want to try it yourself. Developer is on vacation so this may not be merged soon.

linghu-cell commented 5 months ago

谢谢你的细节。这是一个错误，PR已提交。如果您想自己尝试，这是对代码的相当简单的更改。开发人员正在休假，因此可能不会很快合并。

您好，怎么修改？

zappityzap commented 5 months ago

The change has been merged so you can update the extension. If you want to see the details, look here: https://github.com/continue-revolution/sd-webui-animatediff/pull/525/files

continue-revolution / sd-webui-animatediff