[Bug]: Not working as expected

WeirdScienceX commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

Have you read FAQ on README?

[X] I have updated WebUI and this extension to the latest version

What happened?

Errors are thrown up during generation, results not as expected.

Steps to reproduce the problem

Run according to instructions, both txt2img and img2img

What should have happened?

Generate result as indicated

Commit where the problem happens

webui: f865d3e11647dfd6c7b2cdf90dde24680e58acd8 extension: 88a04c32

What browsers do you use to access the UI ?

No response

Command Line Arguments

None

Console logs

2023-07-23 20:08:16,895 - AnimateDiff - INFO - AnimateDiff process start with video Max frames 16, FPS 25, duration 0.64,  motion module mm_sd_v15.ckpt.
2023-07-23 20:08:16,895 - AnimateDiff - INFO - Loading motion module mm_sd_v15.ckpt from C:\AI\stable-diffusion-webui\extensions\sd-webui-animatediff\model\mm_sd_v15.ckpt
2023-07-23 20:08:20,731 - AnimateDiff - WARNING - Missing keys <All keys matched successfully>
2023-07-23 20:08:21,179 - AnimateDiff - INFO - Injecting motion module mm_sd_v15.ckpt into SD1.5 UNet input blocks.
2023-07-23 20:08:21,179 - AnimateDiff - INFO - Injecting motion module mm_sd_v15.ckpt into SD1.5 UNet output blocks.
2023-07-23 20:08:21,180 - AnimateDiff - INFO - Injection finished.
 25%|█████████████████████                                                               | 1/4 [00:09<00:29,  9.97s/it]Exception in callback H11Protocol.timeout_keep_alive_handler()                                    | 0/4 [00:00<?, ?it/s]
handle: <TimerHandle when=21760.078 H11Protocol.timeout_keep_alive_handler()>
Traceback (most recent call last):
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11\_state.py", line 249, in _fire_event_triggered_transitions
    new_state = EVENT_TRIGGERED_TRANSITIONS[role][state][event_type]
KeyError: <class 'h11._events.ConnectionClosed'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\d3xp2\AppData\Local\Programs\Python\Python310\lib\asyncio\events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 383, in timeout_keep_alive_handler
    self.conn.send(event)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 468, in send
    data_list = self.send_with_data_passthrough(event)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 493, in send_with_data_passthrough
    self._process_event(self.our_role, event)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 242, in _process_event
    self._cstate.process_event(role, type(event), server_switch_event)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11\_state.py", line 238, in process_event
    self._fire_event_triggered_transitions(role, event_type)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11\_state.py", line 251, in _fire_event_triggered_transitions
    raise LocalProtocolError(
h11._util.LocalProtocolError: can't handle event type ConnectionClosed when role=SERVER and state=SEND_RESPONSE
API error: POST: http://127.0.0.1:7860/api/predict {'error': 'LocalProtocolError', 'detail': '', 'body': '', 'errors': "Can't send data when our state is ERROR"}
                                                                                                                       ╭───────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────╮
│ C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py:162 in __call__                   │
│                                                                                                                      │
│ C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py:109 in __call__                     │
│                                                                                                                      │
│                                               ... 7 frames hidden ...                                                │
│                                                                                                                      │
│ C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py:468 in send                                   │
│                                                                                                                      │
│   467 │   │   """                                                                                                    │
│ ❱ 468 │   │   data_list = self.send_with_data_passthrough(event)                                                     │
│   469 │   │   if data_list is None:                                                                                  │
│                                                                                                                      │
│ ╭───────────────────────────────────────────────────── locals ─────────────────────────────────────────────────────╮ │
│ │ event = Response(status_code=200, headers=<Headers([(b'date', b'Sun, 23 Jul 2023 19:09:05 GMT'), (b'server',     │ │
│ │         b'uvicorn'), (b'content-length', b'22667'), (b'content-type', b'application/json'), (b'x-process-time',  │ │
│ │         b'7.2564')])>, http_version=b'1.1', reason=b'OK')                                                        │ │
│ │  self = <h11._connection.Connection object at 0x00000200E8A84E80>                                                │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                                      │
│ C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py:483 in send_with_data_passthrough             │
│                                                                                                                      │
│   482 │   │   if self.our_state is ERROR:                                                                            │
│ ❱ 483 │   │   │   raise LocalProtocolError("Can't send data when our state is ERROR")                                │
│   484 │   │   try:                                                                                                   │
│                                                                                                                      │
│ ╭───────────────────────────────────────────────────── locals ─────────────────────────────────────────────────────╮ │
│ │ event = Response(status_code=200, headers=<Headers([(b'date', b'Sun, 23 Jul 2023 19:09:05 GMT'), (b'server',     │ │
│ │         b'uvicorn'), (b'content-length', b'22667'), (b'content-type', b'application/json'), (b'x-process-time',  │ │
│ │         b'7.2564')])>, http_version=b'1.1', reason=b'OK')                                                        │ │
│ │  self = <h11._connection.Connection object at 0x00000200E8A84E80>                                                │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
LocalProtocolError: Can't send data when our state is ERROR
 50%|██████████████████████████████████████████                                          | 2/4 [00:21<00:21, 10.75s/it]ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 428, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\fastapi\applications.py", line 273, in __call__
    await super().__call__(scope, receive, send)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 109, in __call__
    await response(scope, receive, send)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 270, in __call__
    async with anyio.create_task_group() as task_group:
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 597, in __aexit__
    raise exceptions[0]
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 273, in wrap
    await func()
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 134, in stream_response
    return await super().stream_response(send)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 255, in stream_response
    await send(
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 159, in _send
    await send(message)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 512, in send
    output = self.conn.send(event)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 468, in send
    data_list = self.send_with_data_passthrough(event)
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 483, in send_with_data_passthrough
    raise LocalProtocolError("Can't send data when our state is ERROR")
h11._util.LocalProtocolError: Can't send data when our state is ERROR
Task exception was never retrieved
future: <Task finished name='dhvudlkwb2_590' coro=<Queue.process_events() done, defined at C:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\queueing.py:343> exception=ValueError('[<gradio.queueing.Event object at 0x00000200E8A878B0>] is not in list')>
Traceback (most recent call last):
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\queueing.py", line 370, in process_events
    while response.json.get("is_generating", False):
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 538, in json
    return self._json_response_data
AttributeError: 'AsyncRequest' object has no attribute '_json_response_data'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\queueing.py", line 432, in process_events
    self.active_jobs[self.active_jobs.index(events)] = None
ValueError: [<gradio.queueing.Event object at 0x00000200E8A878B0>] is not in list
100%|████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:43<00:00, 10.94s/it]
2023-07-23 20:09:39,536 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet input blocks.33<00:00,  9.21s/it]
2023-07-23 20:09:39,537 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet output blocks.
2023-07-23 20:09:39,538 - AnimateDiff - INFO - Removal finished.
2023-07-23 20:09:39,538 - AnimateDiff - INFO - Merging images into GIF.
2023-07-23 20:09:41,505 - AnimateDiff - INFO - AnimateDiff process end.
Total progress: 100%|████████████████████████████████████████████████████████████████████| 4/4 [00:40<00:00, 10.06s/it]
Total progress: 100%|████████████████████████████████████████████████████████████████████| 4/4 [00:40<00:00,  9.21s/it]

Additional information

No response

WeirdScienceX commented 1 year ago

287b30af

`100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:03<00:00, 6.08it/s] Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:02<00:00, 7.36it/s] 2023-07-24 18:49:14,716 - AnimateDiff - INFO - AnimateDiff process start with video Max frames 16, FPS 8, duration 2.0, motion module mm_sd_v15.ckpt. 2023-07-24 18:49:14,717 - AnimateDiff - INFO - Loading motion module mm_sd_v15.ckpt from C:\AI\stable-diffusion-webui\extensions\sd-webui-animatediff\model\mm_sd_v15.ckpt 2023-07-24 18:49:19,556 - AnimateDiff - WARNING - Missing keys 2023-07-24 18:49:20,007 - AnimateDiff - INFO - Hacking GroupNorm32 forward function. 2023-07-24 18:49:20,008 - AnimateDiff - INFO - Injecting motion module mm_sd_v15.ckpt into SD1.5 UNet input blocks. 2023-07-24 18:49:20,008 - AnimateDiff - INFO - Injecting motion module mm_sd_v15.ckpt into SD1.5 UNet output blocks. 2023-07-24 18:49:20,008 - AnimateDiff - INFO - Injection finished. 5%|████▏ | 1/20 [00:14<04:33, 14.41s/it]Exception in callback H11Protocol.timeout_keep_alive_handler() | 0/20 [00:00<?, ?it/s] handle: <TimerHandle when=14510.781 H11Protocol.timeout_keep_alive_handler()> Traceback (most recent call last): File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11_state.py", line 249, in _fire_event_triggered_transitions new_state = EVENT_TRIGGERED_TRANSITIONS[role][state][event_type] KeyError: <class 'h11._events.ConnectionClosed'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\d3xp2\AppData\Local\Programs\Python\Python310\lib\asyncio\events.py", line 80, in _run self._context.run(self._callback, *self._args) File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 383, in timeout_keep_alive_handler self.conn.send(event) File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11_connection.py", line 468, in send data_list = self.send_with_data_passthrough(event) File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11_connection.py", line 493, in send_with_data_passthrough self._process_event(self.our_role, event) File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11_connection.py", line 242, in _process_event self._cstate.process_event(role, type(event), server_switch_event) File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11_state.py", line 238, in process_event self._fire_event_triggered_transitions(role, event_type) File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11_state.py", line 251, in _fire_event_triggered_transitions raise LocalProtocolError( h11._util.LocalProtocolError: can't handle event type ConnectionClosed when role=SERVER and state=SEND_RESPONSE API error: POST: http://127.0.0.1:7860/api/predict {'error': 'LocalProtocolError', 'detail': '', 'body': '', 'errors': "Can't send data when our state is ERROR"} ╭───────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────╮ │ C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py:162 in call │ │ │ │ C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py:109 in call │ │ │ │ ... 7 frames hidden ... │ │ │ │ C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11_connection.py:468 in send │ │ │ │ 467 │ │ """ │ │ ❱ 468 │ │ data_list = self.send_with_data_passthrough(event) │ │ 469 │ │ if data_list is None: │ │ │ │ ╭───────────────────────────────────────────────────── locals ─────────────────────────────────────────────────────╮ │ │ │ event = Response(status_code=200, headers=<Headers([(b'date', b'Mon, 24 Jul 2023 17:49:19 GMT'), (b'server', │ │ │ │ b'uvicorn'), (b'content-length', b'22670'), (b'content-type', b'application/json'), (b'x-process-time', │ │ │ │ b'13.6494')])>, http_version=b'1.1', reason=b'OK') │ │ │ │ self = <h11._connection.Connection object at 0x000001B780C0E110> │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11_connection.py:483 in send_with_data_passthrough │ │ │ │ 482 │ │ if self.our_state is ERROR: │ │ ❱ 483 │ │ │ raise LocalProtocolError("Can't send data when our state is ERROR") │ │ 484 │ │ try: │ │ │ │ ╭───────────────────────────────────────────────────── locals ─────────────────────────────────────────────────────╮ │ │ │ event = Response(status_code=200, headers=<Headers([(b'date', b'Mon, 24 Jul 2023 17:49:19 GMT'), (b'server', │ │ │ │ b'uvicorn'), (b'content-length', b'22670'), (b'content-type', b'application/json'), (b'x-process-time', │ │ │ │ b'13.6494')])>, http_version=b'1.1', reason=b'OK') │ │ │ │ self = <h11._connection.Connection object at 0x000001B780C0E110> │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ LocalProtocolError: Can't send data when our state is ERROR ERROR: Exception in ASGI application Traceback (most recent call last): File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 428, in run_asgi result = await app( # type: ignore[func-returns-value] File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 78, in call return await self.app(scope, receive, send) File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\fastapi\applications.py", line 273, in call await super().call(scope, receive, send) File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\applications.py", line 122, in call await self.middleware_stack(scope, receive, send) File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 184, in call raise exc File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 162, in call await self.app(scope, receive, _send) File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 109, in call await response(scope, receive, send) File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 270, in call async with anyio.create_task_group() as task_group: File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 597, in aexit raise exceptions[0] File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 273, in wrap await func() File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 134, in stream_response return await super().stream_response(send) File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 255, in stream_response await send( File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 159, in _send await send(message) File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 512, in send output = self.conn.send(event) File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11_connection.py", line 468, in send data_list = self.send_with_data_passthrough(event) File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\h11_connection.py", line 483, in send_with_data_passthrough raise LocalProtocolError("Can't send data when our state is ERROR") h11._util.LocalProtocolError: Can't send data when our state is ERROR Task exception was never retrieved future: <Task finished name='r0o7mxv6xsp_786' coro=<Queue.process_events() done, defined at C:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\queueing.py:343> exception=ValueError('[<gradio.queueing.Event object at 0x000001B78074DC60>] is not in list')> Traceback (most recent call last): File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\queueing.py", line 370, in process_events while response.json.get("is_generating", False): File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 538, in json return self._json_response_data AttributeError: 'AsyncRequest' object has no attribute '_json_response_data'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\queueing.py", line 432, in process_events self.active_jobs[self.active_jobs.index(events)] = None ValueError: [<gradio.queueing.Event object at 0x000001B78074DC60>] is not in list `

Estimated 30 mins for 16 frames 8 fps using txt2img 512x768 Euler 20 steps

WeirdScienceX commented 1 year ago

Ended up as

`100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [14:29<00:00, 43.48s/it] 2023-07-24 19:04:11,783 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet input blocks.15<00:00, 44.67s/it] 2023-07-24 19:04:11,783 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet output blocks. 2023-07-24 19:04:11,783 - AnimateDiff - INFO - Restoring GroupNorm32 forward function. 2023-07-24 19:04:11,784 - AnimateDiff - INFO - Removal finished. 2023-07-24 19:04:11,784 - AnimateDiff - INFO - Merging images into GIF. 2023-07-24 19:04:12,849 - AnimateDiff - INFO - AnimateDiff process end. Error completing request Arguments: ('task(bgmstxu407blk0i)', 'pretty girl', '', [], 20, 0, False, False, 1, 1, 7, 906066021.0, -1.0, 0, 0, 0, False, 768, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 0, '', '', [], 0, True, 0, 16, 8, 'mm_sd_v15.ckpt', <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x000001B780C5DD80>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x000001B78074CA90>, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, 5, 'all', 'all', 'all', '', '', '', '1', 'none', False, '', '', 'comma', '', True, '', '20', 'all', 'all', 'all', 'all', None, None, False, None, None, False, 50) {} Traceback (most recent call last): File "C:\AI\stable-diffusion-webui\modules\call_queue.py", line 55, in f res = list(func(*args, *kwargs)) File "C:\AI\stable-diffusion-webui\modules\call_queue.py", line 35, in f res = func(args, *kwargs) File "C:\AI\stable-diffusion-webui\modules\txt2img.py", line 57, in txt2img processed = processing.process_images(p) File "C:\AI\stable-diffusion-webui\extensions\sd-webui-prompt-history\lib_history\image_process_hijacker.py", line 25, in process_images global_state.add_config( uuid.uuid4().hex, res.prompt[:64], shared.opts.sd_model_checkpoint, res.infotexts[0], res.images[0]) File "C:\AI\stable-diffusion-webui\extensions\sd-webui-prompt-history\scripts\prompt_history_script.py", line 76, in add_config new_height = int(new_width img.height / img.width) AttributeError: 'str' object has no attribute 'height'

` Starting Image:

00000-906066021

Created Gif:

00000-906066021

It seems like a great idea but it needs some work to be worthwhile using

youwen87 commented 1 year ago

RuntimeError: CUDA error: invalid configuration argument CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

continue-revolution commented 1 year ago

@youwen87 https://github.com/continue-revolution/sd-webui-animatediff#update v1.5.2

continue-revolution / sd-webui-animatediff