search

continue-revolution / sd-webui-animatediff

AnimateDiff for AUTOMATIC1111 Stable Diffusion WebUI
Other
3.04k stars 252 forks source link

[Bug]: Adetailer still not working after the fix? #475

Closed gamefan77 closed 6 months ago

gamefan77 commented 6 months ago

Is there an existing issue for this?

Have you read FAQ on README?

What happened?

When I use adetatiler with animatediff is not woriking (tx2img or img2img); I have updated automatic1111 and animatediff to the latest version; previous versions worked well. I know this bug it has already been reported and closed, but still not working for me; I have delete animatediff folder and the venv folder, and reinstalled the extension, stll not working.

Steps to reproduce the problem

  1. Go to .Enable Animatediff - enable adetailer
  2. Press .Generate
  3. ...

What should have happened?

It should generate a gif with fixed face;

Commit where the problem happens

webui: 1.8.0 python: 3.10.6 torch: 2.1.2+cu121 xformers: 0.0.23.post1

extension: https://github.com/continue-revolution/sd-webuianimatediff/commit/a81565d906c0f16c9a9b95cd80af7b8bafda7cb3

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--xformers

Console logs

2024-03-17 15:52:51,342 - AnimateDiff - INFO - AnimateDiff process start.
2024-03-17 15:52:51,348 - AnimateDiff - INFO - Loading motion module mm_sd15_v3.safetensors from C:\Users\xxx\stable-diffusion-webui\extensions\sd-webui-animatediff\model\mm_sd15_v3.safetensors
2024-03-17 15:52:51,382 - AnimateDiff - INFO - Guessed mm_sd15_v3.safetensors architecture: MotionModuleType.AnimateDiffV3
2024-03-17 15:52:54,437 - AnimateDiff - INFO - Injecting motion module mm_sd15_v3.safetensors into SD1.5 UNet input blocks.
2024-03-17 15:52:54,437 - AnimateDiff - INFO - Injecting motion module mm_sd15_v3.safetensors into SD1.5 UNet output blocks.
2024-03-17 15:52:54,437 - AnimateDiff - INFO - Setting DDIM alpha.
2024-03-17 15:52:54,524 - AnimateDiff - INFO - Injection finished.
2024-03-17 15:52:54,524 - AnimateDiff - INFO - AnimateDiff + ControlNet will generate 16 frames.
  0%|                                                                                           | 0/30 [00:00<?, ?it/s]2024-03-17 15:52:56,682 - AnimateDiff - INFO - inner model forward hooked
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:26<00:00,  1.13it/s]
2024-03-17 15:53:24,136 - AnimateDiff - INFO - Restoring DDIM alpha.███████████████████| 30/30 [00:24<00:00,  1.20it/s]
2024-03-17 15:53:24,136 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet input blocks.
2024-03-17 15:53:24,136 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet output blocks.
2024-03-17 15:53:24,136 - AnimateDiff - INFO - Removal finished.
2024-03-17 15:53:24,136 - AnimateDiff - INFO - Saving output formats: GIF, TXT
2024-03-17 15:53:24,136 - AnimateDiff - INFO - AnimateDiff process end.

0: 640x448 1 face, 81.6ms
Speed: 0.0ms preprocess, 81.6ms inference, 12.8ms postprocess per image at shape (1, 3, 640, 448)
2024-03-17 15:53:24,542 - AnimateDiff - INFO - AnimateDiff process start.
2024-03-17 15:53:24,558 - AnimateDiff - INFO - Injecting motion module mm_sd15_v3.safetensors into SD1.5 UNet input blocks.
2024-03-17 15:53:24,558 - AnimateDiff - INFO - Injecting motion module mm_sd15_v3.safetensors into SD1.5 UNet output blocks.
2024-03-17 15:53:24,558 - AnimateDiff - INFO - Setting DDIM alpha.
2024-03-17 15:53:24,558 - AnimateDiff - INFO - Injection finished.
2024-03-17 15:53:24,558 - AnimateDiff - INFO - AnimateDiff + ControlNet will generate 16 frames.
2024-03-17 15:53:25,205 - AnimateDiff - INFO - Randomizing init_latent according to [1.0, 0.96875, 0.9375, 0.90625, 0.875, 0.84375, 0.8125, 0.78125, 0.75, 0.71875, 0.6875, 0.65625, 0.625, 0.59375, 0.5625, 0.53125].
  0%|                                                                                           | 0/13 [00:00<?, ?it/s]2024-03-17 15:53:25,992 - AnimateDiff - INFO - inner model forward hooked
100%|██████████████████████████████████████████████████████████████████████████████████| 13/13 [00:10<00:00,  1.20it/s]
2024-03-17 15:53:37,986 - AnimateDiff - INFO - Restoring DDIM alpha.
2024-03-17 15:53:37,986 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet input blocks.
2024-03-17 15:53:37,986 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet output blocks.
2024-03-17 15:53:37,986 - AnimateDiff - INFO - Removal finished.
2024-03-17 15:53:37,986 - AnimateDiff - INFO - Saving output formats: GIF, TXT
Total frames to FILM-interpolate: 30. Total frame-pairs: 29.
FILM progress: 100%|███████████████████████████████████████████████████████████████████| 29/29 [00:04<00:00,  6.10it/s]
Interpolation done in 5.27 seconds!
Installing sd-webui-animatediff GIF palette optimization requirement: PyExifTool
2024-03-17 15:53:51,018 - AnimateDiff - WARNING - exiftool not found, required for infotext with optimized GIF palette, try: apt install libimage-exiftool-perl or https://exiftool.org/
2024-03-17 15:53:51,033 - AnimateDiff - INFO - AnimateDiff process end.
*** Error completing request
*** Arguments: ('task(d9265j3mc0sdiaz)', 0, '((best quality)), ((masterpiece)), (detailed), photorealistic, (a woman sitting in a chair)
 <PIL.Image.Image image mode=RGBA size=512x512 at 0x1C130B94AC0>, None, None, None, None, None, None, 30, 'DPM++ 2M Karras', 4, 0, 1, 1, 1, 7, 1.5, 0.75, 0.0, 512, 512, 1, 0, 0, 32, 0, '', '', '', [], False, [], '', <gradio.routes.Request object at 0x000001C13172E590>, 0, False, 1, 0.5, 4, 0, 0.5, 2, False, '', 0.8, 268940427, False, -1, 0, 0, 0, True, False, {'ad_model': 'face_yolov8n.pt', 'ad_model_classes': '', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_model_classes': '', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, <scripts.animatediff_ui.AnimateDiffProcess object at 0x000001C1306418A0>, UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=False, hr_option='Both', save_detected_map=True, advanced_weighting=None), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=False, hr_option='Both', save_detected_map=True, advanced_weighting=None), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=False, hr_option='Both', save_detected_map=True, advanced_weighting=None), False, False, 'Matrix', 'Columns', 'Mask', 'Prompt', '1,1', '0.2', False, False, False, 'Attention', [False], '0', '0', '0.4', None, '0', '0', False, '* `CFG Scale` should be 2 or lower.', True, True, '', '', True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, 'start', '', '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 0, 2, 1, '', [], 0, '', [], 0, '', [], True, False, False, False, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, 50, [], 30, '', 4, [], 1, '', '', '', '', '<p style="margin-bottom:0.75em">Will upscale the image depending on the selected target size type</p>', 512, 0, 8, 32, 64, 0.35, 32, 0, True, 0, False, 8, 0, 0, 2048, 2048, 2) {}
    Traceback (most recent call last):
      File "C:\Users\xxx\stable-diffusion-webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "C:\Users\xxx\stable-diffusion-webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "C:\Users\xxx\stable-diffusion-webui\modules\img2img.py", line 235, in img2img
        processed = process_images(p)
      File "C:\Users\xxx\stable-diffusion-webui\modules\processing.py", line 785, in process_images
        res = process_images_inner(p)
      File "C:\Users\xxx\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 48, in processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "C:\Users\xxx\stable-diffusion-webui\modules\processing.py", line 1008, in process_images_inner
        image, original_denoised_image = apply_overlay(image, p.paste_to, overlay_image)
      File "C:\Users\xxx\stable-diffusion-webui\modules\processing.py", line 77, in apply_overlay
        return image, image.copy()
    AttributeError: 'str' object has no attribute 'copy'

Additional information

Txt2img generate a picure in the folder (not in the UI), but it's poorly animated and the face is bad; Img2img generate 16 gifs in the folder (not in the UI), but they are almost the same and poorly animated;

continue-revolution commented 6 months ago

screenshot everything in your UI (not your terminal) that can reproduce this error

gamefan77 commented 6 months ago

Screenshot 2024-03-17 214406 image image image image

continue-revolution commented 6 months ago

there is no way that I can reproduce this error. I don't know what happened. Even with the settings in your screenshot, things work fine in my machine. You can also tell this in https://github.com/continue-revolution/sd-webui-animatediff/issues/446#issuecomment-2002281776

also I don't recommend adetailer unless your face is really small. If the face is large, your video will only become worse. There is no way to fix this, due to the internal of adetailer. There is no motion module in your stable diffusion checkpoint, when you are doing face fix via adetailer.

no adetailer w/ adetailer
00004-1408095377 00005-1408095377
moonsunstars39 commented 5 months ago

there is no way that I can reproduce this error. I don't know what happened. Even with the settings in your screenshot, things work fine in my machine. You can also tell this in #446 (comment)

also I don't recommend adetailer unless your face is really small. If the face is large, your video will only become worse. There is no way to fix this, due to the internal of adetailer. There is no motion module in your stable diffusion checkpoint, when you are doing face fix via adetailer.

Is that possible to hijack the Adetailer process in order to add the motion module into Adetailer's checkpoint ? Maybe we can get some good faces with good animation :D