[Bug]: Video masks won't work if resolution set to non-divisible by 64 values

andyxr commented 1 year ago

Have you read the latest version of the FAQ?

[X] I have visited the FAQ page right now and my issue is not present there

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

Are you using the latest version of the Deforum extension?

[X] I have Deforum updated to the lastest version and I still have the issue.

What happened?

Deforum throws an exception if I manually set the W and H values in the Run tab and try to use a video mask.

Steps to reproduce the problem

It is assumed that the video and video mask being used are of the exact same dimensions. In my case it was 852 x 480
Set W & H manually to the width and height of your input video.
Set Noise type to Uniform (bug happens with Uniform and Perlin) because Uniform lets you use non-64 divisible W & H
Set animation mode to Video Input
Specify input video in Init tab and check Use Mask Video
Specify path to mask video
Set up mask in Mask Init
Leave default prompts
Hit Generate

What should have happened?

Video should have been generated, respecting the video mask specified.

WebUI and Deforum extension Commit IDs

webui commit id - https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/5ab7f213bec2f816f9c5644becb32eb72c8ffb89 deforum exten commit id - e392e0ac

Torch version

2.0.0+cu118

On which platform are you launching the webui with the extension?

Local PC setup (Windows)

Deforum settings

mask_bug_settings.txt

Webui core settings

ui-config.json.txt

Console logs

venv "D:\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Commit hash: 5ab7f213bec2f816f9c5644becb32eb72c8ffb89
Installing requirements

Installing requirements for TemporalKit extension

Launching Web UI with arguments: --opt-sdp-no-mem-attention --ckpt-dir D:\sd-models\Stable-diffusion
No module 'xformers'. Proceeding without it.
ControlNet v1.1.134
ControlNet v1.1.134
Loading weights [e72b33b7a2] from D:\sd-models\Stable-diffusion\cheeseDaddysLandscapes_20.safetensors
Creating model from config: D:\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Loading VAE weights specified in settings: D:\stable-diffusion-webui\models\VAE\vae-ft-mse-840000-ema-pruned.safetensors
Applying scaled dot product cross attention optimization (without memory efficient attention).
Textual inversion embeddings loaded(1): EasyNegative
Textual inversion embeddings skipped(3): nartfixer, nfixer, nrealfixer
Model loaded in 2.4s (load weights from disk: 0.1s, create model: 0.2s, apply weights to model: 0.5s, apply half(): 0.4s, move model to device: 0.4s, load textual inversion embeddings: 0.7s).
*Deforum ControlNet support: enabled*
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 7.5s (import torch: 0.9s, import gradio: 0.6s, import ldm: 0.3s, other imports: 0.5s, load scripts: 1.1s, load SD checkpoint: 2.8s, create ui: 1.1s, gradio launch: 0.1s).
reading custom settings from mask_bug_settings.txt
Deforum extension for auto1111 webui, v2.4b
Git commit: e392e0ac (Fri May  5 13:51:02 2023)
Exporting Video Frames (1 every 2) frames to D:\a1111_outputs\img2img-images\Deforum_masktest_20230506191610\inputframes...
Trying to extract frames from video with input FPS of 30.0. Please wait patiently.
Extracted 180 frames from video in 0.23 seconds!
Loading 90 input frames from D:\a1111_outputs\img2img-images\Deforum_masktest_20230506191610\inputframes and saving video frames to D:\a1111_outputs\img2img-images\Deforum_masktest_20230506191610
Exporting Video Frames (1 every 2) frames to D:\a1111_outputs\img2img-images\Deforum_masktest_20230506191610\maskframes...
Trying to extract frames from video with input FPS of 30.0. Please wait patiently.
Extracted 179 frames from video in 0.19 seconds!
Saving animation frames to:
D:\a1111_outputs\img2img-images\Deforum_masktest_20230506191610
Animation frame: 0/90
Seed: 951618567
Prompt: tiny cute swamp bunny, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus, by tomasz alen kopera
Neg Prompt: nsfw, nude  nsfw, nude  nsfw, nude
Using video init frame D:\a1111_outputs\img2img-images\Deforum_masktest_20230506191610\inputframes\mask_test_orig000000001.jpg
use_alpha_as_mask==True: Using the alpha channel from the init image as a mask, but the alpha channel is blank.
ignoring alpha as mask.
mask is blank. ignoring mask
╭─────┬───┬───────╮
│Steps│CFG│Denoise│
├─────┼───┼───────┤
│ 25  │7.0│ 0.35  │
╰─────┴───┴───────╯
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:01<00:00,  6.59it/s]
Deforum progress:   1%|█                                                                                               | 9/810 [00:00<01:00, 13.16it/s]
*START OF TRACEBACK*
Traceback (most recent call last):
  File "D:\stable-diffusion-webui\extensions\deforum-for-automatic1111-webui\scripts\deforum.py", line 113, in run_deforum
    render_input_video(args, anim_args, video_args, parseq_args, loop_args, controlnet_args, root.animation_prompts, root)#TODO: prettify code
  File "D:\stable-diffusion-webui/extensions/deforum-for-automatic1111-webui/scripts\deforum_helpers\render_modes.py", line 53, in render_input_video
    render_animation(args, anim_args, video_args, parseq_args, loop_args, controlnet_args, animation_prompts, root)
  File "D:\stable-diffusion-webui/extensions/deforum-for-automatic1111-webui/scripts\deforum_helpers\render.py", line 559, in render_animation
    image = do_overlay_mask(args, anim_args, image, frame_idx)
  File "D:\stable-diffusion-webui/extensions/deforum-for-automatic1111-webui/scripts\deforum_helpers\masks.py", line 31, in do_overlay_mask
    img = Image.composite(img, current_frame, current_mask)
  File "D:\stable-diffusion-webui\venv\lib\site-packages\PIL\Image.py", line 3341, in composite
    image.paste(image1, None, mask)
  File "D:\stable-diffusion-webui\venv\lib\site-packages\PIL\Image.py", line 1731, in paste
    self.im.paste(im, box, mask.im)
ValueError: images do not match
*END OF TRACEBACK*

User friendly error message:                                                                                                                           ead of \.mages do not match. Check your schedules/ init values please. Also make sure you don't have a backwards slash in any of your PATHs - use / insted

Additional information

No response

github-actions[bot] commented 1 year ago

This issue has been closed due to incorrect formatting. Please address the following mistakes and reopen the issue (click on the 'Reopen' button below):

Include THE FULL LOG FROM THE START OF THE WEBUI in the issue description.

github-actions[bot] commented 1 year ago

This issue has been closed due to incorrect formatting. Please address the following mistakes and reopen the issue (click on the 'Reopen' button below):

Include THE FULL LOG FROM THE START OF THE WEBUI in the issue description.

github-actions[bot] commented 1 year ago

Thanks for addressing your formatting mistakes. The issue has been reopened now.

deforum-art / sd-webui-deforum