[Bug]: Subtle changing outside of the mask

Sh4d0wHunt3rX commented 1 year ago

Have you read the latest version of the FAQ?

[X] I have visited the FAQ page right now and my issue is not present there

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

Are you using the latest version of the Deforum extension?

[X] I have Deforum updated to the lastest version and I still have the issue.

What happened?

I'm using an init image + image mask. So I want animation inside of my mask area. The issue is that the noise is applying outside of the mask too. So over time, the init image will become pixelated.

Steps to reproduce the problem

You can reproduce this issue by the default settings. Default mask and an init image.

What should have happened?

We shouldn't see any noise outside of mask area. The init image should not get pixelated.

WebUI and Deforum extension Commit IDs

Deforum 5d7e0501 (Wed Mar 29 00:54:34 2023) python: 3.10.9 • torch: 2.0.0+cu118 • xformers: 0.0.18.dev489 • gradio: 3.16.2 • commit: [a9eab236]

On which platform are you launching the webui with the extension?

Windows 10 64bit

Deforum settings

{
    "W": 560,
    "H": 312,
    "tiling": false,
    "restore_faces": false,
    "seed": 86007651,
    "sampler": "Euler a",
    "seed_resize_from_w": 0,
    "seed_resize_from_h": 0,
    "steps": 25,
    "ddim_eta": 0.0,
    "save_settings": true,
    "save_sample_per_step": false,
    "batch_name": "Deforum_20230330161738",
    "seed_behavior": "iter",
    "seed_iter_N": 1,
    "use_init": true,
    "strength_0_no_init": true,
    "strength": 0.8,
    "init_image": "C:/Amir/Stable Diffusion/Check/Mask/1.png",
    "use_mask": true,
    "use_alpha_as_mask": true,
    "invert_mask": false,
    "overlay_mask": false,
    "mask_file": "C:/Amir/Stable Diffusion/Check/Mask/mask4.png",
    "mask_contrast_adjust": 1.0,
    "mask_brightness_adjust": 1.0,
    "mask_overlay_blur": 4,
    "fill": 1,
    "full_res_mask": true,
    "full_res_mask_padding": 4,
    "reroll_blank_frames": "reroll",
    "precision": "autocast",
    "timestring": "20230330161738",
    "prompts": {
        "0": " children play football --neg "
    },
    "positive_prompts": "",
    "negative_prompts": "",
    "outdir": "C:\\Users\\Amir\\stable-diffusion-webui\\outputs/img2img-images\\Deforum_20230330161738",
    "animation_mode": "2D",
    "max_frames": 120,
    "border": "wrap",
    "angle": "0:(0)",
    "zoom": "0:(1.0)",
    "translation_x": "0:(0)",
    "translation_y": "0:(0)",
    "translation_z": "0:(1.75)",
    "transform_center_x": "0:(0.5)",
    "transform_center_y": "0:(0.5)",
    "rotation_3d_x": "0:(0)",
    "rotation_3d_y": "0:(0)",
    "rotation_3d_z": "0:(0)",
    "enable_perspective_flip": false,
    "perspective_flip_theta": "0:(0)",
    "perspective_flip_phi": "0:(0)",
    "perspective_flip_gamma": "0:(0)",
    "perspective_flip_fv": "0:(53)",
    "noise_schedule": "0: (0.065)",
    "strength_schedule": "0: (0.65)",
    "contrast_schedule": "0: (1.0)",
    "cfg_scale_schedule": "0: (7)",
    "pix2pix_img_cfg_scale_schedule": "0:(1.5)",
    "enable_subseed_scheduling": false,
    "subseed_schedule": "0:(1)",
    "subseed_strength_schedule": "0:(0)",
    "enable_steps_scheduling": false,
    "steps_schedule": "0: (25)",
    "fov_schedule": "0: (70)",
    "aspect_ratio_schedule": "0: (1)",
    "near_schedule": "0: (200)",
    "far_schedule": "0: (10000)",
    "seed_schedule": "0:(s), 1:(-1), \"max_f-2\":(-1), \"max_f-1\":(s)",
    "enable_sampler_scheduling": false,
    "sampler_schedule": "0: (\"Euler a\")",
    "mask_schedule": "0: (\"{video_mask}\")",
    "use_noise_mask": false,
    "noise_mask_schedule": "0: (\"{video_mask}\")",
    "enable_checkpoint_scheduling": false,
    "checkpoint_schedule": "0: (\"model1.ckpt\"), 100: (\"model2.ckpt\")",
    "enable_clipskip_scheduling": false,
    "clipskip_schedule": "0: (2)",
    "enable_noise_multiplier_scheduling": true,
    "noise_multiplier_schedule": "0: (1.05)",
    "kernel_schedule": "0: (5)",
    "sigma_schedule": "0: (1.0)",
    "amount_schedule": "0: (0.1)",
    "threshold_schedule": "0: (0.0)",
    "color_coherence": "LAB",
    "color_coherence_image_path": "",
    "color_coherence_video_every_N_frames": 1.0,
    "color_force_grayscale": false,
    "diffusion_cadence": "2",
    "optical_flow_cadence": "None",
    "optical_flow_redo_generation": false,
    "diffusion_redo": "0",
    "noise_type": "perlin",
    "perlin_octaves": 4,
    "perlin_persistence": 0.5,
    "use_depth_warping": true,
    "midas_weight": 0.2,
    "padding_mode": "border",
    "sampling_mode": "bicubic",
    "save_depth_maps": false,
    "video_init_path": "https://deforum.github.io/a1/V1.mp4",
    "extract_nth_frame": 1,
    "extract_from_frame": 0,
    "extract_to_frame": -1,
    "overwrite_extracted_frames": false,
    "use_mask_video": false,
    "video_mask_path": "https://deforum.github.io/a1/VM1.mp4",
    "resume_from_timestring": false,
    "resume_timestring": "20230129210106",
    "hybrid_generate_inputframes": false,
    "hybrid_generate_human_masks": "None",
    "hybrid_use_first_frame_as_init_image": true,
    "hybrid_motion": "None",
    "hybrid_motion_use_prev_img": false,
    "hybrid_flow_method": "DIS Fine",
    "hybrid_composite": false,
    "hybrid_comp_mask_type": "None",
    "hybrid_comp_mask_inverse": false,
    "hybrid_comp_mask_equalize": "None",
    "hybrid_comp_mask_auto_contrast": false,
    "hybrid_comp_save_extra_frames": false,
    "hybrid_comp_alpha_schedule": "0:(0.5)",
    "hybrid_flow_factor_schedule": "0:(1)",
    "hybrid_comp_mask_blend_alpha_schedule": "0:(0.5)",
    "hybrid_comp_mask_contrast_schedule": "0:(1)",
    "hybrid_comp_mask_auto_contrast_cutoff_high_schedule": "0:(100)",
    "hybrid_comp_mask_auto_contrast_cutoff_low_schedule": "0:(0)",
    "parseq_manifest": "",
    "parseq_use_deltas": true,
    "use_looper": false,
    "init_images": "{\n    \"0\": \"https://deforum.github.io/a1/Gi1.png\",\n    \"max_f/4-5\": \"https://deforum.github.io/a1/Gi2.png\",\n    \"max_f/2-10\": \"https://deforum.github.io/a1/Gi3.png\",\n    \"3*max_f/4-15\": \"https://deforum.github.io/a1/Gi4.jpg\",\n    \"max_f-20\": \"https://deforum.github.io/a1/Gi1.png\"\n}",
    "image_strength_schedule": "0:(0.75)",
    "blendFactorMax": "0:(0.35)",
    "blendFactorSlope": "0:(0.25)",
    "tweening_frames_schedule": "0:(20)",
    "color_correction_factor": "0:(0.075)",
    "controlnet_overwrite_frames": true,
    "controlnet_vid_path": "",
    "controlnet_mask_vid_path": "",
    "controlnet_enabled": false,
    "controlnet_guess_mode": false,
    "controlnet_invert_image": false,
    "controlnet_rgbbgr_mode": false,
    "controlnet_lowvram": false,
    "controlnet_module": "none",
    "controlnet_model": "None",
    "controlnet_weight": 1,
    "controlnet_guidance_start": 0,
    "controlnet_guidance_end": 1,
    "controlnet_processor_res": 64,
    "controlnet_threshold_a": 64,
    "controlnet_threshold_b": 64,
    "controlnet_resize_mode": "Scale to Fit (Inner Fit)",
    "skip_video_creation": false,
    "fps": 15,
    "make_gif": false,
    "delete_imgs": false,
    "ffmpeg_location": "C:\\Users\\Amir\\stable-diffusion-webui\\venv\\lib\\site-packages\\imageio_ffmpeg\\binaries\\ffmpeg-win64-v4.2.2.exe",
    "ffmpeg_crf": "17",
    "ffmpeg_preset": "slow",
    "add_soundtrack": "None",
    "soundtrack_path": "https://deforum.github.io/a1/A1.mp3",
    "r_upscale_video": false,
    "r_upscale_model": "realesr-animevideov3",
    "r_upscale_factor": "x2",
    "r_upscale_keep_imgs": true,
    "store_frames_in_ram": false,
    "frame_interpolation_engine": "None",
    "frame_interpolation_x_amount": 2,
    "frame_interpolation_slow_mo_enabled": false,
    "frame_interpolation_slow_mo_amount": 2,
    "frame_interpolation_keep_imgs": false,
    "deforum_git_commit_id": "5d7e0501 (Wed Mar 29 00:54:34 2023)"
}

Webui core settings

-

Console logs

Additional information

https://user-images.githubusercontent.com/15929497/228886530-3e09d94a-e428-4cc9-a962-76cbb3ae4608.mp4

Marcusgeorgiadis commented 1 year ago

Do we know what's causing this? Is there any workaround?

reallybigname commented 1 year ago

Overlay mask setting DOES NOTHING in the code. That's the problem. It might as well say "Placebo". No workaround except for someone to restore the code.

reallybigname commented 1 year ago

A more accurate title would be that Masks are not working after generation. They work before generation, which doesn't do anybody much good at all. That's the reason Overlay mask is enabled by default - because masks are pretty useless without it, except for artistic stuff.

Sh4d0wHunt3rX commented 1 year ago

Thanks. I guess also "invert mask" not works as well.

reallybigname commented 1 year ago

Thanks. I guess also "invert mask" not works as well.

Actually, I think that works... But, it doesn't work on the overlay mask.

hithereai commented 1 year ago

Overlay masks and Invert mask fixed by @reallybigname so this issue is no longer active.

deforum-art / sd-webui-deforum