Issues regarding Masks and 3D mode

[AiKills]

Hello, Ive been getting two different errors that i am unable to workout on my own. The first being an issue with 3d mode. when using it, I get nothing but noise frames after the first image such as you see below. I also get an actual error if i try disabling 3d warping but not sure if thats normal. Noisy Image:

Kept all settings at default other than slightly raising the samples. (may notice the translation_z at 1 but same results at any number) Settings: 20221027023619_settings.txt

The second error i have is when trying to use a mask. Everything works fine until the second i enable the mask. Also happens with the default mask link. Error and setting below.

Settings: 20221027024630_settings.txt

Error: Error completing request Arguments: (False, '', '3D', 100, 'replicate', '0:(0)', '0:(1)', '0:(0)', '0:(0)', '0:(10)', '0:(0)', '0:(0)', '0:(0)', False, '0:(0)', '0:(t%15)', '0:(0)', '0:(53)', '0: (0.08)', '0: (0.6)', '0: (1.0)', '0: (7)', '0: (40)', '0: (200)', '0: (10000)', '0: (t%4294967293)', 'Match Frame 0 LAB', 1, True, 0.3, 200.0, 10000.0, 40.0, 'border', 'bicubic', False, '/content/video_in.mp4', 1, False, False, '/content/videoin.mp4', False, 4, False, '20220829210106', '[\n "a beautiful forest by Asher Brown Durand, trending on Artstation",\n "a beautiful portrait of a woman by Artgerm, trending on Artstation"\n]\n', '{\n "0": "apple:where(cos(6.28*t/10)>0, 2*cos(6.28*t/10), 0.001), strawberry:where(cos(6.28*t/10)<0, -2*cos(6.28*t/10), 0.001), snow, detailed painting by greg rutkowski --neg apple:where(cos(6.28*t/10)<0, -2*cos(6.28*t/10), 0.001), strawberry:where(cos(6.28*t/10)>0, 2*cos(6.28*t/10), 0.001)",\n "60": "a beautiful (((banana))), trending on Artstation",\n "80": "a beautiful coconut --neg photo, realistic",\n "100": "a beautiful durian, trending on Artstation"\n}\n', 512, 512, False, False, False, 0, 0, -1, 0, False, -1, 0, 0, 0, 30, 0.0, 1, False, 1, True, True, False, False, False, False, 'testing', '{timestring}{index}_{prompt}.png', 'iter', True, False, True, 0.7, 'https://user-images.githubusercontent.com/14872007/195867706-d067cdc6-28cd-450b-a61e-55e25bc67010.png', True, False, False, True, 'https://www.filterforge.com/wiki/images/archive/b/b7/20080927223728%21Polygonal_gradient_thumb.jpg', 1.0, 5.0, False, 12.0, 'FFMPEG mp4', 'ffmpeg', False, 'snowfall.mp3', False, False, 200.0, 'x0pred', '/content/drive/MyDrive/AI/StableDiffusion/2022-09/20220903000939%05d.png', '/content/drive/MyDrive/AI/StableDiffusion/content/drive/MyDrive/AI/StableDiffusion/2022-09/kabachuha/2022-09/20220903000939.mp4', '

Deforum v0.5-webui-beta

', "

Made by deforum.github.io, port for AUTOMATIC1111's webui maintained by kabachuha

", '

Original Deforum Github repo github.com/deforum/stable-diffusion

', "

This fork for auto1111's webui github.com/deforum-art/deforum-for-automatic1111-webui

", '

Join the official Deforum Discord discord.gg/deforum to share your creations and suggestions

', '

User guide for v0.5 docs.google.com/document/d/1pEobUknMFMkn8F5TMsv8qRzamXX_75BShMMXV8IFslI/edit

', '

Math keyframing explanation docs.google.com/document/d/1pfW1PwbDIuW0cv-dnuyYj1UzPqe23BlSLTJsqazffXM/edit?usp=sharing

', '

Import settings from file

', '

Animation settings

', '

Motion parameters:

', '

2D and 3D settings

', '

Prespective flip — Low VRAM pseudo-3D mode:

', '

Coherence:

', '

3D Depth Warping:

', '

Video Input:

', '

Interpolation (turned off atm)

', '

Resume animation:

', '

Prompts

', '

animation_mode: None batches on list of prompts. (Batch mode disabled atm, only animation_prompts are working)

', '

Important change from vanilla Deforum!

', '

This script uses the built-in webui weighting settings.

', '

So if you want to use math functions as prompt weights,

', '

keep the values above zero in both parts

', '

Negative prompt part can be specified with --neg

', '

Run settings

', '

Sampling settings

', '

3D Fov settings:

', '', '', '', '

Batch settings

', '

Init settings

', '

3D settings

', '

Generation settings:

', '

Video output settings

', '

To enable seed schedule select seed behavior — \'schedule\'

') {} Traceback (most recent call last): File "C:\Users\reidb\Desktop\SUPER SD\stable-diffusion-webui\modules\ui.py", line 221, in f res = list(func(*args, *kwargs)) File "C:\Users\reidb\Desktop\SUPER SD\stable-diffusion-webui\webui.py", line 63, in f res = func(args, **kwargs) File "C:\Users\reidb\Desktop\SUPER SD\stable-diffusion-webui\extensions\deforum\scripts\deforum.py", line 219, in run_deforum processed = DeforumScript.run(None, p, override_settings_with_file, custom_settings_file, animation_mode, max_frames, border, angle, zoom, translation_x, translation_y, translation_z, rotation_3d_x, rotation_3d_y, rotation_3d_z, flip_2d_perspective, perspective_flip_theta, perspective_flip_phi, perspective_flip_gamma, perspective_flip_fv, noise_schedule, strength_schedule, contrast_schedule, cfg_scale_schedule, fov_schedule, near_schedule, far_schedule, seed_schedule, color_coherence, diffusion_cadence, use_depth_warping, midas_weight, near_plane, far_plane, fov, padding_mode, sampling_mode, save_depth_maps, video_init_path, extract_nth_frame, overwrite_extracted_frames, use_mask_video, video_mask_path, interpolate_key_frames, interpolate_x_frames, resume_from_timestring, resume_timestring, prompts, animation_prompts, W, H, restore_faces, tiling, enable_hr, firstphase_width, firstphase_height, seed, sampler, seed_enable_extras, subseed, subseed_strength, seed_resize_from_w, seed_resize_from_h, steps, ddim_eta, n_batch, make_grid, grid_rows, save_settings, save_samples, display_samples, save_sample_per_step, show_sample_per_step, override_these_with_webui, batch_name, filename_format, seed_behavior, use_init, from_img2img_instead_of_link, strength_0_no_init, strength, init_image, use_mask, use_alpha_as_mask, invert_mask, overlay_mask, mask_file, mask_brightness_adjust, mask_overlay_blur, skip_video_for_run_all, fps, output_format, ffmpeg_location, add_soundtrack, soundtrack_path, use_manual_settings, render_steps, max_video_frames, path_name_modifier, image_path, mp4_path, i1, i2, i3, i4, i5, i6, i7, i8, i9, i10, i11, i12, i13, i14, i15, i16, i17, i18, i19, i20, i21, i22, i23, i24, i25, i26, i27, i28, i29, i30, i31, i32, i33, i34, i35, i36) File "C:\Users\reidb\Desktop\SUPER SD\stable-diffusion-webui\extensions\deforum\scripts\deforum.py", line 66, in run render_animation(args, anim_args, root.animation_prompts, root) File "C:\Users\reidb\Desktop\SUPER SD\stable-diffusion-webui\extensions\deforum\scripts\deforum\render.py", line 212, in render_animation sample, image = generate(args, root, frame_idx, return_sample=True) File "C:\Users\reidb\Desktop\SUPER SD\stable-diffusion-webui\extensions\deforum\scripts\deforum\generate.py", line 199, in generate init_image.shape, File "C:\Users\reidb\Desktop\SUPER SD\stable-diffusion-webui\venv\lib\site-packages\PIL\Image.py", line 517, in getattr raise AttributeError(name) AttributeError: shape

Any help is much appreciated. This is my 4th day trying to fix it.

[kabachuha]

Try if it has been fixed by @Funofabot 's commits today

[AiKills]

The mask seems to be fixed but still having the same issue with 3D mode.

[Funofabot]

I believe that it is not fixed because of noising. I am working on currently: The add_noise function which will require multiple file edits. However this is the focus on this edit for masking. Is the noise problem. I just got off work and and I will be sleeping soon however the problem is with the add_noise function and that it currently has no idea of a mask. My first attempt is to modify or create a new function for add_noise. Either way based on the current implementation, the mask needs a pathway to the add_noise function. So that is where the next update that I envision will be at. Unless you guys have a better idea where I should spend my time regarding this problem. I have read the issue #40 and partially it is solved, the noise problem still exists.

On Fri, Nov 4, 2022 at 8:38 PM AiKills @.***> wrote:

The mask seems to be fixed but still having the same issue with 3D mode.

— Reply to this email directly, view it on GitHub https://github.com/deforum-art/deforum-for-automatic1111-webui/issues/40#issuecomment-1304382613, or unsubscribe https://github.com/notifications/unsubscribe-auth/A35L6JSYQD7XSP3XGAFEHP3WGXCDPANCNFSM6AAAAAARPYJGCY . You are receiving this because you were mentioned.Message ID: @.*** com>

[AiKills]

Now when i try to use masks i get this error: "m\generate.py", line 37, in load_img image = Image.open(requests.get(path, stream=True).raw) File "C:\Users\reidb\Desktop\SUPER SD\stable-diffusion-webui\venv\lib\site-packages\PIL\Image.py", line 3147, in open raise UnidentifiedImageError( PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x00000226BFC23F10>"

However the noise issue I was referring to doesnt have to do with masks. The noise just happens when i try to use 3d mode. It only renders one frame then spits out images like you see in #40

[AiKills]

I believe that it is not fixed because of noising. I am working on currently: The add_noise function which will require multiple file edits. However this is the focus on this edit for masking. Is the noise problem. I just got off work and and I will be sleeping soon however the problem is with the add_noise function and that it currently has no idea of a mask. My first attempt is to modify or create a new function for add_noise. Either way based on the current implementation, the mask needs a pathway to the add_noise function. So that is where the next update that I envision will be at. Unless you guys have a better idea where I should spend my time regarding this problem. I have read the issue #40 and partially it is solved, the noise problem still exists. … On Fri, Nov 4, 2022 at 8:38 PM AiKills @.> wrote: The mask seems to be fixed but still having the same issue with 3D mode. — Reply to this email directly, view it on GitHub <#40 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/A35L6JSYQD7XSP3XGAFEHP3WGXCDPANCNFSM6AAAAAARPYJGCY . You are receiving this because you were mentioned.Message ID: @. com>

actually just updated to the git pull version and the masking error is gone. However i still cant use 3d mode

[Funofabot]

Thank you. Still needs fixing. =(

[Funofabot]

Im looking at this for the next PR currently

[Funofabot]

Can we close this issue and work on the one that you reposted in #95 Continued Issue, Unable to use 3d mode #95

[hithereai]

Closing as the continuation in #95 is closed too. Reopen 95 if needed.