[Bug]: Controlnet input image sequence/video doesnt work?

Have you read the latest version of the FAQ?

[X] I have visited the FAQ page right now and my issue is not present there

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

Are you using the latest version of the Deforum extension?

[X] I have Deforum updated to the lastest version and I still have the issue.

What happened?

Hey! I've been watching a lot of tutorials, and all of them set the controlnet path input as such : "C:/myPath/myVideo.mp4"

When I do so, here is the error I get :

Error: 'Video file C:\ai\stable-diffusion-webui has format 'c:\ai\stable-diffusion-webui', which is not supported. Supported formats are: ['mov', 'mpeg', 'mp4', 'm4v', 'avi', 'mpg', 'webm']'. Before reporting, please check your schedules/ init values. Full error message is in your terminal/ cli.

Ideally, I'd love to input image sequence, but the result is the same. When I input C:/Z/ as the folder that has all my images, here is the error i get : 'Video file C:\Z has format 'c:\z', which is not supported. Supported formats are: ['mov', 'mpeg', 'mp4', 'm4v', 'avi', 'mpg', 'webm']'. Before reporting, please check your schedules/ init values. Full error message is in your terminal/ cli.

Steps to reproduce the problem

Go ControlNet Input Video/ Image Path
Set a path
press the "generate" button

What should have happened/how would you fix it?

Process of the image sequence/video should start

Torch version

torch: 2.0.1+cu118

On which platform are you launching the webui with the extension?

Local PC setup (Windows)

Deforum settings

20230817140605_settings.txt

{ "W": 512, "H": 512, "show_info_on_ui": true, "tiling": false, "restore_faces": false, "seed_resize_from_w": 0, "seed_resize_from_h": 0, "seed": 4214302718, "sampler": "Euler a", "steps": 25, "batch_name": "Deforum_20230817140605", "seed_behavior": "iter", "seed_iter_N": 1, "use_init": false, "strength": 0.8, "strength_0_no_init": true, "init_image": null, "use_mask": false, "use_alpha_as_mask": false, "mask_file": "https://deforum.github.io/a1/M1.jpg", "invert_mask": false, "mask_contrast_adjust": 1.0, "mask_brightness_adjust": 1.0, "overlay_mask": true, "mask_overlay_blur": 4, "fill": 1, "full_res_mask": true, "full_res_mask_padding": 4, "reroll_blank_frames": "ignore", "reroll_patience": 10.0, "motion_preview_mode": false, "prompts": { "0": " a bird", "14": " a cat " }, "positive_prompts": "", "negative_prompts": "", "animation_mode": "2D", "max_frames": 120, "border": "replicate", "angle": "0: (0)", "zoom": "0: (1)", "translation_x": "0: (0)", "translation_y": "0: (0)", "translation_z": "0: (1.75)", "transform_center_x": "0: (0)", "transform_center_y": "0: (0)", "rotation_3d_x": "0: (0)", "rotation_3d_y": "0: (0)", "rotation_3d_z": "0: (0)", "enable_perspective_flip": false, "perspective_flip_theta": "0: (0)", "perspective_flip_phi": "0: (0)", "perspective_flip_gamma": "0: (0)", "perspective_flip_fv": "0: (53)", "noise_schedule": "0: (0.065)", "strength_schedule": "0: (0.65)", "contrast_schedule": "0: (1.0)", "cfg_scale_schedule": "0: (7)", "enable_steps_scheduling": false, "steps_schedule": "0: (25)", "fov_schedule": "0: (70)", "aspect_ratio_schedule": "0: (1)", "aspect_ratio_use_old_formula": false, "near_schedule": "0: (200)", "far_schedule": "0: (10000)", "seed_schedule": "0:(s), 1:(-1), \"max_f-2\":(-1), \"max_f-1\":(s)", "pix2pix_img_cfg_scale_schedule": "0:(1.5)", "enable_subseed_scheduling": false, "subseed_schedule": "0: (1)", "subseed_strength_schedule": "0: (0)", "enable_sampler_scheduling": false, "sampler_schedule": "0: (\"Euler a\")", "use_noise_mask": false, "mask_schedule": "0: (\"{video_mask}\")", "noise_mask_schedule": "0: (\"{video_mask}\")", "enable_checkpoint_scheduling": false, "checkpoint_schedule": "0: (\"model1.ckpt\"), 100: (\"model2.safetensors\")", "enable_clipskip_scheduling": false, "clipskip_schedule": "0: (2)", "enable_noise_multiplier_scheduling": true, "noise_multiplier_schedule": "0: (1.05)", "resume_from_timestring": false, "resume_timestring": null, "enable_ddim_eta_scheduling": false, "ddim_eta_schedule": "0: (0)", "enable_ancestral_eta_scheduling": false, "ancestral_eta_schedule": "0: (1)", "amount_schedule": "0: (0.1)", "kernel_schedule": "0: (5)", "sigma_schedule": "0: (1)", "threshold_schedule": "0: (0)", "color_coherence": "LAB", "color_coherence_image_path": "", "color_coherence_video_every_N_frames": 1, "color_force_grayscale": false, "legacy_colormatch": false, "diffusion_cadence": 1, "optical_flow_cadence": "None", "cadence_flow_factor_schedule": "0: (1)", "optical_flow_redo_generation": "None", "redo_flow_factor_schedule": "0: (1)", "diffusion_redo": "0", "noise_type": "perlin", "perlin_octaves": 4, "perlin_persistence": 0.5, "use_depth_warping": true, "depth_algorithm": "Midas-3-Hybrid", "midas_weight": 0.2, "padding_mode": "border", "sampling_mode": "bicubic", "save_depth_maps": false, "video_init_path": "https://deforum.github.io/a1/V1.mp4", "extract_nth_frame": 1, "extract_from_frame": 0, "extract_to_frame": -1, "overwrite_extracted_frames": false, "use_mask_video": false, "video_mask_path": "https://deforum.github.io/a1/VM1.mp4", "hybrid_comp_alpha_schedule": "0:(0.5)", "hybrid_comp_mask_blend_alpha_schedule": "0:(0.5)", "hybrid_comp_mask_contrast_schedule": "0:(1)", "hybrid_comp_mask_auto_contrast_cutoff_high_schedule": "0:(100)", "hybrid_comp_mask_auto_contrast_cutoff_low_schedule": "0:(0)", "hybrid_flow_factor_schedule": "0:(1)", "hybrid_generate_inputframes": false, "hybrid_generate_human_masks": "None", "hybrid_use_first_frame_as_init_image": true, "hybrid_motion": "None", "hybrid_motion_use_prev_img": false, "hybrid_flow_consistency": false, "hybrid_consistency_blur": 2, "hybrid_flow_method": "RAFT", "hybrid_composite": "None", "hybrid_use_init_image": false, "hybrid_comp_mask_type": "None", "hybrid_comp_mask_inverse": false, "hybrid_comp_mask_equalize": "None", "hybrid_comp_mask_auto_contrast": false, "hybrid_comp_save_extra_frames": false, "parseq_manifest": "", "parseq_use_deltas": true, "use_looper": false, "init_images": "{\n \"0\": \"https://deforum.github.io/a1/Gi1.png\",\n \"max_f/4-5\": \"https://deforum.github.io/a1/Gi2.png\",\n \"max_f/2-10\": \"https://deforum.github.io/a1/Gi3.png\",\n \"3*max_f/4-15\": \"https://deforum.github.io/a1/Gi4.jpg\",\n \"max_f-20\": \"https://deforum.github.io/a1/Gi1.png\"\n}", "image_strength_schedule": "0:(0.75)", "blendFactorMax": "0:(0.35)", "blendFactorSlope": "0:(0.25)", "tweening_frames_schedule": "0:(20)", "color_correction_factor": "0:(0.075)", "cn_1_overwrite_frames": false, "cn_1_vid_path": "C:/ai/animations/4Lwd/03squatting/reverse/Z/", "cn_1_mask_vid_path": "", "cn_1_enabled": false, "cn_1_low_vram": false, "cn_1_pixel_perfect": false, "cn_1_module": "none", "cn_1_model": "control_v11f1p_sd15_depth [cfd03158]", "cn_1_weight": "0:(1)", "cn_1_guidance_start": "0:(0.0)", "cn_1_guidance_end": "0:(1.0)", "cn_1_processor_res": 64, "cn_1_threshold_a": 64, "cn_1_threshold_b": 64, "cn_1_resize_mode": "Inner Fit (Scale to Fit)", "cn_1_control_mode": "Balanced", "cn_1_loopback_mode": false, "cn_2_overwrite_frames": true, "cn_2_vid_path": "", "cn_2_mask_vid_path": "", "cn_2_enabled": false, "cn_2_low_vram": false, "cn_2_pixel_perfect": false, "cn_2_module": "none", "cn_2_model": "None", "cn_2_weight": "0:(1)", "cn_2_guidance_start": "0:(0.0)", "cn_2_guidance_end": "0:(1.0)", "cn_2_processor_res": 64, "cn_2_threshold_a": 64, "cn_2_threshold_b": 64, "cn_2_resize_mode": "Inner Fit (Scale to Fit)", "cn_2_control_mode": "Balanced", "cn_2_loopback_mode": false, "cn_3_overwrite_frames": true, "cn_3_vid_path": "", "cn_3_mask_vid_path": "", "cn_3_enabled": false, "cn_3_low_vram": false, "cn_3_pixel_perfect": false, "cn_3_module": "none", "cn_3_model": "None", "cn_3_weight": "0:(1)", "cn_3_guidance_start": "0:(0.0)", "cn_3_guidance_end": "0:(1.0)", "cn_3_processor_res": 64, "cn_3_threshold_a": 64, "cn_3_threshold_b": 64, "cn_3_resize_mode": "Inner Fit (Scale to Fit)", "cn_3_control_mode": "Balanced", "cn_3_loopback_mode": false, "cn_4_overwrite_frames": true, "cn_4_vid_path": "", "cn_4_mask_vid_path": "", "cn_4_enabled": false, "cn_4_low_vram": false, "cn_4_pixel_perfect": false, "cn_4_module": "none", "cn_4_model": "None", "cn_4_weight": "0:(1)", "cn_4_guidance_start": "0:(0.0)", "cn_4_guidance_end": "0:(1.0)", "cn_4_processor_res": 64, "cn_4_threshold_a": 64, "cn_4_threshold_b": 64, "cn_4_resize_mode": "Inner Fit (Scale to Fit)", "cn_4_control_mode": "Balanced", "cn_4_loopback_mode": false, "cn_5_overwrite_frames": true, "cn_5_vid_path": "", "cn_5_mask_vid_path": "", "cn_5_enabled": false, "cn_5_low_vram": false, "cn_5_pixel_perfect": false, "cn_5_module": "none", "cn_5_model": "None", "cn_5_weight": "0:(1)", "cn_5_guidance_start": "0:(0.0)", "cn_5_guidance_end": "0:(1.0)", "cn_5_processor_res": 64, "cn_5_threshold_a": 64, "cn_5_threshold_b": 64, "cn_5_resize_mode": "Inner Fit (Scale to Fit)", "cn_5_control_mode": "Balanced", "cn_5_loopback_mode": false, "skip_video_creation": false, "fps": 15, "make_gif": false, "delete_imgs": false, "delete_input_frames": false, "add_soundtrack": "None", "soundtrack_path": "https://deforum.github.io/a1/A1.mp3", "r_upscale_video": false, "r_upscale_factor": "x2", "r_upscale_model": "realesr-animevideov3", "r_upscale_keep_imgs": true, "store_frames_in_ram": false, "frame_interpolation_engine": "None", "frame_interpolation_x_amount": 2, "frame_interpolation_slow_mo_enabled": false, "frame_interpolation_slow_mo_amount": 2, "frame_interpolation_keep_imgs": true, "frame_interpolation_use_upscaled": false, "sd_model_name": "beautyfool_v15.safetensors", "sd_model_hash": "35697588", "deforum_git_commit_id": "19023b1d" }

Webui core settings

https://drive.google.com/file/d/1A55S6cF-iC-CbkLlhyJQ7Q9TfXd2gey7/view?usp=sharing

Console logs

venv "C:\ai\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.4.0-286-ga3ddf464
Commit hash: a3ddf464a2ed24c999f67ddfef7969f8291567be

Installing sd-webui-controlnet requirement: changing opencv-python version from 4.7.0.72 to 4.8.0

Checking roop requirements
Install insightface==0.7.3
Installing sd-webui-roop requirement: insightface==0.7.3
Install onnx==1.14.0
Installing sd-webui-roop requirement: onnx==1.14.0
Install onnxruntime==1.15.0
Installing sd-webui-roop requirement: onnxruntime==1.15.0
Install opencv-python==4.7.0.72
Installing sd-webui-roop requirement: opencv-python==4.7.0.72

Launching Web UI with arguments: --xformers
Civitai Helper: Get Custom Model Folder
Civitai Helper: Load setting from: C:\ai\stable-diffusion-webui\extensions\Stable-Diffusion-Webui-Civitai-Helper\setting.json
Civitai Helper: No setting file, use default
Tag Autocomplete: Could not locate model-keyword extension, Lora trigger word completion will be limited to those added through the extra networks menu.
[-] ADetailer initialized. version: 23.8.0, num models: 9
2023-08-17 14:35:03,515 - ControlNet - INFO - ControlNet v1.1.238
ControlNet preprocessor location: C:\ai\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads
2023-08-17 14:35:03,647 - ControlNet - INFO - ControlNet v1.1.238
2023-08-17 14:35:04,105 - roop - INFO - roop v0.0.2
2023-08-17 14:35:04,106 - roop - INFO - roop v0.0.2
Loading weights [5db118dc11] from C:\ai\stable-diffusion-webui\models\Stable-diffusion\beautyfool_v15.safetensors
Creating model from config: C:\ai\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
*Deforum ControlNet support: enabled*
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
DiffusionWrapper has 859.52 M params.
Startup time: 37.1s (launcher: 19.3s, import torch: 4.3s, import gradio: 1.6s, setup paths: 1.4s, other imports: 1.2s, setup codeformer: 0.1s, load scripts: 6.6s, create ui: 1.2s, gradio launch: 0.4s, app_started_callback: 0.7s).
Loading VAE weights specified in settings: C:\ai\stable-diffusion-webui\models\VAE\klF8Anime2VAE_klF8Anime2VAE.ckpt
Applying attention optimization: xformers... done.
Model loaded in 15.9s (load weights from disk: 0.9s, create model: 1.1s, apply weights to model: 9.5s, apply half(): 1.0s, load VAE: 0.8s, move model to device: 2.1s, load textual inversion embeddings: 0.1s, calculate empty prompt: 0.2s).
started run_deforum
times_to_run: 1
Deforum extension for auto1111 webui
Git commit: 19023b1d
Starting job task(4hmn9nnduv2qfy0)-0...
Unpacking ControlNet 1 base video
Exporting Video Frames to C:\ai\stable-diffusion-webui\outputs\img2img-images\Deforum_20230817143623\controlnet_1_inputframes...
Trying to extract frames from video with input FPS of 24.0. Please wait patiently.
Extracted 14 frames from video in 0.13 seconds!
Loading 120 input frames from C:\ai\stable-diffusion-webui\outputs\img2img-images\Deforum_20230817143623\controlnet_1_inputframes and saving video frames to C:\ai\stable-diffusion-webui\outputs\img2img-images\Deforum_20230817143623
ControlNet 1 base video unpacked!
Unpacking ControlNet 1 video mask
Exporting Video Frames to C:\ai\stable-diffusion-webui\outputs\img2img-images\Deforum_20230817143623\controlnet_1_maskframes...

*START OF TRACEBACK*
Traceback (most recent call last):
  File "C:\ai\stable-diffusion-webui\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\run_deforum.py", line 110, in run_deforum
    render_animation(args, anim_args, video_args, parseq_args, loop_args, controlnet_args, root)
  File "C:\ai\stable-diffusion-webui\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\render.py", line 81, in render_animation
    unpack_controlnet_vids(args, anim_args, controlnet_args)
  File "C:\ai\stable-diffusion-webui\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\deforum_controlnet.py", line 343, in unpack_controlnet_vids
    process_controlnet_input_frames(args, anim_args, controlnet_args, None, mask_path, 'maskframes', i)
  File "C:\ai\stable-diffusion-webui\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\deforum_controlnet.py", line 313, in process_controlnet_input_frames
    vid2frames(
  File "C:\ai\stable-diffusion-webui\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\video_audio_utilities.py", line 87, in vid2frames
    if is_vid_path_valid(video_path):
  File "C:\ai\stable-diffusion-webui\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\video_audio_utilities.py", line 171, in is_vid_path_valid
    raise ValueError(f"Video file {video_path} has format '{extension}', which is not supported. Supported formats are: {file_formats}")
ValueError: Video file C:\ai\stable-diffusion-webui has format 'c:\ai\stable-diffusion-webui', which is not supported. Supported formats are: ['mov', 'mpeg', 'mp4', 'm4v', 'avi', 'mpg', 'webm']
*END OF TRACEBACK*

User friendly error message:
Error: Video file C:\ai\stable-diffusion-webui has format 'c:\ai\stable-diffusion-webui', which is not supported. Supported formats are: ['mov', 'mpeg', 'mp4', 'm4v', 'avi', 'mpg', 'webm']. Please, check your schedules/ init values.

Additional information

No response

deforum-art / sd-webui-deforum