[Bug]:

[highjohnconquer]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

Are you using the latest version of the extension?

• [X] I have the modelscope text2video extension updated to the lastest version and I still have the issue.

What happened?

E:\Documents\AI\stable-diffusion-webui\outputs/img2img-images\text2video-modelscope\20230331045111\vid.mp4 Stitching video... Stitching video... Traceback (most recent call last): File "E:\Documents\AI\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\video_audio_utils.py", line 147, in ffmpeg_stitch_video process = subprocess.Popen( File "C:\Python310\lib\subprocess.py", line 969, in init self._execute_child(args, executable, preexec_fn, close_fds, File "C:\Python310\lib\subprocess.py", line 1438, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, PermissionError: [WinError 5] Access is denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "E:\Documents\AI\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope-text2vid.py", line 174, in process ffmpeg_stitch_video(ffmpeg_location=ffmpeg_location, fps=fps, outmp4_path=outdir_current + os.path.sep + f"vid.mp4", imgs_path=os.path.join(outdir_current, File "E:\Documents\AI\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\video_audio_utils.py", line 158, in ffmpeg_stitch_video raise Exception( Exception: Error stitching frames to video. Actual runtime error:[WinError 5] Access is denied Exception occurred: Error stitching frames to video. Actual runtime error:[WinError 5] Access is denied

ERROR: Exception in ASGI application Traceback (most recent call last): File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\anyio\streams\memory.py", line 94, in receive return self.receive_nowait() File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\anyio\streams\memory.py", line 89, in receive_nowait raise WouldBlock anyio.WouldBlock

Steps to reproduce the problem

Simply running the app

What should have happened?

Videos should have been stitched together and exported into folder

WebUI and Deforum extension Commit IDs

webui commit id - a9fed7c3 txt2vid commit id - ModelScope text2video extension for auto1111 — version 1.0b.

What GPU were you using for launching?

RTX 3070

On which platform are you launching the webui backend with the extension?

Local PC setup (Windows)

Settings

I'm not sure what to put here

Console logs

error: Your local changes to the following files would be overwritten by merge:
        requirements_versions.txt
Please commit your changes or stash them before you merge.
Aborting
Updating a9fed7c3..22bcc7be
venv "E:\Documents\AI\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.7 (tags/v3.10.7:6cc6b13, Sep  5 2022, 14:08:36) [MSC v.1933 64 bit (AMD64)]
Commit hash: a9fed7c364061ae6efb37f797b6b522cb3cf7aa2
Installing requirements for Web UI

Installing requirements for scikit_learn

current transparent-background 1.2.3

Installing requirements for Prompt Gallery

Installing sd-dynamic-prompts requirements.txt

Initializing Riffusion

Initializing Dreambooth
If submitting an issue on github, please provide the below text for debugging purposes:

Python revision: 3.10.7 (tags/v3.10.7:6cc6b13, Sep  5 2022, 14:08:36) [MSC v.1933 64 bit (AMD64)]
Dreambooth revision: da2e40415f1cb63cc4de46d6dc97eb8676c6e30c
SD-WebUI revision: a9fed7c364061ae6efb37f797b6b522cb3cf7aa2

[+] torch version 2.0.0+cu118 installed.
[+] torchvision version 0.15.1+cu118 installed.
[+] xformers version 0.0.17rc482 installed.
[+] accelerate version 0.17.1 installed.
[+] diffusers version 0.14.0 installed.
[+] transformers version 4.27.2 installed.
[+] bitsandbytes version 0.35.4 installed.

loading Smart Crop reqs from E:\Documents\AI\stable-diffusion-webui\extensions\sd_smartprocess\requirements.txt
Checking Smart Crop requirements.

Installing imageio-ffmpeg requirement for depthmap script
Installing pyqt5 requirement for depthmap script

Installing requirements for Unprompted - img2pez
Installing requirements for Unprompted - pix2pix_zero

Installing video2video requirement: sk-video

Launching Web UI with arguments: --xformers --api --no-half-vae
E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\torchvision\transforms\functional_tensor.py:5: UserWarning: The torchvision.transforms.functional_tensor module is deprecated in 0.15 and will be **removed in 0.17**. Please don't rely on it. You probably just need to use APIs in torchvision.transforms.functional or in torchvision.transforms.v2.functional.
  warnings.warn(
Additional Network extension not installed, Only hijack built-in lora
LoCon Extension hijack built-in lora successfully
Error loading script: simple_depthmap.py
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\modules\scripts.py", line 248, in load_scripts
    script_module = script_loading.load_module(scriptfile.path)
  File "E:\Documents\AI\stable-diffusion-webui\modules\script_loading.py", line 11, in load_module
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "E:\Documents\AI\stable-diffusion-webui\extensions\multi-subject-render\scripts\simple_depthmap.py", line 11, in <module>
    from repositories.midas.midas.dpt_depth import DPTDepthModel
  File "E:\Documents\AI\stable-diffusion-webui\repositories\midas\midas\dpt_depth.py", line 5, in <module>
    from .blocks import (
  File "E:\Documents\AI\stable-diffusion-webui\repositories\midas\midas\blocks.py", line 4, in <module>
    from .backbones.beit import (
  File "E:\Documents\AI\stable-diffusion-webui\repositories\midas\midas\backbones\beit.py", line 9, in <module>
    from timm.models.beit import gen_relative_position_index
ModuleNotFoundError: No module named 'timm.models.beit'

[AddNet] Updating model hashes...
0it [00:00, ?it/s]
[AddNet] Updating model hashes...
0it [00:00, ?it/s]
Hypernetwork-MonkeyPatch-Extension not found
Error loading script: riffusion.py
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\modules\scripts.py", line 248, in load_scripts
    script_module = script_loading.load_module(scriptfile.path)
  File "E:\Documents\AI\stable-diffusion-webui\modules\script_loading.py", line 11, in load_module
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "E:\Documents\AI\stable-diffusion-webui\extensions\sd-webui-riffusion\scripts\riffusion.py", line 11, in <module>    import torchaudio
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\torchaudio\__init__.py", line 1, in <module>
    from torchaudio import (  # noqa: F401
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\torchaudio\_extension.py", line 135, in <module>
    _init_extension()
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\torchaudio\_extension.py", line 105, in _init_extension
    _load_lib("libtorchaudio")
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\torchaudio\_extension.py", line 52, in _load_lib
    torch.ops.load_library(path)
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\torch\_ops.py", line 643, in load_library
    ctypes.CDLL(path)
  File "C:\Python310\lib\ctypes\__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'E:\Documents\AI\stable-diffusion-webui\venv\Lib\site-packages\torchaudio\lib\libtorchaudio.pyd' (or one of its dependencies). Try using the full path with constructor syntax.

Error loading script: patch_fixer.py
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\modules\scripts.py", line 248, in load_scripts
    script_module = script_loading.load_module(scriptfile.path)
  File "E:\Documents\AI\stable-diffusion-webui\modules\script_loading.py", line 11, in load_module
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "E:\Documents\AI\stable-diffusion-webui\extensions\sd_auto_fix\scripts\patch_fixer.py", line 16, in <module>
    from modules.sd_hijack_inpainting import do_inpainting_hijack, should_hijack_inpainting
ImportError: cannot import name 'should_hijack_inpainting' from 'modules.sd_hijack_inpainting' (E:\Documents\AI\stable-diffusion-webui\modules\sd_hijack_inpainting.py)

[text2prompt] Following databases are available:
    all-mpnet-base-v2 : danbooru_strict
Loading Unprompted v7.6.0 by Therefore Games
(SETUP) Initializing Unprompted object...
(SETUP) Loading configuration files...
(SETUP) Debug mode is False
Loading weights [76b00ee812] from E:\Documents\AI\stable-diffusion-webui\models\Stable-diffusion\icomix_V02Pruned.safetensors
Creating model from config: E:\Documents\AI\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying xformers cross attention optimization.
Error loading embedding aivazovsky.pt:
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 206, in load_from_dir
    self.load_from_file(fullfn, fn)
  File "E:\Documents\AI\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 164, in load_from_file
    if 'string_to_param' in data:
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\torch\_tensor.py", line 999, in __contains__
    raise RuntimeError(
RuntimeError: Tensor.__contains__ only supports Tensor or scalar, but you passed in a <class 'str'>.

Error loading embedding cloudcore.pt:
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 206, in load_from_dir
    self.load_from_file(fullfn, fn)
  File "E:\Documents\AI\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 164, in load_from_file
    if 'string_to_param' in data:
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\torch\_tensor.py", line 999, in __contains__
    raise RuntimeError(
RuntimeError: Tensor.__contains__ only supports Tensor or scalar, but you passed in a <class 'str'>.

Error loading embedding fantasy.pt:
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 206, in load_from_dir
    self.load_from_file(fullfn, fn)
  File "E:\Documents\AI\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 164, in load_from_file
    if 'string_to_param' in data:
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\torch\_tensor.py", line 999, in __contains__
    raise RuntimeError(
RuntimeError: Tensor.__contains__ only supports Tensor or scalar, but you passed in a <class 'str'>.

Error loading embedding flower_plant.pt:
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 206, in load_from_dir
    self.load_from_file(fullfn, fn)
  File "E:\Documents\AI\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 164, in load_from_file
    if 'string_to_param' in data:
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\torch\_tensor.py", line 999, in __contains__
    raise RuntimeError(
RuntimeError: Tensor.__contains__ only supports Tensor or scalar, but you passed in a <class 'str'>.

Error loading embedding gloomcore.pt:
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 206, in load_from_dir
    self.load_from_file(fullfn, fn)
  File "E:\Documents\AI\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 164, in load_from_file
    if 'string_to_param' in data:
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\torch\_tensor.py", line 999, in __contains__
    raise RuntimeError(
RuntimeError: Tensor.__contains__ only supports Tensor or scalar, but you passed in a <class 'str'>.

Error loading embedding glowwave.pt:
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 206, in load_from_dir
    self.load_from_file(fullfn, fn)
  File "E:\Documents\AI\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 164, in load_from_file
    if 'string_to_param' in data:
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\torch\_tensor.py", line 999, in __contains__
    raise RuntimeError(
RuntimeError: Tensor.__contains__ only supports Tensor or scalar, but you passed in a <class 'str'>.

Error loading embedding laion_7plus.pt:
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 206, in load_from_dir
    self.load_from_file(fullfn, fn)
  File "E:\Documents\AI\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 164, in load_from_file
    if 'string_to_param' in data:
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\torch\_tensor.py", line 999, in __contains__
    raise RuntimeError(
RuntimeError: Tensor.__contains__ only supports Tensor or scalar, but you passed in a <class 'str'>.

Error loading embedding sac_8plus.pt:
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 206, in load_from_dir
    self.load_from_file(fullfn, fn)
  File "E:\Documents\AI\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 164, in load_from_file
    if 'string_to_param' in data:
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\torch\_tensor.py", line 999, in __contains__
    raise RuntimeError(
RuntimeError: Tensor.__contains__ only supports Tensor or scalar, but you passed in a <class 'str'>.

Textual inversion embeddings loaded(58): 7dirtywords, advntr, angry512, arcan3, arcan3v2, art by Smoose2, bad-artist-anime, bad-artist, bad-hands-5, bad_prompt, bad_prompt_version2, cardstyle15, charturnerv2, clrs, corneo_spitroast, darkskin_style, defiance512, easynegative, eonn, flame_surge_style, fs2023, ghst-3000, gigaschizonegs, grin512, gustavedore, happy512, hoppagames, laugh512, magicalinterior, nervous512, ng_deepnegative_v1_75t, PlanIt, rfktr_bwmnga, RFKTR_plastic, rosalinenobodysd15, sad512, shock512, smile512, Style-Autumn, style-empire-neg, style-empire, style-hamunaptra, Style-Moana-neg, Style-Moana, Style-NebMagic, Style-Necromancy, Style-Petal-neg, Style-Petal, Style-Psycho-neg, Style-Renaissance-neg, Style-Renaissance, style-rustmagic, Style-Winter-neg, Style-Winter, tarot512, wholesomegrandpas, wholesomegrannies, _stardeaf-greenmageddon_
Textual inversion embeddings skipped(6): 21charturnerv2, InkPunk768, inksketchcolour1subtle, SDA768, UlukInkSketch2, Zootopiav4
Model loaded in 8.4s (create model: 0.4s, apply weights to model: 1.1s, apply half(): 0.6s, move model to device: 0.9s, load textual inversion embeddings: 5.3s).
INFO:     Started server process [13156]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
ERROR:    [Errno 10048] error while attempting to bind on address ('127.0.0.1', 5173): only one usage of each socket address (protocol/network address/port) is normally permitted
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
no config file: E:\Documents\AI\stable-diffusion-webui\extensions\Stable-Diffusion-Webui-Prompt-Translator\prompt_translator.cfg
CUDA SETUP: Loading binary E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cudaall.dll...
[text2prompt] Loading database with name "all-mpnet-base-v2 : danbooru_strict"...
[text2prompt] Database loaded
Running on local URL:  http://127.0.0.1:7861

To create a public link, set `share=True` in `launch()`.
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\anyio\streams\memory.py", line 94, in receive
    return self.receive_nowait()
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\anyio\streams\memory.py", line 89, in receive_nowait
    raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 77, in call_next
    message = await recv_stream.receive()
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\anyio\streams\memory.py", line 114, in receive
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 407, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\fastapi\applications.py", line 271, in __call__
    await super().__call__(scope, receive, send)
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\applications.py", line 125, in __call__
    await self.middleware_stack(scope, receive, send)
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 104, in __call__
    response = await self.dispatch_func(request, call_next)
  File "E:\Documents\AI\stable-diffusion-webui\modules\api\api.py", line 96, in log_and_time
    res: Response = await call_next(req)
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 80, in call_next
    raise app_exc
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 69, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\gzip.py", line 24, in __call__
    await responder(scope, receive, send)
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\gzip.py", line 44, in __call__
    await self.app(scope, receive, self.send_with_gzip)
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in __call__
    raise e
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 706, in __call__
    await route.handle(scope, receive, send)
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 69, in app
    await response(scope, receive, send)
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 334, in __call__
    raise RuntimeError(f"File at path {self.path} does not exist.")
RuntimeError: File at path E:\Documents\AI\stable-diffusion-webui\static\background.png does not exist.
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 337, in run_predict
    output = await app.get_blocks().process_api(
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1018, in process_api
    data = self.postprocess_data(fn_index, result["prediction"], state)
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 935, in postprocess_data
    if predictions[i] is components._Keywords.FINISHED_ITERATING:
IndexError: tuple index out of range
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 337, in run_predict
    output = await app.get_blocks().process_api(
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1018, in process_api
    data = self.postprocess_data(fn_index, result["prediction"], state)
  File "E:\Documents\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 935, in postprocess_data
    if predictions[i] is components._Keywords.FINISHED_ITERATING:
IndexError: tuple index out of range
ModelScope text2video extension for auto1111 webui
Git commit: 9f9bd657 (Fri Mar 24 22:49:32 2023)
Starting text2video
Pipeline setup
config namespace(framework='pytorch', task='text-to-video-synthesis', model={'type': 'latent-text-to-video-synthesis', 'model_args': {'ckpt_clip': 'open_clip_pytorch_model.bin', 'ckpt_unet': 'text2video_pytorch_model.pth', 'ckpt_autoencoder': 'VQGAN_autoencoder.pth', 'max_frames': 16, 'tiny_gpu': 1}, 'model_cfg': {'unet_in_dim': 4, 'unet_dim': 320, 'unet_y_dim': 768, 'unet_context_dim': 1024, 'unet_out_dim': 4, 'unet_dim_mult': [1, 2, 4, 4], 'unet_num_heads': 8, 'unet_head_dim': 64, 'unet_res_blocks': 2, 'unet_attn_scales': [1, 0.5, 0.25], 'unet_dropout': 0.1, 'temporal_attention': 'True', 'num_timesteps': 1000, 'mean_type': 'eps', 'var_type': 'fixed_small', 'loss_type': 'mse'}}, pipeline={'type': 'latent-text-to-video-synthesis'})
device cuda
Working in txt2vid mode
latents torch.Size([1, 4, 24, 32, 32]) tensor(0.0032, device='cuda:0') tensor(1.0001, device='cuda:0')
DDIM sampling tensor(1): 100%|█████████████████████████████████████████████████████████| 31/31 [00:18<00:00,  1.64it/s]
STARTING VAE ON GPU. 24 CHUNKS TO PROCESS
VAE HALVED
DECODING FRAMES
VAE FINISHED
torch.Size([24, 3, 256, 256])
output/mp4s/20230331_044231808812.mp4
  0%|                                                                                            | 0/1 [00:00<?, ?it/s]latents torch.Size([1, 4, 24, 32, 32]) tensor(-0.0048, device='cuda:0') tensor(1.0027, device='cuda:0')
DDIM sampling tensor(1): 100%|█████████████████████████████████████████████████████████| 31/31 [00:17<00:00,  1.77it/s]
STARTING VAE ON GPU. 24 CHUNKS TO PROCESS██████████████████████████████████████████████| 31/31 [00:17<00:00,  1.77it/s]
VAE HALVED
DECODING FRAMES
VAE FINISHED
torch.Size([24, 3, 256, 256])
output/mp4s/20230331_044258693495.mp4
text2video finished, saving frames to E:\Documents\AI\stable-diffusion-webui\outputs/img2img-images\text2video-modelscope\20230331044134
Got a request to stitch frames to video using FFmpeg.
Frames:
E:\Documents\AI\stable-diffusion-webui\outputs/img2img-images\text2video-modelscope\20230331044134\%06d.png
To Video:
E:\Documents\AI\stable-diffusion-webui\outputs/img2img-images\text2video-modelscope\20230331044134\vid.mp4
Stitching *video*...
Stitching *video*...
Video stitching done in 0.60 seconds!
t2v complete, result saved at E:\Documents\AI\stable-diffusion-webui\outputs/img2img-images\text2video-modelscope\20230331044134
ModelScope text2video extension for auto1111 webui
Git commit: 9f9bd657 (Fri Mar 24 22:49:32 2023)
Starting text2video
Pipeline setup
config namespace(framework='pytorch', task='text-to-video-synthesis', model={'type': 'latent-text-to-video-synthesis', 'model_args': {'ckpt_clip': 'open_clip_pytorch_model.bin', 'ckpt_unet': 'text2video_pytorch_model.pth', 'ckpt_autoencoder': 'VQGAN_autoencoder.pth', 'max_frames': 16, 'tiny_gpu': 1}, 'model_cfg': {'unet_in_dim': 4, 'unet_dim': 320, 'unet_y_dim': 768, 'unet_context_dim': 1024, 'unet_out_dim': 4, 'unet_dim_mult': [1, 2, 4, 4], 'unet_num_heads': 8, 'unet_head_dim': 64, 'unet_res_blocks': 2, 'unet_attn_scales': [1, 0.5, 0.25], 'unet_dropout': 0.1, 'temporal_attention': 'True', 'num_timesteps': 1000, 'mean_type': 'eps', 'var_type': 'fixed_small', 'loss_type': 'mse'}}, pipeline={'type': 'latent-text-to-video-synthesis'})
device cuda
Working in txt2vid mode
latents torch.Size([1, 4, 24, 32, 32]) tensor(0.0035, device='cuda:0') tensor(0.9993, device='cuda:0')
DDIM sampling tensor(1): 100%|█████████████████████████████████████████████████████████| 31/31 [00:17<00:00,  1.79it/s]
STARTING VAE ON GPU. 24 CHUNKS TO PROCESS
VAE HALVED
DECODING FRAMES
VAE FINISHED
torch.Size([24, 3, 256, 256])
output/mp4s/20230331_044453768088.mp4
  0%|                                                                                            | 0/1 [00:00<?, ?it/s]latents torch.Size([1, 4, 24, 32, 32]) tensor(-0.0033, device='cuda:0') tensor(0.9998, device='cuda:0')
DDIM sampling tensor(1): 100%|█████████████████████████████████████████████████████████| 31/31 [00:17<00:00,  1.79it/s]
STARTING VAE ON GPU. 24 CHUNKS TO PROCESS██████████████████████████████████████████████| 31/31 [00:17<00:00,  1.80it/s]
VAE HALVED
DECODING FRAMES
VAE FINISHED
torch.Size([24, 3, 256, 256])
output/mp4s/20230331_044520326138.mp4
text2video finished, saving frames to E:\Documents\AI\stable-diffusion-webui\outputs/img2img-images\text2video-modelscope\20230331044402
Got a request to stitch frames to video using FFmpeg.
Frames:
E:\Documents\AI\stable-diffusion-webui\outputs/img2img-images\text2video-modelscope\20230331044402\%06d.png
To Video:
E:\Documents\AI\stable-diffusion-webui\outputs/img2img-images\text2video-modelscope\20230331044402\vid.mp4
Stitching *video*...
Stitching *video*...
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\video_audio_utils.py", line 147, in ffmpeg_stitch_video
    process = subprocess.Popen(
  File "C:\Python310\lib\subprocess.py", line 969, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Python310\lib\subprocess.py", line 1438, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
PermissionError: [WinError 5] Access is denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope-text2vid.py", line 174, in process
    ffmpeg_stitch_video(ffmpeg_location=ffmpeg_location, fps=fps, outmp4_path=outdir_current + os.path.sep + f"vid.mp4", imgs_path=os.path.join(outdir_current,
  File "E:\Documents\AI\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\video_audio_utils.py", line 158, in ffmpeg_stitch_video
    raise Exception(
Exception: Error stitching frames to video. Actual runtime error:[WinError 5] Access is denied
Exception occurred: Error stitching frames to video. Actual runtime error:[WinError 5] Access is denied
ModelScope text2video extension for auto1111 webui
Git commit: 9f9bd657 (Fri Mar 24 22:49:32 2023)
Starting text2video
Pipeline setup
config namespace(framework='pytorch', task='text-to-video-synthesis', model={'type': 'latent-text-to-video-synthesis', 'model_args': {'ckpt_clip': 'open_clip_pytorch_model.bin', 'ckpt_unet': 'text2video_pytorch_model.pth', 'ckpt_autoencoder': 'VQGAN_autoencoder.pth', 'max_frames': 16, 'tiny_gpu': 1}, 'model_cfg': {'unet_in_dim': 4, 'unet_dim': 320, 'unet_y_dim': 768, 'unet_context_dim': 1024, 'unet_out_dim': 4, 'unet_dim_mult': [1, 2, 4, 4], 'unet_num_heads': 8, 'unet_head_dim': 64, 'unet_res_blocks': 2, 'unet_attn_scales': [1, 0.5, 0.25], 'unet_dropout': 0.1, 'temporal_attention': 'True', 'num_timesteps': 1000, 'mean_type': 'eps', 'var_type': 'fixed_small', 'loss_type': 'mse'}}, pipeline={'type': 'latent-text-to-video-synthesis'})
device cuda
Working in txt2vid mode
latents torch.Size([1, 4, 24, 32, 32]) tensor(0.0019, device='cuda:0') tensor(0.9985, device='cuda:0')
DDIM sampling tensor(1): 100%|█████████████████████████████████████████████████████████| 31/31 [00:17<00:00,  1.78it/s]
STARTING VAE ON GPU. 24 CHUNKS TO PROCESS
VAE HALVED
DECODING FRAMES
VAE FINISHED
torch.Size([24, 3, 256, 256])
output/mp4s/20230331_044710418252.mp4
  0%|                                                                                            | 0/1 [00:00<?, ?it/s]latents torch.Size([1, 4, 24, 32, 32]) tensor(0.0051, device='cuda:0') tensor(1.0006, device='cuda:0')
DDIM sampling tensor(1): 100%|█████████████████████████████████████████████████████████| 31/31 [00:17<00:00,  1.77it/s]
STARTING VAE ON GPU. 24 CHUNKS TO PROCESS██████████████████████████████████████████████| 31/31 [00:17<00:00,  1.79it/s]
VAE HALVED
DECODING FRAMES
VAE FINISHED
torch.Size([24, 3, 256, 256])
output/mp4s/20230331_044737439069.mp4
text2video finished, saving frames to E:\Documents\AI\stable-diffusion-webui\outputs/img2img-images\text2video-modelscope\20230331044618
Got a request to stitch frames to video using FFmpeg.
Frames:
E:\Documents\AI\stable-diffusion-webui\outputs/img2img-images\text2video-modelscope\20230331044618\%06d.png
To Video:
E:\Documents\AI\stable-diffusion-webui\outputs/img2img-images\text2video-modelscope\20230331044618\vid.mp4
Stitching *video*...
Stitching *video*...
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\video_audio_utils.py", line 147, in ffmpeg_stitch_video
    process = subprocess.Popen(
  File "C:\Python310\lib\subprocess.py", line 969, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Python310\lib\subprocess.py", line 1438, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
PermissionError: [WinError 5] Access is denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope-text2vid.py", line 174, in process
    ffmpeg_stitch_video(ffmpeg_location=ffmpeg_location, fps=fps, outmp4_path=outdir_current + os.path.sep + f"vid.mp4", imgs_path=os.path.join(outdir_current,
  File "E:\Documents\AI\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\video_audio_utils.py", line 158, in ffmpeg_stitch_video
    raise Exception(
Exception: Error stitching frames to video. Actual runtime error:[WinError 5] Access is denied
Exception occurred: Error stitching frames to video. Actual runtime error:[WinError 5] Access is denied
ModelScope text2video extension for auto1111 webui
Git commit: 9f9bd657 (Fri Mar 24 22:49:32 2023)
Starting text2video
Pipeline setup
config namespace(framework='pytorch', task='text-to-video-synthesis', model={'type': 'latent-text-to-video-synthesis', 'model_args': {'ckpt_clip': 'open_clip_pytorch_model.bin', 'ckpt_unet': 'text2video_pytorch_model.pth', 'ckpt_autoencoder': 'VQGAN_autoencoder.pth', 'max_frames': 16, 'tiny_gpu': 1}, 'model_cfg': {'unet_in_dim': 4, 'unet_dim': 320, 'unet_y_dim': 768, 'unet_context_dim': 1024, 'unet_out_dim': 4, 'unet_dim_mult': [1, 2, 4, 4], 'unet_num_heads': 8, 'unet_head_dim': 64, 'unet_res_blocks': 2, 'unet_attn_scales': [1, 0.5, 0.25], 'unet_dropout': 0.1, 'temporal_attention': 'True', 'num_timesteps': 1000, 'mean_type': 'eps', 'var_type': 'fixed_small', 'loss_type': 'mse'}}, pipeline={'type': 'latent-text-to-video-synthesis'})
device cuda
Working in txt2vid mode
latents torch.Size([1, 4, 24, 32, 32]) tensor(0.0024, device='cuda:0') tensor(0.9942, device='cuda:0')
DDIM sampling tensor(1): 100%|█████████████████████████████████████████████████████████| 31/31 [00:17<00:00,  1.80it/s]
STARTING VAE ON GPU. 24 CHUNKS TO PROCESS
VAE HALVED
DECODING FRAMES
VAE FINISHED
torch.Size([24, 3, 256, 256])
output/mp4s/20230331_045203363720.mp4
  0%|                                                                                            | 0/1 [00:00<?, ?it/s]latents torch.Size([1, 4, 24, 32, 32]) tensor(-0.0020, device='cuda:0') tensor(1.0070, device='cuda:0')
DDIM sampling tensor(1): 100%|█████████████████████████████████████████████████████████| 31/31 [00:17<00:00,  1.78it/s]
STARTING VAE ON GPU. 24 CHUNKS TO PROCESS██████████████████████████████████████████████| 31/31 [00:17<00:00,  1.79it/s]
VAE HALVED
DECODING FRAMES
VAE FINISHED
torch.Size([24, 3, 256, 256])
output/mp4s/20230331_045229893721.mp4
text2video finished, saving frames to E:\Documents\AI\stable-diffusion-webui\outputs/img2img-images\text2video-modelscope\20230331045111
Got a request to stitch frames to video using FFmpeg.
Frames:
E:\Documents\AI\stable-diffusion-webui\outputs/img2img-images\text2video-modelscope\20230331045111\%06d.png
To Video:
E:\Documents\AI\stable-diffusion-webui\outputs/img2img-images\text2video-modelscope\20230331045111\vid.mp4
Stitching *video*...
Stitching *video*...
Traceback (most recent call last):
  File "E:\Documents\AI\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\video_audio_utils.py", line 147, in ffmpeg_stitch_video
    process = subprocess.Popen(
  File "C:\Python310\lib\subprocess.py", line 969, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Python310\lib\subprocess.py", line 1438, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
PermissionError: [WinError 5] Access is denied

Additional information

No response

[hithereai]

Your webui version is 17 days old. Update to the latest, this bug is most probably not related to our extension.

[github-actions[bot]]

This issue has been closed due to incorrect formatting. Please address the following mistakes and reopen the issue:

• Make sure the issue title has at least 3 words.
• Provide a valid commit ID in the format 'commit id - [commit_hash]' both for the WebUI and the Extension.