[Bug]: Exception occurred: 'unet_dim'

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

Are you using the latest version of the extension?

[X] I have the modelscope text2video extension updated to the lastest version and I still have the issue.

What happened?

I get this error when I click Generate:

Git commit: 066a9e13 (Sun Mar 26 15:10:21 2023) Starting text2video Pipeline setup config namespace(framework='pytorch', task='text-to-video-synthesis', model={'type': 'latent-text-to-video-synthesis', 'model_args': {'ckpt_clip': 'open_clip_pytorch_model.bin', 'ckpt_unet': 'text2video_pytorch_model.pth', 'ckpt_autoencoder': 'VQGAN_autoencoder.pth', 'max_frames': 16, 'tiny_gpu': 1}, 'model_cfg': {'unet_dim': 320, 'unet_y_dim': 768, 'unet_context_dim': 1024, 'unet_out_dim': 4, 'unet_dim_mult': [1, 2, 4, 4], 'unet_num_heads': 8, 'unet_head_dim': 64, 'unet_res_blocks': 2, 'unet_attn_scales': [1, 0.5, 0.25], 'unet_dropout': 0.1, 'temporal_attention': 'True', 'num_timesteps': 1000, 'mean_type': 'eps', 'var_type': 'fixed_small', 'loss_type': 'mse'}}, pipeline={'type': 'latent-text-to-video-synthesis'}) Traceback (most recent call last): File "C:\A1111 Web UI Autoinstaller\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope-text2vid.py", line 74, in process pipe = setup_pipeline() File "C:\A1111 Web UI Autoinstaller\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope-text2vid.py", line 30, in setup_pipeline return TextToVideoSynthesis(ph.models_path+'/ModelScope/t2v') File "C:\A1111 Web UI Autoinstaller\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\t2v_pipeline.py", line 71, in init in_dim=cfg['unet_in_dim'], KeyError: 'unet_in_dim' Exception occurred: 'unet_in_dim'

It seems there is something missing in the configuratuon.json, but its identical to the version on hugginface.

Steps to reproduce the problem

Go to ....
Press ....
...

What should have happened?

No response

WebUI and Deforum extension Commit IDs

webui commit id - txt2vid commit id -

What GPU were you using for launching?

3060

On which platform are you launching the webui backend with the extension?

No response

Settings

default

Console logs

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  Loading A111 WebUI Launcher
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 i   Settings file found, loading
 →   Updating Settings File  ✓
 i   Launcher Version 1.7.0
 i   Found a custom WebUI Config
 i   No Launcher launch options
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 →   Checking requirements :
 i   Python 3.10.6150.1013 found in registry:  C:\Users\User\AppData\Local\Programs\Python\Python310\
 i   Clearing PATH of any mention of Python
 →   Adding python 3.10 to path  ✓
 i   Git found and already in PATH:  C:\Program Files\Git\cmd\git.exe
 i   Automatic1111 SD WebUI found:  C:\A1111 Web UI Autoinstaller\stable-diffusion-webui
 i   One or more checkpoint models were found
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  Loading Complete, opening launcher
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 i   No arguments set
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 ↺   Updating Webui
remote: Enumerating objects: 314, done.
remote: Counting objects: 100% (246/246), done.
remote: Compressing objects: 100% (55/55), done.
remote: Total 314 (delta 212), reused 211 (delta 191), pack-reused 68Receiving objects:  90% (283/314)
Receiving objects: 100% (314/314), 127.80 KiB | 25.56 MiB/s, done.
Resolving deltas: 100% (215/215), completed with 68 local objects.
From https://github.com/AUTOMATIC1111/stable-diffusion-webui
   c1294d84..955df775  master       -> origin/master
 * [new branch]        lora_inplace -> origin/lora_inplace
 ✓   Done
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 ↺   Updating Extension: sd-webui-modelscope-text2video
remote: Enumerating objects: 227, done.
remote: Counting objects: 100% (215/215), done.
remote: Compressing objects: 100% (75/75), done.
remote: Total 227 (delta 139), reused 206 (delta 138), pack-reused 12
Receiving objects: 100% (227/227), 101.65 KiB | 8.47 MiB/s, done.
Resolving deltas: 100% (143/143), completed with 9 local objects.
From https://github.com/deforum-art/sd-webui-modelscope-text2video
   fddb4e8..066a9e1  main       -> origin/main
   02a35bf..28e5b56  extras     -> origin/extras
 ✓   Done
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 ↺   Updating Extension: sd-webui-modelscope-text2video
 ✓   Done
 i   No arguments set
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  WEBUI LAUNCHING VIA EMS LAUNCHER, EXIT THIS WINDOW TO STOP THE WEBUI
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 !   Any error happening after 'commit hash : XXXX' is not related to the launcher. Please report them on Automatic1111's github instead :
 ☁   https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/new/choose
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Cancel
venv "C:\A1111 Web UI Autoinstaller\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Commit hash: 955df7751eef11bb7697e2d77f6b8a6226b21e13
Installing requirements for Web UI

Launching Web UI with arguments: --autolaunch
No module 'xformers'. Proceeding without it.
Loading weights [27a4ac756c] from C:\A1111 Web UI Autoinstaller\stable-diffusion-webui\models\Stable-diffusion\SD15NewVAEpruned.ckpt
Creating model from config: C:\A1111 Web UI Autoinstaller\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying cross attention optimization (Doggettx).
Textual inversion embeddings loaded(0):
Model loaded in 3.9s (load weights from disk: 1.1s, create model: 0.3s, apply weights to model: 0.4s, apply half(): 0.5s, move model to device: 0.5s, load textual inversion embeddings: 1.2s).
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 12.3s (import torch: 2.5s, import gradio: 1.9s, import ldm: 0.7s, other imports: 1.9s, load scripts: 0.8s, load SD checkpoint: 3.9s, create ui: 0.3s, gradio launch: 0.3s).
ModelScope text2video extension for auto1111 webui
Git commit: 066a9e13 (Sun Mar 26 15:10:21 2023)
Starting text2video
Pipeline setup
config namespace(framework='pytorch', task='text-to-video-synthesis', model={'type': 'latent-text-to-video-synthesis', 'model_args': {'ckpt_clip': 'open_clip_pytorch_model.bin', 'ckpt_unet': 'text2video_pytorch_model.pth', 'ckpt_autoencoder': 'VQGAN_autoencoder.pth', 'max_frames': 16, 'tiny_gpu': 1}, 'model_cfg': {'unet_dim': 320, 'unet_y_dim': 768, 'unet_context_dim': 1024, 'unet_out_dim': 4, 'unet_dim_mult': [1, 2, 4, 4], 'unet_num_heads': 8, 'unet_head_dim': 64, 'unet_res_blocks': 2, 'unet_attn_scales': [1, 0.5, 0.25], 'unet_dropout': 0.1, 'temporal_attention': 'True', 'num_timesteps': 1000, 'mean_type': 'eps', 'var_type': 'fixed_small', 'loss_type': 'mse'}}, pipeline={'type': 'latent-text-to-video-synthesis'})
Traceback (most recent call last):
  File "C:\A1111 Web UI Autoinstaller\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope-text2vid.py", line 74, in process
    pipe = setup_pipeline()
  File "C:\A1111 Web UI Autoinstaller\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\modelscope-text2vid.py", line 30, in setup_pipeline
    return TextToVideoSynthesis(ph.models_path+'/ModelScope/t2v')
  File "C:\A1111 Web UI Autoinstaller\stable-diffusion-webui\extensions\sd-webui-modelscope-text2video\scripts\t2v_pipeline.py", line 71, in __init__
    in_dim=cfg['unet_in_dim'],
KeyError: 'unet_in_dim'
Exception occurred: 'unet_in_dim'

Additional information

No response

kabachuha / sd-webui-text2video