search

vladmandic / automatic

SD.Next: Advanced Implementation of Stable Diffusion and other Diffusion-based generative image models
https://github.com/vladmandic/automatic
GNU Affero General Public License v3.0
5.68k stars 421 forks source link

[Issue]: Error using Text2Video script - 'in <string>' requires string as left operand, not CheckpointInfo #2661

Closed itswhateverman closed 10 months ago

itswhateverman commented 10 months ago

Issue Description

After launching with --upgrade and --reinstall flags (and --backend diffusers), I do not seem to be able to use the new Text2Video script. 

2023-12-29 11:40:05,078 | sd | DEBUG | text2video | Text2Video loading: model=<modules.sd_models.CheckpointInfo object at 0x000001F62DF14340>
2023-12-29 11:40:05,080 | sd | ERROR | call_queue | Exception: 'in <string>' requires string as left operand, not CheckpointInfo

I've tried with both SDXL1 and SD 1.5 models but the error is similar. I tried "--backend original" as well but I do not see the script at all when I do this.

I have also tried using each of the different Text2Video model types but that does not affect outcome either.

Version Platform Description

Version: app=sd.next updated=2023-12-29 hash=c77505fa url=https://github.com/vladmandic/automatic/tree/master python=3.10.6 Windows 11 Edge browser

Relevant log output

2023-12-29 11:39:24,995 | sd | INFO | launch | Starting SD.Next
2023-12-29 11:39:24,995 | sd | INFO | installer | Logger: file="Z:\vladtest\automatic\sdnext.log" level=DEBUG size=65 mode=create
2023-12-29 11:39:25,010 | sd | INFO | installer | Python 3.10.6 on Windows
2023-12-29 11:39:25,174 | sd | INFO | installer | Version: app=sd.next updated=2023-12-29 hash=c77505fa url=https://github.com/vladmandic/automatic/tree/master
2023-12-29 11:39:25,542 | sd | INFO | launch | Platform: arch=AMD64 cpu=Intel64 Family 6 Model 167 Stepping 1, GenuineIntel system=Windows release=Windows-10-10.0.22621-SP0 python=3.10.6
2023-12-29 11:39:25,543 | sd | DEBUG | installer | Setting environment tuning
2023-12-29 11:39:25,545 | sd | DEBUG | installer | Cache folder: C:\Users\evilevilbob\.cache\huggingface\hub
2023-12-29 11:39:25,551 | sd | DEBUG | installer | Torch overrides: cuda=False rocm=False ipex=False diml=False openvino=False
2023-12-29 11:39:25,552 | sd | DEBUG | installer | Torch allowed: cuda=True rocm=True ipex=True diml=True openvino=True
2023-12-29 11:39:25,555 | sd | INFO | installer | nVidia CUDA toolkit detected: nvidia-smi present
2023-12-29 11:39:25,647 | sd | DEBUG | installer | Repository update time: Fri Dec 29 11:14:09 2023
2023-12-29 11:39:25,649 | sd | INFO | launch | Startup: standard
2023-12-29 11:39:25,649 | sd | INFO | installer | Verifying requirements
2023-12-29 11:39:25,653 | sd | INFO | installer | Verifying packages
2023-12-29 11:39:25,654 | sd | INFO | installer | Verifying submodules
2023-12-29 11:39:28,453 | sd | DEBUG | installer | Submodule: extensions-builtin/sd-extension-chainner / main
2023-12-29 11:39:28,565 | sd | DEBUG | installer | Submodule: extensions-builtin/sd-extension-system-info / main
2023-12-29 11:39:28,660 | sd | DEBUG | installer | Submodule: extensions-builtin/sd-webui-agent-scheduler / main
2023-12-29 11:39:28,771 | sd | DEBUG | installer | Submodule: extensions-builtin/sd-webui-controlnet / main
2023-12-29 11:39:28,896 | sd | DEBUG | installer | Submodule: extensions-builtin/stable-diffusion-webui-images-browser / main
2023-12-29 11:39:28,999 | sd | DEBUG | installer | Submodule: extensions-builtin/stable-diffusion-webui-rembg / master
2023-12-29 11:39:29,102 | sd | DEBUG | installer | Submodule: modules/k-diffusion / master
2023-12-29 11:39:29,206 | sd | DEBUG | installer | Submodule: modules/lora / main
2023-12-29 11:39:29,312 | sd | DEBUG | installer | Submodule: wiki / master
2023-12-29 11:39:29,424 | sd | DEBUG | paths | Register paths
2023-12-29 11:39:29,523 | sd | DEBUG | installer | Installed packages: 221
2023-12-29 11:39:29,525 | sd | DEBUG | installer | Extensions all: ['Lora', 'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg']
2023-12-29 11:39:29,709 | sd | DEBUG | installer | Running extension installer: Z:\vladtest\automatic\extensions-builtin\sd-extension-system-info\install.py
2023-12-29 11:39:30,004 | sd | DEBUG | installer | Running extension installer: Z:\vladtest\automatic\extensions-builtin\sd-webui-agent-scheduler\install.py
2023-12-29 11:39:30,286 | sd | DEBUG | installer | Running extension installer: Z:\vladtest\automatic\extensions-builtin\stable-diffusion-webui-images-browser\install.py
2023-12-29 11:39:30,573 | sd | DEBUG | installer | Running extension installer: Z:\vladtest\automatic\extensions-builtin\stable-diffusion-webui-rembg\install.py
2023-12-29 11:39:30,863 | sd | DEBUG | installer | Extensions all: ['sd-dynamic-prompts']
2023-12-29 11:39:30,865 | sd | DEBUG | installer | Running extension installer: Z:\vladtest\automatic\extensions\sd-dynamic-prompts\install.py
2023-12-29 11:39:31,168 | sd | INFO | installer | Extensions enabled: ['Lora', 'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'sd-dynamic-prompts']
2023-12-29 11:39:31,170 | sd | INFO | installer | Verifying requirements
2023-12-29 11:39:31,174 | sd | DEBUG | launch | Setup complete without errors: 1703871571
2023-12-29 11:39:31,191 | sd | INFO | installer | Extension preload: {'extensions-builtin': 0.01, 'extensions': 0.0}
2023-12-29 11:39:31,192 | sd | DEBUG | launch | Starting module: <module 'webui' from 'Z:\\vladtest\\automatic\\webui.py'>
2023-12-29 11:39:31,194 | sd | INFO | launch | Command line args: ['--backend', 'diffusers', '--debug'] backend=diffusers debug=True
2023-12-29 11:39:31,195 | sd | DEBUG | launch | Env flags: []
2023-12-29 11:39:34,477 | sd | INFO | loader | Load packages: torch=2.1.2+cu121 diffusers=0.25.0 gradio=3.43.2
2023-12-29 11:39:35,061 | sd | DEBUG | shared | Read: file="config.json" json=46 bytes=2029 time=0.000
2023-12-29 11:39:35,073 | sd | DEBUG | shared | Unknown settings: ['dp_write_raw_template', 'depthmap_script_extra_stereomodes', 'queue_paused', 'diffusers_lora_loader']
2023-12-29 11:39:35,073 | sd | INFO | shared | Engine: backend=Backend.DIFFUSERS compute=cuda mode=no_grad device=cuda cross-optimization="Scaled-Dot-Product"
2023-12-29 11:39:35,144 | sd | INFO | shared | Device: device=NVIDIA GeForce RTX 3080 Ti n=1 arch=sm_90 cap=(8, 6) cuda=12.1 cudnn=8801 driver=546.17
2023-12-29 11:39:37,227 | sd | DEBUG | webui | Entering start sequence
2023-12-29 11:39:37,230 | sd | DEBUG | webui | Initializing
2023-12-29 11:39:37,233 | sd | INFO | sd_vae | Available VAEs: path="models\VAE" items=0
2023-12-29 11:39:37,234 | sd | INFO | extensions | Disabled extensions: ['sd-webui-controlnet']
2023-12-29 11:39:37,236 | sd | DEBUG | modelloader | Scanning diffusers cache: models\Diffusers models\Diffusers items=1 time=0.00
2023-12-29 11:39:37,447 | sd | DEBUG | shared | Read: file="metadata.json" json=5364 bytes=73638412 time=0.209
2023-12-29 11:39:37,464 | sd | DEBUG | shared | Read: file="cache.json" json=2 bytes=1268 time=0.000
2023-12-29 11:39:37,478 | sd | INFO | sd_models | Available models: path="models\Stable-diffusion" items=66 time=0.24
2023-12-29 11:39:37,636 | sd | DEBUG | webui | Load extensions
2023-12-29 11:39:37,839 | sd | ERROR | faceid | FaceID: No module named 'insightface'
2023-12-29 11:39:37,853 | sd | INFO | script_loading | Extension: script='scripts\faceid.py' 11:39:37-839690 ERROR    FaceID: No module named 'insightface'
2023-12-29 11:39:39,335 | sd | INFO | script_loading | Extension: script='extensions-builtin\sd-webui-agent-scheduler\scripts\task_scheduler.py' Using sqlite file: extensions-builtin\sd-webui-agent-scheduler\task_scheduler.sqlite3
2023-12-29 11:39:40,314 | sd | INFO | webui | Extensions time: 2.68 { automatic=0.24 Lora=1.19 sd-webui-agent-scheduler=0.23 stable-diffusion-webui-images-browser=0.19 stable-diffusion-webui-rembg=0.76 }
2023-12-29 11:39:40,357 | sd | DEBUG | shared | Read: file="html/upscalers.json" json=4 bytes=2672 time=0.000
2023-12-29 11:39:40,359 | sd | DEBUG | shared | Read: file="extensions-builtin\sd-extension-chainner\models.json" json=24 bytes=2719 time=0.000
2023-12-29 11:39:40,361 | sd | DEBUG | chainner_model | chaiNNer models: path="models\chaiNNer" defined=24 discovered=0 downloaded=0
2023-12-29 11:39:40,364 | sd | DEBUG | modelloader | Load upscalers: total=52 downloaded=0 user=0 time=0.05 ['None', 'Lanczos', 'Nearest', 'ChaiNNer', 'ESRGAN', 'LDSR', 'RealESRGAN', 'SCUNet', 'SD', 'SwinIR']
2023-12-29 11:39:40,380 | sd | DEBUG | styles | Load styles: folder="models\styles" items=288 time=0.01
2023-12-29 11:39:40,383 | sd | DEBUG | webui | Creating UI
2023-12-29 11:39:40,385 | sd | INFO | theme | Load UI theme: name="black-teal" style=Auto base=sdnext.css
2023-12-29 11:39:40,420 | sd | DEBUG | shared | Read: file="html\reference.json" json=31 bytes=16496 time=0.000
2023-12-29 11:39:44,377 | sd | DEBUG | ui_extra_networks | Extra networks: page='model' items=97 subfolders=3 tab=txt2img folders=['models\\Stable-diffusion', 'models\\Diffusers', 'models\\Reference'] list=0.03 desc=0.01 info=0.00 workers=2
2023-12-29 11:39:44,398 | sd | DEBUG | ui_extra_networks | Extra networks: page='style' items=288 subfolders=2 tab=txt2img folders=['models\\styles', 'html'] list=0.01 desc=0.00 info=0.00 workers=2
2023-12-29 11:39:44,401 | sd | DEBUG | ui_extra_networks | Extra networks: page='embedding' items=0 subfolders=1 tab=txt2img folders=['models\\embeddings'] list=0.00 desc=0.00 info=0.00 workers=2
2023-12-29 11:39:44,403 | sd | DEBUG | ui_extra_networks | Extra networks: page='hypernetwork' items=0 subfolders=1 tab=txt2img folders=['models\\hypernetworks'] list=0.00 desc=0.00 info=0.00 workers=2
2023-12-29 11:39:44,405 | sd | DEBUG | ui_extra_networks | Extra networks: page='vae' items=0 subfolders=1 tab=txt2img folders=['models\\VAE'] list=0.00 desc=0.00 info=0.00 workers=2
2023-12-29 11:39:44,637 | sd | DEBUG | ui_extra_networks | Extra networks: page='lora' items=5046 subfolders=1 tab=txt2img folders=['models\\Lora', 'models\\LyCORIS'] list=3.93 desc=1.38 info=1.61 workers=2
2023-12-29 11:39:44,782 | sd | DEBUG | ui_control | Control initialize: models=models\control
2023-12-29 11:39:44,925 | sd | DEBUG | shared | Read: file="ui-config.json" json=57 bytes=4031 time=0.000
2023-12-29 11:39:45,036 | sd | DEBUG | theme | Themes: builtin=9 default=5 external=55
2023-12-29 11:39:45,417 | sd | DEBUG | script_callbacks | Script: 0.32 ui_tabs Z:\vladtest\automatic\extensions-builtin\stable-diffusion-webui-images-browser\scripts\image_browser.py
2023-12-29 11:39:45,434 | sd | DEBUG | shared | Read: file="Z:\vladtest\automatic\html\extensions.json" json=335 bytes=197161 time=0.000
2023-12-29 11:39:46,057 | sd | DEBUG | ui_extensions | Extension list: processed=319 installed=8 enabled=7 disabled=1 visible=319 hidden=0
2023-12-29 11:39:46,802 | sd | INFO | webui | Local URL: http://127.0.0.1:7860/
2023-12-29 11:39:46,803 | sd | DEBUG | webui | Gradio functions: registered=1993
2023-12-29 11:39:46,805 | sd | INFO | middleware | Initializing middleware
2023-12-29 11:39:46,810 | sd | DEBUG | webui | Creating API
2023-12-29 11:39:46,950 | sd | INFO | task_runner | [AgentScheduler] Runner is paused
2023-12-29 11:39:46,952 | sd | INFO | api | [AgentScheduler] Registering APIs
2023-12-29 11:39:47,057 | sd | DEBUG | webui | Scripts setup: ['Dynamic Prompts v2.17.1:0.014']
2023-12-29 11:39:47,058 | sd | DEBUG | sd_models | Model metadata: file="metadata.json" no changes
2023-12-29 11:39:47,060 | sd | DEBUG | webui | Model auto load disabled
2023-12-29 11:39:47,072 | sd | DEBUG | shared | Save: file="config.json" json=46 bytes=1958 time=0.011
2023-12-29 11:39:47,081 | sd | DEBUG | shared | Unused settings: ['depthmap_script_extra_stereomodes', 'diffusers_lora_loader']
2023-12-29 11:39:47,082 | sd | INFO | webui | Startup time: 15.86 { torch=2.48 gradio=0.77 libraries=2.75 extensions=2.68 models=0.24 face-restore=0.16 upscalers=0.05 ui-extra-networks=4.33 ui-settings=0.18 ui-extensions=0.98 ui-defaults=0.25 launch=0.49 api=0.07 app-started=0.18 }
2023-12-29 11:39:56,953 | sd | DEBUG | shared | Model requested: fn=txt2img
2023-12-29 11:39:56,955 | sd | INFO | sd_models | Select: model="custom9 [93006354c8]"
2023-12-29 11:39:56,957 | sd | DEBUG | sd_models | Load model weights: existing=False target=Z:\vladtest\automatic\models\Stable-diffusion\custom9.safetensors info=None
2023-12-29 11:39:57,094 | sd | DEBUG | devices | Desired Torch parameters: dtype=FP16 no-half=False no-half-vae=False upscast=False
2023-12-29 11:39:57,096 | sd | INFO | devices | Setting Torch parameters: device=cuda dtype=torch.float16 vae=torch.float16 unet=torch.float16 context=no_grad fp16=True bf16=False
2023-12-29 11:39:57,099 | sd | DEBUG | sd_models | Diffusers loading: path="Z:\vladtest\automatic\models\Stable-diffusion\custom9.safetensors"
2023-12-29 11:39:57,101 | sd | INFO | sd_models | Diffusers: model="Stable Diffusion XL" class=StableDiffusionXLPipeline file="Z:\vladtest\automatic\models\Stable-diffusion\custom9.safetensors" size=6617MB
2023-12-29 11:40:00,192 | sd | DEBUG | launch | Server: alive=True jobs=1 requests=21 uptime=25 memory=19.6/63.88 backend=Backend.DIFFUSERS state=job="txt2img" 0/-1
2023-12-29 11:40:02,117 | sd | DEBUG | sd_models | Setting model: pipeline=StableDiffusionXLPipeline config={'low_cpu_mem_usage': True, 'torch_dtype': torch.float16, 'load_connected_pipeline': True, 'variant': 'fp16', 'extract_ema': True, 'force_zeros_for_empty_prompt ': True, 'requires_aesthetics_score': False, 'use_safetensors': True}
2023-12-29 11:40:02,119 | sd | DEBUG | sd_models | Setting model VAE: name=None upcast=True
2023-12-29 11:40:02,121 | sd | DEBUG | sd_models | Setting model: enable VAE slicing
2023-12-29 11:40:02,122 | sd | DEBUG | sd_models | Setting model: enable VAE tiling
2023-12-29 11:40:04,784 | sd | INFO | textual_inversion | Load embeddings: loaded=0 skipped=0 time=0.00
2023-12-29 11:40:05,060 | sd | DEBUG | devices | gc: collected=10901 device=cuda {'ram': {'used': 7.19, 'total': 63.88}, 'gpu': {'used': 8.0, 'total': 12.0}, 'retries': 0, 'oom': 0}
2023-12-29 11:40:05,069 | sd | INFO | sd_models | Load model: time=7.83 { load=7.83 } native=1024 {'ram': {'used': 7.19, 'total': 63.88}, 'gpu': {'used': 8.0, 'total': 12.0}, 'retries': 0, 'oom': 0}
2023-12-29 11:40:05,076 | sd | DEBUG | text2video | Text2Video: model={'name': 'Potat v1', 'path': 'camenduru/potat1', 'params': [24, 1024, 576]} defaults=False frames=24, video=MP4 duration=5 loop=True pad=1 interpolate=10
2023-12-29 11:40:05,078 | sd | DEBUG | text2video | Text2Video loading: model=<modules.sd_models.CheckpointInfo object at 0x000001F62DF14340>
2023-12-29 11:40:05,080 | sd | ERROR | call_queue | Exception: 'in <string>' requires string as left operand, not CheckpointInfo
2023-12-29 11:40:05,081 | sd | ERROR | call_queue | Arguments: args=('task(frut91r8ad4prz1)', 'cute kitten playing in flowers', 'text, fake', [], 7, 2, 2, True, False, False, 1, 1, 3.5, 2.3, 0.6, 1, -1.0, -1.0, 0, 0, 0, 1600, 1024, False, 0.6, 1.5, 'RealESRGAN 4x+', True, 5, 0, 0, 0, 0, '', '', False, 4, 0.95, False, 1, 1, True, 0.6, 1, [], 4, 3, 1, 1, 0.8, 8, 64, True, False, False, 'positive', 'comma', 0, False, False, '', 'Potat v1', False, 24, 'MP4', 5, True, 1, 10, 0, '', [], 0, '', [], 0, '', [], False, True, False, False, False, False, 0, 'None', 16, 'None', 1, True, 'None', 2, True, 1, 0, True, 'none', 0.5, None, False, False, 1, False, False, False, 1.1, 1.5, 100, 0.7, False, False, True, False, False, 0, 'Gustavosta/MagicPrompt-Stable-Diffusion', '') kwargs={}
2023-12-29 11:40:05,084 | sd | ERROR | errors | gradio call: TypeError

Backend

Diffusers

Branch

Master

Model

SD-XL

Acknowledgements

itswhateverman commented 10 months ago

I believe I misunderstood how this works, as i thought the model options in the script model dropdown augmented existing checkpoints. Manually loading these models as the selected checkpoint then rerunning has been successful.