eyeEmotion
commented
3 months ago Manually cut and rendered my video file again. This time the Depth generation was able to get through the first process all the way through. But now it gave an error with the "Generating output frames" section. It was almost at the last frames, when I suddenly get this error:
Startup time: 8.1s (prepare environment: 1.9s, import torch: 2.2s, import gradio: 0.8s, setup paths: 0.7s, initialize shared: 0.2s, other imports: 0.4s, load scripts: 1.4s, create ui: 0.2s, gradio launch: 0.2s). Creating model from config: C:\AI\stable-diffusion-webui\configs\v1-inference.yaml Applying attention optimization: Doggettx... done. Model loaded in 6.2s (load weights from disk: 1.9s, create model: 0.2s, apply weights to model: 1.2s, apply half(): 1.3s, calculate empty prompt: 1.5s). Generating depthmaps for the video frames DepthMap v0.4.6 (500ee72a) device: cuda Loading model(s) .. Loading model weights from ./models/midas/dpt_beit_large_512.pt Computing output(s) .. 100%|████████████████████████████████████████████████████████████████████████████| 4320/4320 [1:06:43<00:00, 1.08it/s] Computing output(s) done. All done.
Processing generated depthmaps Generating output frames DepthMap v0.4.6 (500ee72a) device: cuda Computing output(s) .. 99%|█████████████████████████████████████████████████████████████████████████████▎| 4283/4320 [04:36<00:02, 15.47it/s] Fail.
Traceback (most recent call last): File "C:\AI\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\src\common_ui.py", line 457, in run_generate ret = video_mode.gen_video( File "C:\AI\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\src\video_mode.py", line 159, in gen_video img_results = list(core.core_generation_funnel(None, input_images, input_depths, None, inp)) File "C:\AI\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\src\core.py", line 322, in core_generation_funnel raise e File "C:\AI\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\src\core.py", line 139, in core_generation_funnel if inputdepthmaps is not None and inputdepthmaps[count] is not None: IndexError: list index out of range
It was a 3 minute video. Nothing seemed to go wrong at any hardware level, when watching in the Task Manager. Didn't have any trouble with space on my RAM or Virtual Memory.
semjon00
Hi,
I'm converting a test-video with several models, with and without Boost on. Most of the models work, although some I couldn't test out because it took too long. But when I select DPT_BEIT_LARGE_512 (still have to test if the 384 has the same problem), I first get some warnings, then it keeps on creating depthmaps per frame, When that's done, it start generating the the output, but fails at it.
Edit: Tried the 384 and that one works fine. Tried the 512 again with a different video-file, that was also bigger/longer. But again, at 13%, got the same warnings.
I'm using the deptmap within Stable Diffusion (Automatic1111 or what is it called) Here is what is outputted in the commandline:
To create a public link, set
share=Trueinlaunch(). Startup time: 28.1s (prepare environment: 9.4s, import torch: 5.2s, import gradio: 3.4s, setup paths: 3.2s, initialize shared: 0.4s, other imports: 2.2s, setup codeformer: 0.3s, load scripts: 3.6s, create ui: 0.2s, gradio launch: 0.4s). Creating model from config: D:\Documenten\stable-diffusion-webui\configs\v1-inference.yaml Applying attention optimization: Doggettx... done. Model loaded in 47.1s (load weights from disk: 36.7s, create model: 0.2s, apply weights to model: 1.2s, apply half(): 1.9s, load textual inversion embeddings: 0.2s, calculate empty prompt: 6.8s). Generating depthmaps for the video frames DepthMap v0.4.6 (500ee72a) device: cuda Loading model(s) .. Loading model weights from ./models/midas/dpt_beit_large_512.pt Computing output(s) .. 13%|█████████▉ | 222/1757 [03:24<23:09, 1.10it/s]WARNING:py.warnings:D:\Documenten\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\src\core.py:196: RuntimeWarning: invalid value encountered in subtract out = (out - out.min()) / (out.max() - out.min()) # normalize to [0; 1]WARNING:py.warnings:D:\Documenten\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\src\core.py:196: RuntimeWarning: invalid value encountered in divide out = (out - out.min()) / (out.max() - out.min()) # normalize to [0; 1]
100%|██████████████████████████████████████████████████████████████████████████████| 1757/1757 [26:14<00:00, 1.12it/s] Computing output(s) done. All done.
Processing generated depthmaps Generating output frames DepthMap v0.4.6 (500ee72a) device: cuda Computing output(s) .. 99%|█████████████████████████████████████████████████████████████████████████████ | 1737/1757 [00:23<00:00, 73.84it/s] Fail.
Traceback (most recent call last): File "D:\Documenten\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\src\common_ui.py", line 457, in run_generate ret = video_mode.gen_video( File "D:\Documenten\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\src\video_mode.py", line 159, in gen_video img_results = list(core.core_generation_funnel(None, input_images, input_depths, None, inp)) File "D:\Documenten\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\src\core.py", line 322, in core_generation_funnel raise e File "D:\Documenten\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\src\core.py", line 139, in core_generation_funnel if inputdepthmaps is not None and inputdepthmaps[count] is not None: IndexError: list index out of range
My hardware is up to the snuff, cause I even can use BOOST without my computer breaking a sweat (it just takes a long time). Got an i7-13700K, 2x16GB 3600 DDR4 RAM with an Nvidia RTX 3060 OC 12GB. (Btw, is there a setting where I can dedicate more VRAM to it. It seems to mostly run around 5GB RAM, sometimes 7GB RAM.
Also, how can I add other models to the dropdown list? For example, I also want to try 'dpt_swin_large_384.pt' and 'dpt_swin2_large_384.pt'.