search

thygate / stable-diffusion-webui-depthmap-script

High Resolution Depth Maps for Stable Diffusion WebUI
MIT License
1.69k stars 159 forks source link

MemoryError (when trying to make a video) #457

Open petermg opened 1 month ago

petermg commented 1 month ago

ERROR: Please report this issue here. Make sure to provide the full stacktrace: Traceback (most recent call last): File "F:\SD\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\src\common_ui.py", line 473, in run_generate ret = video_mode.gen_video( File "F:\SD\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\src\video_mode.py", line 135, in gen_video fps, input_images = open_path_as_images(os.path.abspath(video.name)) File "F:\SD\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\src\video_mode.py", line 61, in open_path_as_images frames = [Image.fromarray(x) for x in list(clip.iter_frames())] File "F:\SD\stable-diffusion-webui\venv\lib\site-packages\moviepy\Clip.py", line 472, in iter_frames frame = self.get_frame(t) File "", line 2, in get_frame File "F:\SD\stable-diffusion-webui\venv\lib\site-packages\moviepy\decorators.py", line 89, in wrapper return f(*new_a, **new_kw) File "F:\SD\stable-diffusion-webui\venv\lib\site-packages\moviepy\Clip.py", line 93, in get_frame return self.make_frame(t) File "F:\SD\stable-diffusion-webui\venv\lib\site-packages\moviepy\video\io\VideoFileClip.py", line 113, in self.make_frame = lambda t: self.reader.get_frame(t) File "F:\SD\stable-diffusion-webui\venv\lib\site-packages\moviepy\video\io\ffmpeg_reader.py", line 184, in get_frame result = self.read_frame() File "F:\SD\stable-diffusion-webui\venv\lib\site-packages\moviepy\video\io\ffmpeg_reader.py", line 120, in read_frame s = self.proc.stdout.read(nbytes) MemoryError Time taken: 59.0 sec.

A: 2.05 GB, R: 2.40 GB, Sys: 4.0/23.9883 GB (16.8%)

semjon00 commented 1 month ago

Try to make a shorter video and/or free computer RAM and VRAM.

ThreeDeeJay commented 3 weeks ago

How come bigger videos need more RAM/VRAM? I'd imagine it'd just handle a few consecutive frames tops on memory when generating a video. does it not store processed frames into a cache on the disk? or does it store the whole thing in RAM to speed things up? or perhaps we'd just need to specify a smaller memory allocation and flush unused data to make room for new resources? It makes me wonder how much we'd need to convert a full feature film that's 1080p or even 4K 🤔

petermg commented 3 weeks ago

How come bigger videos need more RAM/VRAM? I'd imagine it'd just handle a few consecutive frames tops on memory when generating a video. does it not store processed frames into a cache on the disk? or does it store the whole thing in RAM to speed things up? or perhaps we'd just need to specify a smaller memory allocation and flush unused data to make room for new resources? It makes me wonder how much we'd need to convert a full feature film that's 1080p or even 4K 🤔

I wonder the same thing. It seems unnecessary to store the whole video in RAM, doesn't seem practical.

semjon00 commented 3 weeks ago

@ThreeDeeJay Soo.. That implementation is not great tbh. I believe there is code in Depth Anything 2 repository that shows how one can write frames directly to video stream, by appending a video file. Now that's really clever. But... please see the next comment.

semjon00 commented 3 weeks ago

It would take a clever programmer to fix this repo and bring Depth Anything 2 approach to this repo. Are you, the reader, up to the task?

ThreeDeeJay commented 3 weeks ago

I'm not really a programmer (I'd be more than happy to assist testing new code though) but someone with technical knowledge about what needs to be done could open an issue on the DepthAnything repo. That way one of the devs (or someone like the person who helped me here) could offer a few pointers👀🤞