Zulko / moviepy

Video editing with Python
https://zulko.github.io/moviepy/
MIT License
12.58k stars 1.58k forks source link

Closing a video doesn't release all of the memory #1284

Open DrRonne opened 4 years ago

DrRonne commented 4 years ago

I kept running in on Memory Errors, when I use moviepy and I concatenate a bunch of clips, it just stops when the script occupies over 30% of my memory (I have 8gb RAM). Then it just says Memory Error with no further information. I could find a way to give it more memory, but the easier solution for me would be to just split all of it up into batches that won't occupy over 30% of my memory. So that's what I did, I built 2 smaller videos instead of one large video that I could later stitch together, which would in theory use up less memory. I can't confirm this however, because I still got the Memory Error. Maybe it's because the amount of footage stayed the same in the end, it's just occupying less objects now, instead of about 15 minutes of footage in 60 objects, it's now 15 minutes of footage in 2 objects. Anyway, I decided to dig into this thing a bit more because I applied close() on all the VideoFileClips, CompositeVideoClips and ImageClips that I used, but I didn't notice a real drop in Memory Usage when I use the "top" command in linux.

Expected Behavior

Memory usage should be back to the original level. My program uses 1% of my memory when I have not made any MoviePy objects yet, when I then create two objects, it goes up to 2.3% memory usage. When I call close() on the clips, I'd expect the memory usage to drop back down to 1%.

Actual Behavior

When I close the two VideoFileClips, memory usage only goes down to 2%. And this is quite a big problem, because I want the program to keep running and keep editing new videos, but there's increasingly less memory available the more VideoFileClips I import until I inevitably end up in a Memory Error.

Steps to Reproduce the Problem

The simple code I used to test this situation is the following:

from moviepy.editor import VideoFileClip

def importclip(path):
    clip1 = VideoFileClip(path)
    clip2 = clip1.resize(height=720)
    clip1.close()
    clip3 = clip2.on_color(size=(1280, 720), color=(0, 0, 0), col_opacity=0)
    clip2.close()

    return clip3

input("read")

clip1 = importclip("temp0.mp4")
clip2 = importclip("temp1.mp4")

input("close")

clip1.close()
clip2.close()
del clip1
del clip2

input("closed")

The "importclip" method used to be different. Before, it would all just be called clip instead of clip1, clip2 and clip3. I would then not close 'clip' after every effect. This made a tiny difference in memory. When I'd use the same variable for the effects and when I did not close the clip after every effect, the memory went from 1% to 2.3% to 2%. When the script is used like the one above, it goes from 1% to 2.2% to 1.9%. So overall, it's a 0.1% difference, but it wasn't the solution to the problem. It also looks weird to use it like that, is there a different way this would be recommended? I used the bug feature tag, but it might as well be a mistake on my part in this script or something that I don't understand. I couldn't find it on the internet anyway.

Specifications

FliegendeWurst commented 4 years ago

The python interpreter doesn't always release free memory back to the OS. You can use gc.collect() to force a garbage collection, which might return some memory to the OS (it does on my system, but only sometimes).