search

mltframework / mlt

MLT Multimedia Framework
https://www.mltframework.org
GNU Lesser General Public License v2.1
1.48k stars 314 forks source link

[4k 30fps] OOM after 15m while rendering hevc_nvenc profile #1005

Open Hideman85 opened 2 months ago

Hideman85 commented 2 months ago

Dear all,

I'm posting this here cause I havent found a wiki to discuss about it.

I am running Ubuntu 24.04 (Noble), with kdenlive 23.08.5 for edinting and rendering in command line with melt 7.22.0. My project is 4k 30fps as most of my shoots follow this property.

I am facing huge memory issue (probably memory leaks), the first come with editing in kdenlive after 45min kdenlive have eaten the full of my 32G and have to restart it to work again with it. Now for rendering, as kdenlive eat all my ram, I do close kdenlive and use rendering scrip with melt command line so melt is free to use the 32G of ram. Now my project is about 40min and when I do render the project it crash at around 15min with OOM after eating my 32G of ram. I have plenty of different clips, both videos and images and basic effects like fading, zoom and gaussian bluring. I have some issue to understand why it would lead to OOM since the clips can be unloaded after use.

I am using right now a custom profile

ab=700k acodec=flac channels=2 ar=44100 sample_fmt=s16 color_range=pc f=matroska hwaccel=vaapi hwaccel_output_format=vaapi real_time=-1 threads=0 vaapi_device=/dev/dri/renderD128 vb=70000k vcodec=hevc_nvenc

I am not sure if this is the cause of my OOM, as you can see I removed multi-threading so they do not grow exponentially on their own. My hardware is pretty recent and should follow for this project. (AMD Ryzen 9 7940HS + Nvidia RTX 2070 + 32G LPDDR5x 7400mhz)

I you have any advice to solve my problem or even making myself more efficient, I am welcome and would be very greatful.

AlxHnr commented 1 month ago

This doesn't seem to be limited to hevc_nvenc. With other codecs, even just lossless, I'm observing similar behavior trough KDEnlive. The memory usage of the melt process just keeps growing and growing linearly, until it gets OOM killed by the kernel. 4k@60hz project here, with memory usage surpassing 52 GB after ~35 minutes of rendering. Memory leakages seem to be a recurring theme when skimming trough previous GitHub issues.

Hideman85 commented 1 month ago

The only solution I found from other topics to work with melt, is to temporarily create a swapfile of 64G, then it works and compile the whole project.

AlxHnr commented 1 month ago

Or export the video in smaller sections, plus a separate audio file. Then stitch it together with ffmpeg. Has the upside that a small project change won't need a complete rerender.

Hideman85 commented 1 month ago

Yes that is one perspective they want to move to instead of parallel rendering, there is few topics about that already: https://invent.kde.org/multimedia/kdenlive/-/issues/1868 https://invent.kde.org/multimedia/kdenlive/-/issues/439