search

HaveAGitGat / Tdarr

Tdarr - Distributed transcode automation using FFmpeg/HandBrake + Audio/Video library analytics + video health checking (Windows, macOS, Linux & Docker)
Other
3.04k stars 95 forks source link

Extremely high memory usage from Tdarr_Server #354

Closed Jacob-Lasky closed 3 years ago

Jacob-Lasky commented 3 years ago

Describe the bug I am running into memory issues because Tdarr is using 20+ GB of RAM.

To Reproduce Begin transcoding with CPU and GPU and wait for a long time. After 5 or so hours, I notice the high usage.

Expected behavior I expect Tdarr to use RAM when necessary, but I do not want Tdarr to use all of my server's RAM.

Screenshots Highlighted in the screenshot below is the memory usage (%) after 3 hours. My server has 32 GB of RAM so this feels like a lot. This will continue to rise until eventually my server is at 100% usage. image

Desktop (please complete the following information):

Additional context Perhaps we can have an options to limit the used memory.

After taking the screenshot and then going into the browser UI, the RAM usage dropped significantly. I don't know if this is because a job finished but the issue is still annoying.

This problem did not happen before the V2 update so I'm not sure what's going on.

jasonsansone commented 3 years ago

How many concurrent transcodes? What resolution media? I allocate ~4GB per worker for 1080p transcodes. Using 20GB is not at all unreasonable for 4+ concurrent transcode jobs of high resolution, high bitrate media.

Regarding limits: Any LXC container, KVM VM, or Docker container can easily assign memory limits. Unless you are running bare metal, you don't need this feature in Tdarr.

Jacob-Lasky commented 3 years ago

@jasonsansone - thanks for the reply!

I was doing 1 CPU transcode and 2 GPU transcodes. None of my content is 4k, but the bitrate can be pretty high (10kish). As I'm typing this, it's using 26GB of RAM for that same number of transcodes.

I completely forgot that I can limit the memory usage by using the --memory= in the extra parameters, but I appreciate that reminder!! I'll go ahead and do that now.

I think that my biggest eyebrow raise is that I wasn't seeing this type magnitude of RAM usage in the previous version.

Jacob-Lasky commented 3 years ago

Also, it seems that there is a similar problem being reported here: https://www.reddit.com/r/Tdarr/comments/lcd1ix/some_problems_on_unraid_with_v2_templates/

Namely the last paragraph which says:

I added the same Libraries as in v1.3 with same plugins/stack but it seems the v2 containers use up a lot more CPU and the memory usage of the tdarr_node container increases fast while I have the workers set to just use 1 cpu/gpu for both transcodes and health check. Also it seems the node is getting stuck and starts looping a single file more frequently then the v1.3 Tdarr. Looking into the cpu/memory use it looks like ffmpeg is increasingly using more and more CPU/memory.

HaveAGitGat commented 3 years ago

I have not heard this being discussed on Discord. I've added loop protection to 2.00.04 which might help:

https://github.com/HaveAGitGat/Tdarr/issues/345

HaveAGitGat commented 3 years ago

Has anyone encountered this since or can it be closed?

2.00.06 fixes the Tdarr_Node RAM creeping up which some were encountering. It's currently in the dev container.

jasonsansone commented 3 years ago

I am not experiencing any increase in memory usage over time. Appears resolved.

Jacob-Lasky commented 3 years ago

On 2.00.06 - things look to be resolved on my end. Thanks for the fix!

HaveAGitGat commented 3 years ago

Thanks, will close this now. I'm aware of a different server RAM issue which seems unrelated to this one.