search

cwilby / nextcloud-workflow-media-converter

An automated media conversion plugin for NextCloud
https://apps.nextcloud.com/apps/workflow_media_converter
GNU Affero General Public License v3.0
35 stars 15 forks source link

Add parameters that limit the number of active ffmpeg instances #465

Open cwilby opened 3 months ago

cwilby commented 3 months ago

Discussed in https://github.com/cwilby/nextcloud-workflow-media-converter/discussions/464

Originally posted by **lea-nsk** May 26, 2024 Hello. Software: Linux Debian 12; Nextcloud 29.0.1; Media conversion + ffmpeg; Hardware: CPU: 4 core; RAM: 4 Gb; Swap disk part for Linux Debian 12: 8 Gb. Recently I decided to tidy up my working video archive into a single format for further work with it in Nextcloud. I have about 5000+ video files with different file sizes. The problem at the moment of assigning a folder with many files to be transcoded into another format using Media conversion is that while processing the files, several background tasks are running, pulling off ffmpeg, which causes CPU overload and in the progression of running overflowing RAM with it, itself causing a fatal crash in the operating system. In this situation I don't like the fact that scheduled jobs can't be fully marked or paused. And also Media conversion has no functional controls for enabling/disabling/pausing a scheduled media processing job, including functions for setting limits on RAM usage and CPU load. ![Clipboard01](https://github.com/cwilby/nextcloud-workflow-media-converter/assets/78995154/f5e8b584-224a-451b-a93d-97f80b6c2a39) In my opinion, Media conversion lacks additional optional settings regarding: 1. Setting a limit on the maximum amount of parallel streaming of files through ffmpeg. 2. Ability to set a manual limit on CPU load usage (e.g. 25, 50, 75, 100%) when using ffmpeg on weak hardware servers.
cwilby commented 3 months ago

Setting a limit on the maximum amount of parallel streaming of files through ffmpeg.

This field should be added. It can be used as part of a query to the jobs table to determine whether a job can be processed or should be retried later.

Ability to set a manual limit on CPU load usage (e.g. 25, 50, 75, 100%) when using ffmpeg on weak hardware servers.

It is not possible to limit the CPU load on a single thread with FFmpeg (e.g, limit 1 thread to 50% CPU usage), however it is possible to limit the number of threads used by FFmpeg, which (depending on your CPU) may translate to the number of cores that are used, which may also translate to a percentage limit.

The existing Threads value can be used in this case, it detects the number of cores so you can effectively set a "percentage". I don't think this should be changed to an actual percentage control given that may not always be accurate given the CPU architecture.

image