Question: why are we not using full CPU during h264 encoding?

[nesteruk]

What would you like to submit? (put an 'x' inside the bracket that applies)

• [x] question
• [ ] bug report
• [ ] feature request

Issue description

I'm using a VideoFileWriter to make videos that use either the H264 or MPEG codecs. What I find is either encoding process fails to use 100% of the available CPU power. On a quad-core, the usage figure is around 30%.

Now, from what I understand (see e.g. this question), FFMPEG is meant to use an optimal number of threads for encoding. So I was thinking, maybe there is a flag or setting somewhere that I need to apply in Accord.Video.FFMPEG to enable this in my application?

Actually, a related question is how I can pass FFMPEG parameters if I want to. For example, I was thinking of using a custom ffmpeg build with NVENC, but I'm not sure how to specify through Accord which encoder to use.

[cesarsouza]

Hi @nesteruk,

If you are using the latest pre-release version from NuGet (3.8.2-alpha), it should be possible to pass parameters to FFmpeg through the VideoOptions dictionary that is now present as a property of the VideoFileWriter class.

For the moment Accord is just relaying frames to FFmpeg in the simplest (but as correct) manner possible. I need to check whether it is necessary to inform FFmpeg somehow that we plan to use multiple threads with a particular codec when first initializing the library, but for now you can try to set the following option before opening your video:

writer.VideoOptions["threads"] = "8";

A list of other options can be found here. The ones related to multi-threading are at the end of the page.

Hope it helps, Cesar

[nesteruk]

I have just got the Alpha build and plugged in the "threads" setting. My CPU is now using around 45-50% of its power, and no matter what value I set, I cannot get it to use more. This is when using the H264 encoding. On Mpeg4, I get performance in the 30-40% range.

[cesarsouza]

Hi @nesteruk,

I am supposing that this is happening because FFmpeg is processing and saving the frames faster than we can feed frames to it. The problem may be either that the process that is loading the frames is too slow, or that the bottleneck is actually in the video rescaling phase which is not done by x264 but rather by the swscale of FFmpeg and is done in a single thread for now.

How are the frames being loaded into the application? Are you using the VideoFileReader, VideoCaptureDevice or ScreenCaptureStream classes?

Regards, Cesar

[nesteruk]

It typically goes like this:

• I render a frame into a WIC bitmap.
• The WIC bitmap is copied into an ordinary System.Drawing.Bitmap.
• I call WriteVideoFrame()

This sequence happens once for every frame and is blocking.

On 13 November 2017 at 22:06, César Souza notifications@github.com wrote:

Hi @nesteruk https://github.com/nesteruk,

I am supposing that this is happening because FFmpeg is processing and saving the frames faster than we can feed frames to it. The problem may be either that the process that is loading the frames is too slow, or that the bottleneck is actually in the video rescaling phase which is not done by x264 but rather by the swscale of FFmpeg and is done in a single thread for now.

How are the frames being loaded into the application? Are you using the VideoFileReader http://accord-framework.net/docs/html/T_Accord_Video_FFMPEG_VideoFileReader.htm, VideoCaptureDevice http://accord-framework.net/docs/html/T_Accord_Video_DirectShow_VideoCaptureDevice.htm or ScreenCaptureStream http://accord-framework.net/docs/html/T_Accord_Video_ScreenCaptureStream.htm classes?

Regards, Cesar

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/accord-net/framework/issues/1022#issuecomment-344023622, or mute the thread https://github.com/notifications/unsubscribe-auth/AAcoNsA4LJL7i8KPldEBHO_cxt9GUxPsks5s2JM0gaJpZM4QV7DU .

-- Dmitri Nesteruk dmitrinesteruk@gmail.com skype: dmitri.nesteruk

[cesarsouza]

Hi @nesteruk,

If you would like to experiment to see if the bottleneck is in the data loading process, you can try calling WriteVideoFrame in a loop using always the same Bitmap. Theoretically this should work (I haven't tried it yet), and it should make FFmpeg work as fast as it can. If even after this you are not seeing your CPU spike to more than 50%, then indeed the problem is in Accord or FFmpeg.

If the CPU usage does increase, then it means the bottleneck is in the data loading process. What you can do to accelerate things then is to create a parallel pipeline with 2 images where you are always passing one image to FFmpeg and writing to the other image (i.e. copying from the WIC) at the same time. Then you keep alternating between those two. Something similar is done in the ScreeenCaptureStream class.

Regards, Cesar