search

unosquare / ffmediaelement

FFME: The Advanced WPF MediaElement (based on FFmpeg)
https://unosquare.github.io/ffmediaelement/
Other
1.18k stars 244 forks source link

Enhanced memory copy scenarios and possible performance improvements #446

Closed Mitra-M closed 4 years ago

Mitra-M commented 5 years ago

Issue Title (Is it possible to play the video perfectly smoothly?)

I converted sample video (attached) to many different codec/format , but FFme couldn't play any of them completely smooth. All of them played smoothly with ffplay/WPF-ME.

Issue Categories

Version Information

I've tried all new and old releases ( x32 and x64 ) , changed App priority/affinity , but same result. I think this is timing / Thread priority issue, I played with your source code but to be honest couldn't understand it exactly. (sorry for my English)

Steps to Reproduce

  1. Just download the sample file and play it by FFme sample project.

Expected Results

playing smoothly.

VideoSample.zip

mariodivece commented 5 years ago

Let me check what's going on here -- Thanks for the sample video.

zgabi commented 5 years ago

There is an unnecessarry memory copy here on the UI thread: https://github.com/unosquare/ffmediaelement/blob/e0134aad69bb8e6fe3e0d2b3dc5ae694ac878c9f/Unosquare.FFME.Windows/Rendering/VideoRenderer.cs#L335-L339

It is beause you are using WriteableBitmap to show the image. I think InteropBitmap would be a better choice. With that you can directly use sws_scale to copy the data to it's buffer (on the background thread where this call is already), and you have to call only an Invalidate on the UI thread.

You can create it with this three lines:

int stride = 4 * width;
int size = stride * height;

_fileMapping = Win32Api.CreateFileMapping(new IntPtr(-1), IntPtr.Zero, Win32Api.PageAccess.ReadWrite, 0, size, null);
_mapView = Win32Api.MapViewOfFile(_fileMapping, Win32Api.FileMapAccess.AllAccess, 0, 0, (uint)size);

ImageSource = Imaging.CreateBitmapSourceFromMemorySection(_fileMapping, width, height, PixelFormats.Bgra32, stride, 0);

(You can find the definition of the two DllImports on the internet)

And I noticed that there are someunnecessray locks for example here https://github.com/unosquare/ffmediaelement/blob/634ea3b2b0ebca65557b92d9f657cdabf7e94a38/Unosquare.FFME/Container/MediaBlock.cs#L78

and here: https://github.com/unosquare/ffmediaelement/blob/634ea3b2b0ebca65557b92d9f657cdabf7e94a38/Unosquare.FFME/Container/MediaBlock.cs#L83

They are just returning a simple value.

zgabi commented 5 years ago

And the RenderingVideo event is also called on the UI thread. I know that it would be a breaking change to move it to a background thread, but maybe you can add another event which is called on the background thread. So the Sample app (which is showing some information on the left top corner) and everybody who needs that can use it.

mariodivece commented 4 years ago

Ok, these are excellent proposed changes! And they are high priority for this project. Thanks so much for your insight and advice! Will schedule for the next release.