FFmpegInteropX is an open-source project that aims to provide an easy way to use FFmpeg as a decoder library in Windows 10/11 UWP Apps. This allows you to decode a lot of formats that are not natively supported on Windows 10/11. Please note that only decoding is supported currently, we provide no encoding or transcoding support.
FFmpegInteropX is a much improved fork of the original Microsoft project.
We have switched from manual builds to providing NuGet packages. There are two packages:
The easiest way to work with FFmpegInteropX is to add both NuGet packages to your app. This allows full usage of all features, without checking out the repo or installing build tools.
Advanced users and library developers: If you want to be able to debug into FFmpegInteropX right from your app, or to work on the library, you need to clone this repository. Instead of adding the FFmpegInteropX NuGet package to your app, you can directly add the Source\FFmpegInteropX.vcxproj project file to your app solution (it does not matter where the FFmpegInteropX folder is located). Then in your main app project, add a reference to the FFmpegInteropX project. Now you have all the sources directly in your app solution and can debug and enhance the lib.
Full blown: If needed, you can even supply your own custom FFmpeg build to replace our FFmpegInteropX.FFmpegUWP NuGet package.
Check out the build instructions if you want to manually build FFmpgeInteropX or FFmpeg itself.
Using the FFmpegMediaSource object is fairly straightforward and can be observed from the sample applications provided.
IRandomAccessStream for the media you want to playback.FFmpegMediaSource object using FFmpegMediaSource.CreateFromStreamAsync() passing it the stream and optionally a config class instance.CreateMediaPlaybackItem()Important: Store the FFmpegMediaSource instance e.g. in a local field, as long as playback is running. If the object is collected by the GC during playback, playback will stop with an error.
You can use FFmpegMediaSource.GetMediaStreamSource() to get the MediaStreamSource like in the original version of the library. But when using MediaStreamSource, you won't get subtitles. Subtitle support requires using the MediaPlaybackItem!
Use FFmpegMediaSource.CreateFromUriAsync() to create a MediaStreamSource on a streaming source (video stream, live stream, shoutcast audio stream, ...).
There are a lot of FFmpeg parameters that can be set, especially for streaming scenarios. You most definitely want to enable automatic reconnection on errors:
config.FFmpegOptions = new Windows.Foundation.Collections.PropertySet {
{ "reconnect", 1 },
{ "reconnect_streamed", 1 },
{ "reconnect_on_network_error", 1 },
}You should also consider setting timeouts, but be careful since each protocol might have different timeout parameters and meanings.
Check FFmpeg documentation on Protocols and Formats for more information on the available parameters.
If you do web streaming, you should try the new read-ahead buffer feature of FFmpegInteropX, available in the latest prerelease packages. It will read ahead and buffer a configurable amount of stream data to allow uninterrupted playback. This even allows fast forward seeking (stepping) without stream reconnection, if the target position is buffered.
config.ReadAheadBufferEnabled = true;
// Optionally, configure buffer size (max duration and byte size)
config.ReadAheadBufferDuration = TimeSpan.FromSeconds(30);
config.ReadAheadBufferSize = 50*1024*1024;You can use the FrameGrabber class to extract video frames at specific positions of a video file or video stream, optionally using fast seeking to keyframes and optionally downscaling the video resolution. Extracted frames contain raw image data and can be stored to different formats (JPEG, BMP, PNG).
Simplified example calls:
var fileStream = await file.OpenRead();
var frameGrabber = await FrameGrabber.CreateFromStreamAsync(fileStream);
var frame = await frameGrabber.ExtractVideoFrameAsync(TimeSpan.FromSeconds(10));
await frame.EncodeAsJpegAsync(outputStream);FFmpegInterop will automatically load and use all embedded subtitles, supporting all formats through ffmpeg. You have to use the MediaPlaybackItem returned from the FFmpegMediaSource object. Then subtitles can be selected from MediaElement's transport controls.
You can also add external subtitle files by using FFmpegMediaSource.AddExternalSubtitleAsync(), even during playback. See the sample apps for reference. All ffmpeg subtitle formats are supported as external files, except for the two-file "sub/idx" (DVD) format.
Some external text subtitle files are stored with ANSI encoding instead of UTF8 (which is required by ffmpeg). FFmpegInterop can do an automatic conversion to UTF8. This is enabled by default in the config class and will use the system's active codepage by default. You can change the behavior by changing AutoCorrectAnsiSubtitles and AnsiSubtitleEncoding parameters in the config class. Codepage 0 is the system's active codepage.
Note: If your app uses multiple windows using CoreApplication.CreateNewView(), then you must create the FFmpegMediaSource object on the thread of the window where the video is to be shown. Otherwise, subtitles will flicker.
Many more helped with development, bug reports and suggestions. Thank you all!
Thank you also to the Microsoft team who developed the original library!
Feel free to open issues, pull requests, or join discussions.