ImageSharp.AVCodecFormats

FFmpeg decoders for ImageSharp

Install

via NuGet:

PM> Install-Package ImageSharp.AVCodecFormats

Native libs for x64 Linux and Windows:

PM> Install-Package ImageSharp.AVCodecFormats.Native

also, we can install separate native packages:

PM> Install-Package ImageSharp.AVCodecFormats.Native.win-x64
PM> Install-Package ImageSharp.AVCodecFormats.Native.linux-x64

Without native packages you should provide your own shared FFmpeg build and set path:

FFmpegBinaries.Path = "/path/to/native/binaries"

On Linux you have another way to get native libs. Just install ffmpeg from your package manager, but I have no guarantees that it will work as expected.

Usage

using System.IO;

using SixLabors.ImageSharp;
using SixLabors.ImageSharp.Formats.Jpeg;
using SixLabors.ImageSharp.Processing;

using HeyRed.ImageSharp.AVCodecFormats;

// Create custom configuration with all available decoders
var configuration = new Configuration().WithAVDecoders();

// Or only required decoders
var configuration = new Configuration(
    new AviConfigurationModule(),
    new MkvConfigurationModule(),
    new MovConfigurationModule(),
    new Mp4ConfigurationModule(),
    new WebmConfigurationModule(),
    new WmvConfigurationModule(),
    new MpegTsConfigurationModule(),
    new Mp3ConfigurationModule());

// NOTE: Don't forget to set max frames.
// Without this limit you can run into huge memory usage.
var decoderOptions = new DecoderOptions
{
    MaxFrames = 50,
    Configuration = configuration,
    // The TargetSize option is also supported. 
    // This will reduce memory usage in result image.   
};

using var inputStream = File.OpenRead("/path/to/video.mp4");
using var image = Image.Load(decoderOptions, inputStream);

// Resize
image.Mutate(x => x.Resize(image.Width / 2, image.Height / 2)); 

// Save all frames using png encoder
for (int i = 0; i < image.Frames.Count; i++)
{
    image.Frames
        .CloneFrame(i)
        .SaveAsPng($"frame{i}.png");
}

More info https://docs.sixlabors.com/articles/imagesharp/configuration.html

Frame filter

Frame filter is a delegate that provides the way to skip frames based on their content.

It can be useful for various scenarios like black/white frames filter, skip every n frame or something else.

var decoderOptions = new AVDecoderOptions
{
    GeneralOptions = new DecoderOptions
    {
        MaxFrames = 100,
    },
    FrameFilter = (imageFrame, frameNum) =>
    {
        var frame = (ImageFrame<Rgba32>)frame; // Pix format should match with Load/Decode methods

        // Do something with frame

        // Return true when frame should be not present in result image
        return false;
    },
};

using var inputStream = File.OpenRead(/path/to/video.mp4);
using var image = Mp4Decoder.Instance.Decode(decoderOptions, inputStream);

// do something

See tests for basic implementation of black frames filter.

Additional tips

If you want preserve aspect ratio when TargetSize is set:

var decoderOptions = new AVDecoderOptions
{
    PreserveAspectRatio = true,
    GeneralOptions = new DecoderOptions
    {
        TargetSize = new Size(500),
    },
};

An additional format specific information about the file:

// Get infogmation about webm file
AVMetadata metadata = Image.Identify(decoderOptions, inputStream).Metadata.GetWebmMetadata();

More extensions methods

Supported formats

mp4, webm, avi, mkv, mov, ts, wmv, mp3(cover image).

Supported codecs

Native package provides codecs listed below:

H263, H264, H.265(HEVC), VP8, VP9, AV1, MPEG-4, MJPEG, PNG, MS MPEG4(v1,v2,v3), WMV(v1,v2,v3), VC-1, MPEG-1 Audio Layer 3.

License

MIT