radek-k / FFMediaToolkit

FFMediaToolkit is a cross-platform video decoder/encoder library for .NET that uses FFmpeg native libraries. It supports video frames extraction, reading stream metadata and creating videos from bitmaps in any format supported by FFmpeg.
MIT License
362 stars 56 forks source link

Encoded video actual frame rate doesn't match the desired frame rate #139

Open tamirs9876 opened 2 months ago

tamirs9876 commented 2 months ago

Problem: Video file encoded using FFMediaToolkit is created with inaccurate frame rate. I'm setting VideoEncoderSettings.Framerate = 25 but the produced video frame rate is 25.1004016064257.

Repro sample: App: Console application, running .NET Framework 4.6.2 FFMediaToolkit package version: 4.5.1 FFmpeg binaries taken from here: https://github.com/BtbN/FFmpeg-Builds/releases/tag/autobuild-2024-09-28-13-00 I tried to create a 10 second video, with 25 FPS.

code:

using FFMediaToolkit;
using FFMediaToolkit.Decoding;
using FFMediaToolkit.Encoding;
using FFMediaToolkit.Graphics;
using System;
using System.Drawing;
using System.Drawing.Imaging;
using System.IO;
using Point = System.Drawing.Point;
using Size = System.Drawing.Size;

namespace TestVideoWriter
{
    internal class Program
    {
        public static void Main()
        {
            // lib downloaded from: https://github.com/BtbN/FFmpeg-Builds/releases/tag/autobuild-2024-09-28-13-00
            FFmpegLoader.FFmpegPath = @"D:\tools\ffmpeg_lib";

            // Video settings.
            const int Fps = 25;
            var videoLength = TimeSpan.FromSeconds(10);
            var frameSize = new Size(1280, 720);
            var videoWriterSettings = new VideoEncoderSettings(frameSize.Width, frameSize.Height, Fps, VideoCodec.H264);
            var totalFrames = Fps * videoLength.TotalSeconds;
            var videoPath = Path.GetFullPath("test.mp4");

            // Create a new video file.
            using (var outputFile = MediaBuilder.CreateContainer(videoPath).WithVideo(videoWriterSettings).Create())
            {
                var index = 0;
                while (index < totalFrames)
                {
                    var frame = CreateFrame(frameSize, $"Frame: {index++}");
                    WriteFrame(outputFile, frame);
                }
            }

            // After encoding completed, read the new video FPS value.
            using (var inputFile = MediaFile.Open(videoPath))
            {
                Console.WriteLine(inputFile.Video.Info.AvgFrameRate);
            }
        }

        private static void WriteFrame(MediaOutput outputFile, Bitmap frame)
        {
            var rect = new Rectangle(Point.Empty, frame.Size);
            var bitLock = frame.LockBits(rect, ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb);
            var bitmapData = ImageData.FromPointer(bitLock.Scan0, ImagePixelFormat.Bgr24, frame.Size);
            outputFile.Video.AddFrame(bitmapData);
            frame.UnlockBits(bitLock);
        }

        private static Bitmap CreateFrame(Size size, string text)
        {
            var bitmap = new Bitmap(size.Width, size.Height, PixelFormat.Format24bppRgb);
            using (var graphics = Graphics.FromImage(bitmap))
            {
                graphics.Clear(Color.White);
                var font = new Font("Arial", 20);
                var brush = new SolidBrush(Color.Black);
                graphics.DrawString(text, font, brush, new PointF(10, 40));
            }

            return bitmap;
        }
    }
}

running ffprobe shows the following: ffprobe -hide_banner test.mp4

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.16.100
  Duration: 00:00:09.96, start: 0.000000, bitrate: 285 kb/s
  Stream #0:0[0x1](und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(progressive), 1280x720, 283 kb/s, 25.10 fps, 25 tbr, 12800 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]

I ran this command: ffprobe -hide_banner -loglevel warning -select_streams v:0 -show_entries frame=pict_type,pkt_pos,pkt_size,pkt_dts,pkt_dts_time,pkt_duration,best_effort_timestamp,best_effort_timestamp_time -of compact test.mp4

I expected to receive 250 entries (10s*25fps), but I only got 249. The last entry time was 9,92 (instead of 9,96) so looks like it is missing, yet, dividing the expected number of frames by video duration returns the actual FPS (250/9.96=25.10040..) so it was written, partially.

Hope @radek-k will be able to take a quick look.

tamirs9876 commented 1 month ago

Hi @radek-k, I hope you’re doing well! I wanted to draw your attention to the issue I raised earlier, as I haven’t received a response yet. Could you please take a look when you have a moment? Thank you!

Maybe @redbaty will be able to review as well.