System.IO.EndOfStreamException while decoding mp4 file into frames

[MastalerzKamil]

I'm using .NET Core 3.1. What I would like to do is decode whole mp4 file into frames. I have uploaded .mp4 file in uploads directory. I want to decode it and save all frames into pictures directory. My code has been inspired by README and issue #20

namespace MyNamespace
{
    public class VideoProcessing
    {
        private readonly IWebHostEnvironment _webHostEnvironment;
        public VideoProcessing(IWebHostEnvironment env)
        {
            _webHostEnvironment = env;
        }

        public void SplitVideoIntoFrames(string imageId)
        {
            string webRootPath = _webHostEnvironment.WebRootPath;
            var videoRoute = Path.Combine(webRootPath, $"uploads/{imageId}");

            var destinationDirectoryRoute = Path.Combine(webRootPath, $"pictures/{Path.GetFileNameWithoutExtension(imageId)}");
            Directory.CreateDirectory(destinationDirectoryRoute);

            var file = MediaFile.Open(videoRoute);
            for (int i = 0; i < file.Video.Info.FrameCount; i++)
            {
                var frameFileRoute = Path.Combine(destinationDirectoryRoute, $"frame_{i}.png");
                file.Video.ReadFrame(i).ToBitmap().Save(frameFileRoute);
            }
        }
    }

    public static class Extensions
    {
        public static Image<Bgr24> ToBitmap(this ImageData imageData)
        {
            return Image.LoadPixelData<Bgr24>(imageData.Data, imageData.ImageSize.Width, imageData.ImageSize.Height);
        }
    }

}

The exception which I get is following

System.IO.EndOfStreamException: End of the file.
   at FFMediaToolkit.Decoding.Internal.InputContainer.ReadPacket()
   at FFMediaToolkit.Decoding.Internal.InputContainer.GetPacketFromStream(Int32 streamIndex)
   at FFMediaToolkit.Decoding.Internal.InputContainer.ReadNextPacket(Int32 streamIndex)
   at FFMediaToolkit.Decoding.Internal.Decoder`1.DecodePacket()
   at FFMediaToolkit.Decoding.Internal.Decoder`1.ReadNextFrame()
   at FFMediaToolkit.Decoding.Internal.Decoder`1.GetNextFrame()
   at FFMediaToolkit.Decoding.VideoStream.GetNextFrameAsBitmap()
   at FFMediaToolkit.Decoding.VideoStream.ReadFrame(Int32 frameNumber)

[jrz371]

The StreamInfo.FrameCount can't be trusted to be accurate. There are two ways the value is filled.

1. The frame count is provided by the container
2. The frame count is calculated from the video duration and the average frame rate

There is a StreamInfo.IsFrameCountProvidedByContainer you can check to see if it was filled the first way. Otherwise it's the second. Here's the corresponding code. You shouldn't trust the frame count if it's provided by the container either. That isn't always accurate. This library really needs a better method for decoding.

[MastalerzKamil]

What I did to dimiss that bug is using

try {
for() {
....
}
} catch(EndOfStreamException){}

It helped I don't know why

[radek-k]

I recommend using ReadFrame(TimeSpan) instead of ReadFrame(int). It is accurate both in constant and variable frame rate videos. I've also added a bool TryReadNextFrame method that returns false instead of throwing an EndOfStreamException. See example code.