Internal CLR error with square video with 600x600 resolution

[ggolda]

Hello! I have an incredibly weird issue to report. My code snippet below fails only on videos that have 600x600 resolution and works absolutely flawlessly on videos with other resolutions.

Here is an example that can help to reproduce such weird behavior:

MediaOptions options = new MediaOptions()
{
    StreamsToLoad = MediaMode.Video,
    VideoPixelFormat = ImagePixelFormat.Rgba32,
};

var _file = MediaFile.Open(filePath, options);
if (!_file.HasVideo)
{
    throw new Exception($"Can't create video resource. Source file of resource {id} doesn't have video stream.");
}

var _videoStream = _file.Video;

ImageData imageData;
bool success = true;

while (success) {
    success = _videoStream.TryGetNextFrame(out imageData);
}

I tried several different combinations of resolutions, so far I can reproduce it only with 600x600 and 602x602 resolutions. I tested with h264 codec, yuv420p pixel format and mp4 container videos. Below is stream info from ffprobe for a video that I tested:

Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 600x600 [SAR 1:1 DAR 1:1], 429 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)

I tried to transcode it to different resolutions using -vf 'scale=800:800' filter for example using ffmpeg, and this strange issue was gone. I also tried to provide a TargetVideoSize MediaOptions to resize frames to a different resolution - no CLR internal issue in this case.

Can provide video sample that breaks this library if it helps (tried with different source samples with same resolution).

TLDR; Internal CLR error while fetching frames from videos with 600x600 resolution. With every other resolution combination I can't reproduce such issue.

[hey-red]

Can you provide sample video? I'm trying to reproduce with: Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 600x600 [SAR 1:1 DAR 1:1], 505 kb/s, 30 fps, 30 tbr, 15360 tbn (default)

and extracting works without any issue. Also, what ffmpeg build you are using?

[ggolda]

@hey-red here is a WeTransfer with a sample video that crashes for me: https://we.tl/t-oyirFjcUju

I'm using a windows build ffmpeg-n4.3.1-19-gaf2a430bb1-win64-gpl-shared-4.3, but I tried with other ffmpeg builds (including ffmpeg 5 and ffmediatoolkit 4.2.0) and had the same issue.

I can also reproduce it in a linux environment inside docker with ffmpeg ffmpeg version 4.3.2-0york0~18.04 Copyright (c) 2000-2021 the FFmpeg developers

[ggolda]

This is a full example that crashes with video provided above:

using System;
using FFMediaToolkit;
using FFMediaToolkit.Decoding;
using FFMediaToolkit.Graphics;

namespace FfmpegCrashTest
{
    class Program
    {
        static void Main(string[] args)
        {
            FFmpegLoader.FFmpegPath = "C:\\Users\\Salmondx\\Downloads\\ffmpeg-n4.3.1-19-gaf2a430bb1-win64-gpl-shared-4.3\\ffmpeg-n4.3.1-19-gaf2a430bb1-win64-gpl-shared-4.3\\bin";
            MediaOptions options = new MediaOptions()
            {
                StreamsToLoad = MediaMode.Video,
                VideoPixelFormat = ImagePixelFormat.Rgba32,
            };

            var _file = MediaFile.Open("C:\\Users\\Salmondx\\Documents\\job\\videos\\sample600x600-test.mp4", options);
            if (!_file.HasVideo)
            {
                throw new Exception($"Can't create video resource. Source file of resource doesn't have video stream.");
            }

            var _videoStream = _file.Video;

            ImageData imageData;
            bool success = true;

            while (success)
            {
                success = _videoStream.TryGetNextFrame(out imageData);
            }
        }
    }
}

And csproj file:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net5.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="FFMediaToolkit" Version="4.1.1" />
  </ItemGroup>

</Project>

As you can see, I'm using a slightly outdated version of a library, but same error is reproduceable for me even with the latest version and ffmpeg 5.

Fatal error. Internal CLR error. (0x80131506)
   at System.GC.AllocateNewArray(IntPtr, Int32, GC_ALLOC_FLAGS)
   at System.Buffers.TlsOverPerCoreLockedStacksArrayPool`1[[System.Byte, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Rent(Int32)
   at System.Buffers.ArrayMemoryPool`1+ArrayMemoryPoolBuffer[[System.Byte, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]..ctor(Int32)
   at System.Buffers.ArrayMemoryPool`1[[System.Byte, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Rent(Int32)
   at FFMediaToolkit.Graphics.ImageData.CreatePooled(System.Drawing.Size, FFMediaToolkit.Graphics.ImagePixelFormat)
   at FFMediaToolkit.Common.Internal.VideoFrame.ToBitmap(FFMediaToolkit.Common.Internal.ImageConverter, FFMediaToolkit.Graphics.ImagePixelFormat, System.Drawing.Size)
   at FFMediaToolkit.Decoding.VideoStream.GetNextFrame()
   at FFMediaToolkit.Decoding.VideoStream.TryGetNextFrame(FFMediaToolkit.Graphics.ImageData ByRef)
   at FfmpegCrashTest.Program.Main(System.String[])

Dotnet version: 6.0.100

[hey-red]

@ggolda I can reproduce it only for .NET 5, with NET 6 it's works. That's strange..

Also, your csproj contains <TargetFramework>net5.0</TargetFramework> not 6.0

[ggolda]

Indeed, you are right. Tried with net6.0 and no issue. Any ideas what is wrong with net5.0 and why it reproduces only for this specific resolution?

[radek-k]

Try using VideoStream.TryGetNextFrame(IntPtr buffer, int bufferStride) or TryGetNextFrame(Span<byte> buffer) overloads to get the pixel data directly (bypass ImageData). The ImageData allocation method is poorly implemented and the entire class needs a complete redesign.