Issue with FFmpegFileSource OnVideoSourceRawSampleFaster

[rob-baily]

First thank you for these great set of libraries! I have been trying to run the WebRTC sample as shown at https://github.com/sipsorcery-org/sipsorcery and have had limited success with the library as is.

First off, I could not find good FFMPEG libraries for version 4.4 so I have tried it out using version 6.0 and that seems to be mostly OK. My real problem has been that no matter with what version I use when I get the video going from the web page connecting to the socket the server side keeps crashing. By doing some tweaks to this library I found that by commenting out line 60 in FFmpegFileSouce.cs (see code below) that the crash would not happen and the MP4 file would always be played. Oddly, very infrequently with the existing library as is it would work. By by very rarely it was probably less than 1 in 10 times.

I am going to submit a PR for the upgrade for FFMPEG libraries but don't really understand what is happening with this not working. It could be related to the BGR24 conversion that is happening in FFmpegVideoSource when the OnVideoSourceRawSampleFaster handler is not null but again it is hard to tell and I don't know what the purpose of the "raw sample faster" is.

Please let me know if you have any thoughts and I happy to provide more information or answer questions.

[rob-baily]

Hmm, guess I did not include the code as I said above. Here it is.

if (useVideo)
{
    _FFmpegVideoSource = new FFmpegVideoSource();
    _FFmpegVideoSource.CreateVideoDecoder(path, null, repeat, false);
    _FFmpegVideoSource.OnVideoSourceEncodedSample += _FFmpegVideoSource_OnVideoSourceEncodedSample;
    //_FFmpegVideoSource.OnVideoSourceRawSampleFaster += _FFmpegVideoSource_OnVideoSourceRawSampleFaster;
    _FFmpegVideoSource.OnVideoSourceError += _FFmpegVideoSource_OnVideoSourceError;
}

[rob-baily]

@sipsorcery Would it be possible to explain what the intent of the OnVideoSourceRawSampleFaster callback is? I can see in FFmpegVideoSource that it attempts to invoke it after doing a conversion to BGR24. It seems likely that the actual conversion is the problem but I don't know what purpose it is supposed to serve as it seems to work OK without it.

[ChristopheI]

The goal was to avoid to byte[] in parameters See comments in MediaEndPoints.cs

    public delegate void RawVideoSampleFasterDelegate(uint durationMilliseconds, RawImage rawImage); // Avoid to use byte[] to improve performance
    public delegate void VideoSinkSampleDecodedFasterDelegate(RawImage rawImage); // Avoid to use byte[] to improve performance

About new version of ffmepg, new pull requests have been made about it