Using framegrabber to record a live stream

[don-pironet-hatch]

Hi,

I'm currently using the library to play a live RTSP stream. I see there's a frame grabber, but is there also a way to write an RTSP stream to a file?

What I tried now is the following:

1. Create a new file
2. Use Framegrabber to get the current frame (await frameGrabber.ExtractVideoFrameAsync(mediaPlayer.PlaybackSession.Position);) // This doesn't work well because it gives sometimes an older frame
3. Use the grabbed file and encode as jpeg
4. Repeat this loop

Once the user decide to stop recording

5. Loop through all the saved frames
6. Create a mediaclip for every frame
7. Save the MediaComposition to disk

The problem is on point 2 and also on some devices with less good hardware because of all the file access it doesn't work.

Is there another way?

[brabebhin]

Hi. This seems to be the average MediaTransoder API usage for our library. We have some limited support for that, basically you can use this method

https://learn.microsoft.com/en-us/uwp/api/windows.media.transcoding.mediatranscoder.preparemediastreamsourcetranscodeasync?view=winrt-22621#windows-media-transcoding-mediatranscoder-preparemediastreamsourcetranscodeasync(windows-media-core-imediasource-windows-storage-streams-irandomaccessstream-windows-media-mediaproperties-mediaencodingprofile)

Together with the Mediastreamsource object we expose from the FfmpegMediaSource. This limits you to the WinRT output formats and not the full range of formats supported by FFmpeg. I think this will also skip subtitles if there's any.

Unfortunately we currently don't have a dedicated transcoder API and I have no ETA when that would be available. There are some technical and legal issues that stand out. We are mainly a decoder library.

[don-pironet-hatch]

Thank you for your help. The problem with that approach is that I can't use that exposed MediaStreamSource. The TranscodeAsync throws an exception because I can't have more than one listener on that generator.

We are also displaying the video via an Mediaplayer.

_decoder = await FFmpegMediaSource.CreateFromUriAsync(RtspUrl, configuration);
_mediaPlaybackItem = _decoder.CreateMediaPlaybackItem();

_mediaPlayer = new MediaPlayer();
_mediaPlayer.Source = _mediaPlaybackItem;

var source = _decoder.GetMediaStreamSource();
var filename = GetVideoFileName();
var storageFolder = await StorageFolder.GetFolderFromPathAsync(path);
var destinationFile =
    await storageFolder.CreateFileAsync(filename, CreationCollisionOption.ReplaceExisting);
var destination = await destinationFile.OpenAsync(FileAccessMode.ReadWrite);

try
{
    var transcode = await _mediaTranscoder.PrepareMediaStreamSourceTranscodeAsync(source, destination, MediaEncodingProfile.CreateMp4(VideoEncodingQuality.Auto));
    await transcode.TranscodeAsync();
}
catch (Exception exception)
{
// 
}

[brabebhin]

I'd have 2 suggestions here:

1. You create 2 different FFmpegMediaSource, one for transcoding, one for rendering. (this would be the best approach)
2. You use the MediaPlayer frame server mode to get every raw frame and feed it into a transcoder (never tried this with the MediaTranscoder)

The frame grabber is not designed to provide such high performance to be able to accurately reconstruct a stream. That's not what its purpose is.

[lukasf]

Our lib does not support simultaneous playback and recording/transcoding. I also don't think that the frame server mode will help you, since it only gives you video frames, but no audio, and there is no sync mechanism as well.

So your only option would be to create two connections, one for playback and one for recoding. That's not optimal and it could cause bandwidth problems on slower connections. But I don't know of any other way.

[lukasf]

Closing this, since the scenario is just not supported by our lib.