search

ffmpeginteropx / FFmpegInteropX

FFmpeg decoding library for Windows 10 UWP and WinUI 3 Apps
Apache License 2.0
205 stars 52 forks source link

Dynamic format changes #407

Closed brabebhin closed 6 months ago

brabebhin commented 7 months ago

This applies the "Microsoft" approach to dynamic format changes. We did have some degree of format change detection, but there's some differences

We were setting the properties directly onto the sample provider media stream descriptor. Microsoft was setting it on the media stream descriptor that came from the sample request event arguments, and uses MF GUIDs to identify the properties. Technically these should be the same, but who knows?

For video, changing resolution using a filter is an easy way to see the differences -> it will lower the picture quality, but overall, the MediaPlayer will still render the video in the full size of the MPE, and the aspect ratio button doesn't seem to have any effect. The MediaPlayer does rise a NaturalVideoSizeChanged event, so I guess this is ok?

For audio, this PR will allow us to skip resampling when the audio format changes in the stream during playback. We will leave that to MF. This seems to work fairly well - I resampled audio using a filter from 44100 to 88200 and it seems to work as intended. Audio still has the expected speed and will not play "slower", so this means the stream descriptor and MF correctly respond to the change in sampling rate. So essentially, the only time we will be using a resampler is when the sample format is not one of those that we support.

brabebhin commented 7 months ago

It would appear setting the properties on the MediaEncodingProperties also works. So I am not sure why Microsoft uses the GUID approach. I also slightly improved the logic for audio, by following the full procedure we do when opening a file for the first time.

brabebhin commented 6 months ago

Well, the funny part is we don't actually need to pass the sample provider at all, we can do the changes on our own pointers, but they only register during the sample event.

So I think this latest version should also cover seeking.

lukasf commented 6 months ago

Well, the funny part is we don't actually need to pass the sample provider at all, we can do the changes on our own pointers, but they only register during the sample event.

Good find!

I added some improvements to the format switching. We do not support all formats natively, so even after a dynamic format change, we must check if resampler needs update.

brabebhin commented 6 months ago

Nice find with the unsupported formats too!

brabebhin commented 6 months ago

@lukasf The build seems to fail due to winUI project, should we merge it anyway?

lukasf commented 6 months ago

AppVeyor is red since quite a while. I need to fix it up but I am too lazy ^^ Just merge it.

brabebhin commented 6 months ago

I will take a look at the build.

brabebhin commented 6 months ago

Looks like the build is broken because the "debug" configuration tries to build the winui package without also building the dotnet package.

I don't think it is even worth fixing right now on master, seeing how the winui_build is going to change these configs anyway.