Current deinterlacer method is not smooth (Yadif Mode 1 does not work properly)

[chillout23]

Current deinterlacing option in KSLib is not producing smooth TV-like motion. Please try to implement something like VLC's Yadif x2 or MPV's deinterlacer. They are very smooth and looks very good. Can you take this challenge @kingslay ?

[kingslay]

Yadif x2 对应的ffmpeg的filter是什么呢？我这个项目是基于ffmpeg的。如果ffmpeg支持的话，那我就可以实现。

[nakiostudio]

I believe this is how you would setup Yadif x2 in ffmpeg ffmpeg -i input.mp4 -vf "yadif=mode=1" output.mp4

[cdguy]

@nakiostudio @chillout23 In earlier versions of the demo app, when I used the function “yadif=mode=1” the live streams played in slow motion.

1- normally the yadif documentation says "yadif=1 takes true interlaced 50i footage and turns it into astonishingly high-quality 50p output"

however, the live streams are played in slow motion.

@kingslay when I check the bitratea of interlaced streams with a player like Qmplay2 I see that interlaced streams never output 50i consistently.

The bitrate varies between 45i-50i. Besides, when I open an interlaced stream with the demo app, the demo app says that the FPS value is 25p (nothing wrong here, 50i=25p)

As you can imagine if you divide 46i to 2, it is equal to 23p

perhaps if you can find a way to double the frames per second before the stream starts (meaning that the demo app displays the FPS value as 50p) this “might” solve the issue

[nakiostudio]

Maybe Yadif mode=0 is good enough and it doesn't slow the stream down as it doesn't double the frame rate.

[cdguy]

https://stackoverflow.com/questions/45462731/using-ffmpeg-to-change-framerate?ssp=1&darkschemeovr=1&setlang=tr-TR&safesearch=moderate

@kingslay maybe this might help

[chillout23]

Do you guys know what kind of deinterlacer MPVLib uses? The motion is very smooth and performance is good on Apple TV. With Yadif x2 on VLC the performance is not very good on Apple TV. It is not optimized for ARM CPUs. So stuttering happens after 5 minutes as Apple TV cannot keep up with high load. However, MPV provides some similar deinterlacer which works well, but I do not know what kind of method is being used.

So if possible the logic from MPV (which is also based on ffmpeg) is what is needed to solve this - as it works well on ATV.

[cdguy]

Do you guys know what kind of deinterlacer MPVLib uses? The motion is very smooth and performance is good on Apple TV. With Yadif x2 on VLC the performance is not very good on Apple TV. It is not optimized for ARM CPUs. So stuttering happens after 5 minutes as Apple TV cannot keep up with high load. However, MPV provides some similar deinterlacer which works well, but I do not know what kind of method is being used.

So if possible the logic from MPV (which is also based on ffmpeg) is what is needed to solve this - as it works well on ATV.

@chillout23 mpv uses yadif x1

In my opinion, doubling the frames (converting the 50i stream to 50p) “might” solve the issue. Let’s wait and see

[chillout23]

I see. Technically we are talking about 25i don't we? I see that 25i is also known as 50i.

Anyways, sounds like a good suggestion :)

[cdguy]

I see. Technically we are talking about 25i don't we? I see that 25i is also known as 50i.

Anyways, sounds like a good suggestion :)

No, interlaced streams are only encoded in 50i/60i which is also equal to 25p/30p. Let’s wait for @kingslay what he has to say, no need to speculate :)

[kingslay]

我没有deinterlacing的本地视频。你们有人可以提供下对应的视频让我进行测试吗？

[kingslay]

https://stackoverflow.com/questions/45462731/using-ffmpeg-to-change-framerate?ssp=1&darkschemeovr=1&setlang=tr-TR&safesearch=moderate

@kingslay maybe this might help

@cdguy 目前ksplayer是支持fps 这个filter。你可以加一下fps，看下视频播放会不会更流畅点。

[kingslay]

Do you guys know what kind of deinterlacer MPVLib uses? The motion is very smooth and performance is good on Apple TV. With Yadif x2 on VLC the performance is not very good on Apple TV. It is not optimized for ARM CPUs. So stuttering happens after 5 minutes as Apple TV cannot keep up with high load. However, MPV provides some similar deinterlacer which works well, but I do not know what kind of method is being used. So if possible the logic from MPV (which is also based on ffmpeg) is what is needed to solve this - as it works well on ATV.

@chillout23 mpv uses yadif x1

In my opinion, doubling the frames (converting the 50i stream to 50p) “might” solve the issue. Let’s wait and see

@cdguy 我看yadif的mode 参数值如下

mode = 0 : temporal and spatial interlacing check (default).

= 1 : double framerate (bob), temporal and spatial interlacing check.

= 2 : skips spatial interlacing check.

= 3 : double framerate (bob), skips spatial interlacing check.

你的意思是mpv用的是 mode=1 这个参数值吗？

[cdguy]

https://stackoverflow.com/questions/45462731/using-ffmpeg-to-change-framerate?ssp=1&darkschemeovr=1&setlang=tr-TR&safesearch=moderate @kingslay maybe this might help

@cdguy 目前ksplayer是支持fps 这个filter。你可以加一下fps，看下视频播放会不会更流畅点。

@kingslay can you please write the code regarding how to double the fps value for interlaced streams? I mean if the stream is 50i, to make it 50p; and if the stream is 60i, how to make it 60p

[kingslay]

你可以先试下用这个。 class MEOptions: KSOptions { override func process(assetTrack: some MediaPlayerTrack) { if assetTrack.mediaType == .video { if [FFmpegFieldOrder.bb, .bt, .tt, .tb].contains(assetTrack.fieldOrder) { videoFilters.append("yadif=mode=1:parity=-1:deint=1") hardwareDecode = false } } } }

如果不够顺畅的话，那可以试下 class MEOptions: KSOptions { override func process(assetTrack: some MediaPlayerTrack) { if assetTrack.mediaType == .video { if [FFmpegFieldOrder.bb, .bt, .tt, .tb].contains(assetTrack.fieldOrder) { videoFilters.append("yadif=mode=1:parity=-1:deint=1") videoFilters.append("fps=fps=(Int(assetTrack.nominalFrameRate*2))") hardwareDecode = false } } }

[cdguy]

Do you guys know what kind of deinterlacer MPVLib uses? The motion is very smooth and performance is good on Apple TV. With Yadif x2 on VLC the performance is not very good on Apple TV. It is not optimized for ARM CPUs. So stuttering happens after 5 minutes as Apple TV cannot keep up with high load. However, MPV provides some similar deinterlacer which works well, but I do not know what kind of method is being used. So if possible the logic from MPV (which is also based on ffmpeg) is what is needed to solve this - as it works well on ATV.

@chillout23 mpv uses yadif x1 In my opinion, doubling the frames (converting the 50i stream to 50p) “might” solve the issue. Let’s wait and see

@cdguy 我看yadif的mode 参数值如下

mode = 0 : temporal and spatial interlacing check (default).

= 1 : double framerate (bob), temporal and spatial interlacing check.

= 2 : skips spatial interlacing check.

= 3 : double framerate (bob), skips spatial interlacing check.

你的意思是mpv用的是 mode=1 这个参数值吗？

@kingslay I mean, mpv uses the default yadif parameter (=0)

I just installed the desktop version of MPV. the stream I opened is an interlaced stream (50i), the media info says that the stream is 25p. My computer refresh rate is at 59.9 hz; because of that, you can see that the "displayed FPS" value is 59.977 FPS

[kingslay]

@cdguy 那你换下用videoFilters.append("yadif=mode=0:parity=-1:deint=1")，看下效果怎么样。用mode=1的话，会导致cpu占用比较高。不知道是不是因为这个原因所以视频不够流畅。

[cdguy]

@kingslay

Let me express my findings:

1- Unfortunately, the last commit did not solve the "non-smooth" issue 2- if the audio codec of the interlaced stream is MP2, the sound sounds very strange 3a- when you open the same interlaced stream with MPV and KSPLAYER, without the YADIF filter, the smoothness is the same 3b- I suspect the "non smooth" issue happens when you apply the YADIF filter to KSplayer because with MPV it is smooth 4- when I used this code, the FPS value was still 25p, and the YADIF filter did not work:

class MEOptions: KSOptions {
override func process(assetTrack: some MediaPlayerTrack) {
if assetTrack.mediaType == .video {
if [FFmpegFieldOrder.bb, .bt, .tt, .tb].contains(assetTrack.fieldOrder) {
videoFilters.append("yadif=mode=0:parity=-1:deint=1")
videoFilters.append("fps=fps=(Int(assetTrack.nominalFrameRate*2))")
hardwareDecode = false

I will send you in a few minutes some links

[cdguy]

@kingslay the audio tracks encoded in MP2 still have the issue I mentioned. You can try with the stream link I sent to you via e-mail (the one that ends with 32487)

[chillout23]

Is this fixed now? @cdguy

[kingslay]

@cdguy 可以不用加videoFilters.append("fps=fps=(Int(assetTrack.nominalFrameRate*2))")。 只要videoFilters.append("yadif=mode=0:parity=-1:deint=1") 。

[cdguy]

Is this fixed now? @cdguy

@chillout23 Yes it is (almost) fixed. It works perfectly on tvos, and on ios it needs a fine tuning.

I sent an e-mail to Kingslay about it

[kingslay]

Is this fixed now? @cdguy

@chillout23 Yes it is (almost) fixed. It works perfectly on tvos, and on ios it needs a fine tuning.

I sent an e-mail to Kingslay about it

@cdguy 我看了下是因为在iOS preferredFramesPerSecond 不起作用，在tvOS是起作用的。还是会以60hz来刷新，但是视频是25hz。这就会导致不流畅。mpv在iOS也是流畅的吗？

[cdguy]

@kingslay I suspect this “might be” because of the refresh rate of ios devices; as some models have a refresh rate up to 120hz, whereas other ios devices have a “static” screen refresh rate of 60/90 hz

[cdguy]

我看了下是因为在iOS preferredFramesPerSecond 不起作用，在tvOS是起作用的。还是会以60hz来刷新，但是视频是25hz。这就会导致不流畅。mpv在iOS也是流畅的吗？

Yes, just check the link I sent you yesterday (that ends with 6cc3a) you will see that the video with MPV is smooth @kingslay

[kingslay]

是的，我在最新的代码增加了高刷的支持，有设备的人可以尝试下。但是我看了apple的文档 https://developer.apple.com/documentation/quartzcore/optimizing_promotion_refresh_rates_for_iphone_13_pro_and_ipad_pro 。里面没有支持25hz或50hz。所以我猜测还是会有问题。但是如果刷新率高一点的话，不流畅就不会那么明显了。

[cdguy]

Is this fixed now? @cdguy

@chillout23 Yes it is (almost) fixed. It works perfectly on tvos, and on ios it needs a fine tuning. I sent an e-mail to Kingslay about it

@cdguy 我看了下是因为在iOS preferredFramesPerSecond 不起作用，在tvOS是起作用的。还是会以60hz来刷新，但是视频是25hz。这就会导致不流畅。mpv在iOS也是流畅的吗？

@kingslay I know it is easy to say, but if you could log and monitor the behaviour of the refresh rate in MPV lib when using an iphone (X-11-12) that has a refresh rate of 60hz; you might find something

[kingslay]

我对mpv的了解很少，你知道mpv是怎么来实现视频的渲染的吗？我搜索下了下mpv的代码。里面没有用NSTimer和CADisplayLink。 我很好奇他它怎么样来实现定时渲染视频的。

[cdguy]

I have sent you via e-mail a few screenshots that might help you. @kingslay

[chillout23]

@cdguy is the motion smooth as MPV now in this KSPlayer?

[chillout23]

Tested in latest version of Tracy Player, the motion is better than before but still not perfect like MPV or VLCs Linear deinterlacer. @kingslay please reopen this issue

[cdguy]

是的，我在最新的代码增加了高刷的支持，有设备的人可以尝试下。但是我看了apple的文档 https://developer.apple.com/documentation/quartzcore/optimizing_promotion_refresh_rates_for_iphone_13_pro_and_ipad_pro 。里面没有支持25hz或50hz。所以我猜测还是会有问题。但是如果刷新率高一点的话，不流畅就不会那么明显了。

There is nothing to reopen @chillout23; on tvos the image is very smooth, @kingslay will definitely publish a “fix” for ios/ipados devices that does not have ProMotion screens.

[chillout23]

There is. Have you seen MPVs deinterlacer? Or VLCs Linear deinterlacer? The motion is silky smooth. I am not talking about iOS/iPadOS. I am testing on TV and comparing different player engines. Download iPlayTV on tvOS for example and compare VLC/MPV engines with KSPlayer with an interlaced stream.

https://testflight.apple.com/join/irMsXGg0

VLCs deinterlacer flickers, but the motion is silky smooth. MPVs deinterlacer does not flicker and is also perfect. This is what KSPlayer should aim for. If using MPV on iPlay TV you have to enable deinterlacing manually. Auto does not work.

Try it and see for yourself :)

[cdguy]

@kingslay the “fix” you can apply might be the following:

1- identify the ios/ipadOS devices that does have promotion display:

• iphone: 13pro-14pro-15pro (and pro max models)
• Ipad: all pro models from 2021 model to the most recent

2- if the device used is not the device mentioned above do the following:

a- for 25 fps streams (I mean 25p streams); multiply the number of frames by 2,4 so the stream would be “upgraded” to 60p and match the refresh rate of the device’s screen which is 60hz b- for a stream that is 50fps (both 50i or 50p) multiply the number of frames by 1,2 so the stream would be “upgraded” to 60p and match the refresh rate of the device’s screen which is 60hz.

[chillout23]

@cdguy, let me know when you have tested. you will see what I mean.

[cdguy]

There is. Have you seen MPVs deinterlacer? Or VLCs Linear deinterlacer? The motion is silky smooth. I am not talking about iOS/iPadOS. I am testing on TV and comparing different player engines. Download iPlayTV on tvOS for example and compare VLC/MPV engines with KSPlayer with an interlaced stream.

https://testflight.apple.com/join/irMsXGg0

VLCs deinterlacer flickers, but the motion is silky smooth. MPVs deinterlacer does not flicker and is also perfect. This is what KSPlayer should aim for. If using MPV on iPlay TV you have to enable deinterlacing manually. Auto does not work.

Try it and see for yourself :)

The iplayTV beta app does not use the latest ksplayer code, and the stream is still jerky in IplayTV. You should ask to Andre to update first the ksplayer library and let him know that the player is called Ksplayer and not “ffmpeg” as he names it. A bit of respect to @kingslay would be cool. Second, please check that in your appleTV settings that dynamic range and content matching settings are ON.

[chillout23]

@cdguy, I know he is not using the latest version of KSPlayer at the moment yes. But check out VLC and MPV as I told you in my post and see for yourself how interlacing works. Then compare the results with Tracy Player and you will see what I mean.

"Second, please check that in your appleTV settings that dynamic range and content matching settings are ON." Nothing to do with this. I have full control over these settings.

Let us stay within the topic. If you have a proper interlaced stream, try it out with VLC and MPV in iPlayTV Beta.

[Kimentanm]

In the previous version, I added "yadif=mode=0:parity=auto:deint=0" to options.videoFilters, and the deinterlacing runned well. But now it's not working.

[kingslay]

@Kimentanm 你有把hardwareDecode 设置为false吗？你设置了yadif之后。代码有走到 return outputFrame ?? inputFrame 这一行吗？

[UnknownCoder807]

I use the following in my app and people report it works ok for interlaced

videoFilters = ["yadif=mode=0:parity=-1:deint=1"]

[cdguy]

In the previous version, I added "yadif=mode=0:parity=auto:deint=0" to options.videoFilters, and the deinterlacing runned well. But now it's not working.

Check the path Demo/demo-iOS/demo-iOS/AppDelegate.swift and at line 101 you will see the correct deintrrlace settings: https://github.com/kingslay/KSPlayer/blob/114da4bb06de62dc868d09def5593d1bde6b91dc/Demo/demo-iOS/demo-iOS/AppDelegate.swift#L101 @Kimentanm

[cdguy]

And for the memo, here is the dfference between yadif=0 and yadif=1

-> In FFmpeg, the yadif filter is used for deinterlacing video content. It stands for "yet another deinterlacing filter." When specifying yadif=0 or yadif=1, you are setting the filter mode:

1. yadif=0: This mode is called "auto." In this mode, FFmpeg automatically determines whether to apply deinterlacing or not based on the input video's field order and motion analysis. It will apply deinterlacing when needed.

2. yadif=1: This mode is called "bob." In this mode, the filter will always perform deinterlacing by using a basic "bob" deinterlacing algorithm, which creates a frame for each field, effectively doubling the frame rate.

The choice between these modes depends on your specific video content and requirements. If you're unsure, using the default "auto" mode (yadif=0) is generally a safe choice as it allows FFmpeg to decide whether deinterlacing is necessary. However, if you want to force deinterlacing, you can use yadif=1.

[chillout23]

@kingslay can you add all the different options for deinterlacing in Tracy Player so we can change in GUI the different options?

[cdguy]

@kingslay can you add all the different options for deinterlacing in Tracy Player so we can change in GUI the different options?

@chillout23 don't get me wrong but as YADIF is the best deinterlacer mode in terms of image blending and image quality (I mean it is the one that has the less image degradation), why someone would want to use Bob or Linear?

[chillout23]

@cdguy vf=yadif=mode=1:deint=1 is the correct option for having smooth motion, I tested this on PC with mpv and there is night and day difference between yadif-mode=0 and 1.

In fact, I dont want to use Bob or Linear, I want the option to toggle between yadif-mode=0 and 1.

Yadif-mode=0: removes the horizontal lines, but does keep giving 25p output and no frame doubling. Yadif-mode=1: removes the horizontal lines, but gives 50p output with frame doubling (just like how a TV would deinterlace)

[cdguy]

@nakiostudio @chillout23 In earlier versions of the demo app, when I used the function “yadif=mode=1” the live streams played in slow motion.

1- normally the yadif documentation says "yadif=1 takes true interlaced 50i footage and turns it into astonishingly high-quality 50p output"

however, the live streams are played in slow motion.

@kingslay when I check the bitratea of interlaced streams with a player like Qmplay2 I see that interlaced streams never output 50i consistently.

The bitrate varies between 45i-50i. Besides, when I open an interlaced stream with the demo app, the demo app says that the FPS value is 25p (nothing wrong here, 50i=25p)

As you can imagine if you divide 46i to 2, it is equal to 23p

perhaps if you can find a way to double the frames per second before the stream starts (meaning that the demo app displays the FPS value as 50p) this “might” solve the issue

@chillout23 as mentioned above on the 16th of August, with yadif mode =1 the video playback is in slow motion; and @kingslay also mentions here https://github.com/kingslay/KSPlayer/issues/525#issuecomment-1681949322 that the CPU usage is higher with yadif=1

I agree that yadif=1 should be used for a smoother playback but I guess this needs a "better memory" management from ksplayer, I do not know if Kingslay can find a way to improve the performance with the yadif mode =1 filter

[chillout23]

Yes this is exactly what "we" should try to work on. MPV can do this on tvOS. CPU usage is high but I left the stream for 3 hours and I had no lag whatsoever. The box gets hot though and I monitored 170% (not using all cores) CPU usage in average. I think this is all about CPU optimization and using multiple cores. Apple TV is a power horse.

So lets experiment @kingslay , please add the options to toggle between yadif-mode=0 and 1 :)

[chillout23]

Maybe we need to enable Yadif-mode:1 and exclude hardware decoding, so it is only done in software - maybe the reason for the "slow motion" playback @cdguy

However, this filter cannot be enabled for everything. Non-interlaced high quality content will stutter and cause CPU spikes, so player needs to detect 720i and 1080i and enable filter only for interlaced streams. This is done by MPV engine in Lilly Player for example.

[chillout23]

@kingslay I also suggest you reopen this thread. A smooth interlacer is really important in a player like this, there are still so much interlaced content being broadcasted in the world. Let's try to get this yadif-mode=1 working.

[kingslay]

我可以在demo里面增加这个开关。并且现在yadif也支持硬解码了。你们可以试下在yadif_videotoolbox=mode=1的场景下。视频播放的效果怎么样。

[cdguy]

我可以在demo里面增加这个开关。并且现在yadif也支持硬解码了。你们可以试下在yadif_videotoolbox=mode=1的场景下。视频播放的效果怎么样。

@chillout23 is not a developper; he can only try the tracyplayer testflight app if you want, add the switch and update the tracyplayer testflight app