Chunk methods introduce image glitches

[mxsrm]

I've been testing with multiple sources and all the chunk methods and only select is not creating this problem. lsmash, ffms2 and hybrid do.

If av1an selects chunks to close to a keyframe, sometimes vaporsynth will create blocky artifcats:

vspipe -s 1971 -e 1982 loadscript.vpy - -c y4m | mpv -

(like in the source material)

vspipe -s 1972 -e 1982 loadscript.vpy - -c y4m | mpv -

As said, this happens over multiple source materials from different origins. If you encode the movie en-block (e.g. x264, x265, svt-av1) no glitches appear. Only happens in chunked encoding and using anything but select, which is of course insanely slow.

Anybody have a workaround?

Version information:

> av1an --version
av1an 0.4.1-unstable (rev 45a927d) (release)

* Compiler
  rustc 1.68.0 (LLVM 15.0)

* Target Triple
  x86_64-unknown-linux-gnu

* Date Info
  Commit Date:  2023-04-06

* VapourSynth Plugins
  systems.innocent.lsmas : Found
  com.vapoursynth.ffms2  : Found

> vspipe --version
VapourSynth Video Processing Library
Copyright (c) 2012-2022 Fredrik Mellbin
Core R61
API R4.0
API R3.6
Options: -

[mxsrm]

Closed since no one cares that this completely breaks av1an in 99% of the use cases.

[shssoichiro]

Or just nobody saw this by mistake. A polite reminder to look at this would have been fine.

This is most likely because of the decoder having issues with the input format. What format is the input?

[mxsrm]

BluRay remuxes in mkv format, either H264 (for 1080p) or H265 (for 2160p). I saw this bug over multiple different sources.

[Simulping]

the only way to prevent this is to re encode with x264 qp 0 as a lossless intermediary. although this can be annoying for UHD Blu-rays since H.264 does not have any HDR specifications

[shssoichiro]

You don't have to worry about HDR specifications in the lossless intermediary, as long as they get copied over to the final encode one way or another. You can even do this after the encode, since the HDR data is just metadata stored in the video headers.

That being said it would be nice if there was a way to fix this issue... though I don't currently know what that fix would look like.

[Simulping]

would using mkvmerge as a chunk method change anything? I know it can split files

[shssoichiro]

Not likely, unfortunately this seems to be a sourcing issue. When the source filters attempt to seek to the starting point of a chunk in the input, certain sources can cause this to glitch out. To be honest, I haven't seen this in recent versions of LSmashSource much except with more annoying codecs like VC1.

One option that may be worth trying is making sure you have the latest LSmashSource plugin, although this is still known to glitch on some sources as mentioned, it's at least less common from my experience...

[Simulping]

Hm, @mxsrm can you send me the source where it always glitches out? Gonna test something out

[arch1t3cht]

Is there an example file that breaks in this way? Does this still happen with the ffms 5.0 (which got a lot of seeking fixes)?

There is one big class of video files that "break" in ffms2/lsmash that I am aware of. They're m2ts files, or mkv files improperly muxed from them, where the PPS units switch throughout the video, but without the changed PPS repeating before every IDR frame. This causes corruption when seeking to this IDR frame. Unfortunately, this is something that cannot easily be fixed from the source filter's side (unless you use a radically different and slower method like bestsource). I have seen at least two BDMVs with this behaviour, and there are probably more. mkvmerge will fix this on remuxing by duplicating the PPS as necessary, and hence output correctly seekable mkv files. ffmpeg and MakeMKV will not, so remuxes of these BDMV files will still be broken.

Now, I don't know if this is the reason for this issue, but it's the first thing I would suspect. Having an example file would allow verifying this, or otherwise finding out where the corruption comes from.