search

gianni-rosato / svt-av1-psy

The Scalable Video Technology for AV1 (SVT-AV1 Encoder and Decoder) with perceptual enhancements for psychovisually optimal AV1 encoding
https://svt-av1-psy.com
BSD 3-Clause Clear License
259 stars 20 forks source link

[BUG] Major blocking issue with QMs enabled >= preset 5 #56

Closed gianni-rosato closed 5 months ago

gianni-rosato commented 5 months ago

Overview Videos encoded with Preset 5 or greater feature intense visual bugs where flashes occur in video.

Branch In which branch does the issue appear to be occurring?

Reproduction Steps to reproduce the behavior:

  1. Get SVT-AV1 2.1.0
  2. Download balenciaga.mp4
  3. Encode with QMs enabled (sample command: ffmpeg -hide_banner -loglevel error -i balenciaga.mp4 -pix_fmt yuv420p10le -map 0:v:0 -c:v libsvtav1 -svtav1-params preset=5:keyint=300:crf=50:enable-qm=1 -an balenciaga.ivf)
  4. Observe visual errors in the result

Expected behavior No visual bugs should be present

Terminal Output N/A

Platform (please complete the following information):

Version String (please complete the following information): Mainline: SVT-AV1 v2.1.0-14-gced0d040 (release) PSY:

SVT-AV1-PSY v2.1.0 (release)
PSY Release: N/A

Additional context / Relevant Files Link to mainline bug: https://gitlab.com/AOMediaCodec/SVT-AV1/-/issues/2189

gianni-rosato commented 5 months ago

Wanted to mention this in our repo despite being a mainline bug because this is fixed as of 16d93ce252a5a5fcac2be8912df32f38c85afd53 and will be merged to master soon, as this is rather urgent & a fix is crucial to be implemented ASAP.

gitoss commented 5 months ago

It would be great if this & a future actual fix would make it into a Windows binary of the psy branch, too - thanks!

Safari77 commented 5 months ago

Why I can't reproduce it? My video does not have the glitch. svt-av1 2.1.0, libdav1d 1.4.2, mpv git, Fedora 39 x86_64.

juliobbv-p commented 5 months ago

@Safari77 are you using mainline SVT-AV1? Make sure version reads SVT-AV1 Encoder Lib v2.1.0. PSY has the workaround applied already, so it won't repro there. You also have to use the specific video in the repro steps.

Safari77 commented 5 months ago

I use the mainline version, and the command line provided. balenciaga.ivf at position 1.400s is just fine.

Svt[info]: -------------------------------------------
Svt[info]: SVT [version]:   SVT-AV1 Encoder Lib v2.1.0
Svt[info]: SVT [build]  :   GCC 13.2.1 20240316 (Red Hat 13.2.1-7)  64 bit
Svt[info]: LIB Build date: May 18 2024 15:39:56
Svt[info]: -------------------------------------------
Svt[info]: Number of logical cores available: 20
Svt[info]: Number of PPCS 140
Svt[info]: [asm level on system : up to avx2]
Svt[info]: [asm level selected : up to avx2]
Svt[info]: -------------------------------------------
Svt[info]: SVT [config]: main profile   tier (auto) level (auto)
Svt[info]: SVT [config]: width / height / fps numerator / fps denominator       : 1080 / 1920 / 30 / 1
Svt[info]: SVT [config]: bit-depth / color format                   : 10 / YUV420
Svt[info]: SVT [config]: preset / tune / pred struct                    : 5 / PSNR / random access
Svt[info]: SVT [config]: gop size / mini-gop size / key-frame type          : 300 / 32 / key frame
Svt[info]: SVT [config]: BRC mode / rate factor                     : CRF / 50 
Svt[info]: SVT [config]: AQ mode / variance boost                   : 2 / 0
Svt[info]: -------------------------------------------
juliobbv-p commented 5 months ago

@Safari77 can you upload the ivf somewhere for us/the mainline team to analyze? It seems like this bug is environment-specific, which makes it even harder to repro 😕. By any chance, does it repro if you use preset 13 instead?

gianni-rosato commented 5 months ago

Having just tested on both Linux & macOS, I can confirm this appears to only be happening with binaries compiled on macOS. I've updated the issue accordingly.

We're going to continue to try and narrow this down as much as possible to help address this pretty nasty issue. We don't want 16d93ce252a5a5fcac2be8912df32f38c85afd53 to be a permanent fix, as it harms performance. Right now, we can confirm Linux x86_64 AVX2 is unaffected.

Safari77 commented 5 months ago

Oh okay, maybe it is ASM optimization or compiler bug then?

gianni-rosato commented 5 months ago

Update: this issue appears while using Preset 13 even when compiling without ASM (c-only). It could be something entirely unrelated, but if Linux users would like to attempt to reproduce:

ffmpeg -i balenciaga.mp4 -pix_fmt yuv420p10le -strict -2 -f yuv4mpegpipe - | SvtAv1EncApp -i - -b balenciaga_p13_conly_linux.ivf --preset 13 --keyint 300 --crf 50 --enable-qm 1
Safari77 commented 5 months ago

Yes, I can reproduce the bug with preset 13.

Uranite commented 5 months ago

Cant reproduce on windows 11 using ffmpeg 7.0.1 from GyanD using the same command in the reproduction steps uploaded the ivf file just incase i missed anything https://pixeldrain.com/u/3XUqbrfa

Safari77 commented 5 months ago

Cant reproduce on windows 11 using ffmpeg 7.0.1 from GyanD using the same command in the reproduction steps uploaded the ivf file just incase i missed anything https://pixeldrain.com/u/3XUqbrfa

The command line for preset 5 or preset 13?

Uranite commented 5 months ago

Cant reproduce on windows 11 using ffmpeg 7.0.1 from GyanD using the same command in the reproduction steps uploaded the ivf file just incase i missed anything https://pixeldrain.com/u/3XUqbrfa

The command line for preset 5 or preset 13?

preset 5, i'm using this command image

Safari77 commented 5 months ago

can you also try the preset 13 command line from https://github.com/gianni-rosato/svt-av1-psy/issues/56#issuecomment-2156768035

Uranite commented 5 months ago

can you also try the preset 13 command line from #56 (comment)

im able reproduce the issue with preset 13 tested using svt-av1 v2.1.0 compiled with .\build.bat release static no-dec here is the ivf file https://pixeldrain.com/u/9LyC924J edit: i can also reproduce it with the following commands ffmpeg -hide_banner -loglevel error -i balenciaga.mp4 -pix_fmt yuv420p10le -map 0:v:0 -c:v libsvtav1 -svtav1-params preset=13:keyint=300:crf=50:enable-qm=1 -an balenciaga_p13.ivf

juliobbv-p commented 5 months ago

Thanks for the help y'all. After running different commands and analyzing results, it turns out that the preset 13 issue is a different one, so I filed a separate issue here: https://gitlab.com/AOMediaCodec/SVT-AV1/-/issues/2191.

The original preset 5 issue only seems to occur on Apple Silicon environments with NEON opts on, potentially on Windows on ARM too, so I updated the original issue to reflect our findings: https://gitlab.com/AOMediaCodec/SVT-AV1/-/issues/2189.

gitoss commented 5 months ago

The original preset 5 issue only seems to occur on Apple Silicon environments with NEON opts on, potentially on Windows on ARM too, so I updated the original issue to reflect our findings: https://gitlab.com/AOMediaCodec/SVT-AV1/-/issues/2189.

... meaning the QM disable fix is not necessary except for these two architectures. I'm building svt-av1 with avx512 myself now, so I can un-patch the respective commit manually. https://github.com/gianni-rosato/svt-av1-psy/commit/16d93ce252a5a5fcac2be8912df32f38c85afd53

juliobbv-p commented 5 months ago

Yeah, a more will scoped down workaround will be committed soon and be available with a B release.

gianni-rosato commented 5 months ago

d5ca0fcb30999ee551c470770c58d924648aba39 scopes this down to ARM-based systems. Thanks @juliobbv-p. It is currently available in the testing branch (which is not advisable for regular usage) and will be available in master soon.

gianni-rosato commented 5 months ago

I'm going to close this issue for now, since we have "fixed" it on our end with as minimal impact as we can manage with our smaller team. Once mainline takes a look at things, we should be able to revert this.

d5ca0fcb30999ee551c470770c58d924648aba39