search

fraunhoferhhi / vvenc

VVenC, the Fraunhofer Versatile Video Encoder
https://www.hhi.fraunhofer.de/en/departments/vca/technologies-and-solutions/h266-vvc.html
BSD 3-Clause Clear License
957 stars 172 forks source link

Couple of issues in RDO #158

Closed Jovasa closed 2 years ago

Jovasa commented 2 years ago

Hello, I've been using VVenC as a reference while fixing issues in RDO of uvg266. I've also managed to spot couple of potential issues in VVenC

  1. If transform skip is disabled the following the cabac context is not reset properly during intra chroma search, which cause minor degradation when LFNST is enabled, at least on my cursory tests. https://github.com/fraunhoferhhi/vvenc/blob/a361f5931d3cbee07d9a7d557a7a59a92d9121f8/source/Lib/EncoderLib/IntraSearch.cpp#L917-L920
  2. The contexts for split flags are updated only after the split search is performed.
  3. When transform skip is disabled, the following check is true for MTS_DST7_DST7 when it probably shouldn't. https://github.com/fraunhoferhhi/vvenc/blob/a361f5931d3cbee07d9a7d557a7a59a92d9121f8/source/Lib/CommonLib/TrQuant.cpp#L771 Also just wondering, shouldn't the modes be sorted according to the cost instead of just disabling the last mode(s) when all of the modes fulfill the cost threshold?
  4. And related to the previous issue; --MTSIntraMaxCand=4 segfaults.

Though overall I did find many more issues in uvg266 😅

adamjw24 commented 2 years ago

Thanks, we'll look into that, especially the SegFault issue.

About 2., that is a behavior ported from VTM already. I remember doing this test with some early version of VTM or maybe even JEM without much improvement. But we'll try that again, after all the context derivation has changed since than.

Jovasa commented 2 years ago

Yeah I suspect the issue with 2. is that because the MTT prohibits certain conditions it's impossible to know the actual splitting pattern before the whole CU is searched. Also there was one other issue I forgot to mention originally. https://github.com/fraunhoferhhi/vvenc/blob/a361f5931d3cbee07d9a7d557a7a59a92d9121f8/source/Lib/EncoderLib/IntraSearch.cpp#L1735 violatesMtsCoeffConstraint should also be reset here.

Jamaika1 commented 2 years ago

Compatibility of codecs from different developers. uvg266e 0.2.3-5ea9132 vvdec 1.5.0-c9fb1c5

uvg266e.exe -i "113.yuv" -o "output_10bit.266" --no-info --input-bitdepth 8 --threads 4 --input-res 1280x720 --input-fps 30000/1001 --input-format P420 --gop=16 --ref=4 --bipred --gop lp-g16d4t1 -p 256 --bitrate 3000000 -n 200 --preset medium VVDecoderApp.exe -b output_10bit.266 -o yyy.yuv -t 4

Fraunhofer VVC/H.266 Decoder VVdeC, version 1.5.0 [THREADS=4; PARSE_DELAY=4; SIMD=SCALAR]
POC    0 LId:  0 TId: 0 ( I-SLICE, QP 20 ) [DT  0.014] [L0 ] [L1 ]
POC   16 LId:  0 TId: 0 ( P-SLICE, QP 20 ) [DT  0.048] [L0  0c] [L1 ]
POC    8 LId:  0 TId: 0 ( B-SLICE, QP 27 ) [DT  0.025] [L0  0c] [L1 16 ]
POC    4 LId:  0 TId: 0 ( B-SLICE, QP 23 ) [DT  0.026] [L0  0c] [L1 8 16 ]
POC    2 LId:  0 TId: 0 ( B-SLICE, QP 34 ) [DT  0.015] [L0  0c] [L1 4 8 16 ]
POC    1 LId:  0 TId: 0 ( B-SLICE, QP 29 ) [DT  0.020] [L0  0c] [L1 2 4 8 16 ]
POC    6 LId:  0 TId: 0 ( B-SLICE, QP 20 ) [DT  0.021] [L0  4c0 ] [L1 8 16 ]
POC    3 LId:  0 TId: 0 ( B-SLICE, QP 26 ) [DT  0.020] [L0  2c0 ] [L1 4 8 16 ]
POC    5 LId:  0 TId: 0 ( B-SLICE, QP 24 ) [DT  0.017] [L0  4c0 ] [L1 6 8 16 ]
POC    7 LId:  0 TId: 0 ( B-SLICE, QP 24 ) [DT  0.024] [L0  6c4 0 ] [L1 8 16 ]
POC   12 LId:  0 TId: 0 ( B-SLICE, QP 19 ) [DT  0.030] [L0  8c0 ] [L1 16 ]
POC   10 LId:  0 TId: 0 ( B-SLICE, QP 20 ) [DT  0.016] [L0  8c0 ] [L1 12 16 ]
POC    9 LId:  0 TId: 0 ( b-SLICE, QP 23 ) [DT  0.014] [L0  8c0 ] [L1 10 12 16 ]
POC   11 LId:  0 TId: 0 ( b-SLICE, QP 20 ) [DT  0.019] [L0  10c8 0 ] [L1 12 16 ]
POC   14 LId:  0 TId: 0 ( B-SLICE, QP 20 ) [DT  0.019] [L0  12c8 0 ] [L1 16 ]
vvdecapp [error]: decoding failed:  (decoder requires restart) detail: caught unknown exception
ERROR: In function "void vvdec::DecSlice::parseSlice(vvdec::Slice*, vvdec::InputBitstream*, int)" in DecSlice.cpp:179: Expecting a terminating bit
ERROR CONDITION: !binVal
VVCSoftware: VTM Decoder Version 16.2 [Windows][GCC 13.0.0][64 bit]
POC    0 LId:  0 TId: 0 ( IDR_N_LP, I-SLICE, QP 20 ) [DT  0.047] [L0] [L1] [Checksum:07036092,01cbdb60,01cd3e00,(OK)]
POC   16 LId:  0 TId: 0 ( TRAIL, P-SLICE, QP 20 ) [DT  0.078] [L0 0c] [L1] [Checksum:06fd03a4,01b810f2,01d9f2a9,(OK)]
POC    8 LId:  0 TId: 0 ( TRAIL, B-SLICE, QP 27 ) [DT  0.047] [L0 0c] [L1 16] [Checksum:0703025b,01bedb94,01ded136,(OK)]
POC    4 LId:  0 TId: 0 ( TRAIL, B-SLICE, QP 23 ) [DT  0.067] [L0 0c] [L1 8 16] [Checksum:07032fd8,01c60315,01d7059e,(OK)]
POC    2 LId:  0 TId: 0 ( TRAIL, B-SLICE, QP 34 ) [DT  0.031] [L0 0c] [L1 4 8 16] [Checksum:0701feed,01c788e5,01cce173,(OK)]
POC    1 LId:  0 TId: 0 ( TRAIL, B-SLICE, QP 29 ) [DT  0.031] [L0 0c] [L1 2 4 8 16] [Checksum:0703ade6,01caf364,01cd3dec,(OK)]
POC    3 LId:  0 TId: 0 ( TRAIL, B-SLICE, QP 26 ) [DT  0.047] [L0 2c 0] [L1 4 8 16] [Checksum:0703cec2,01c785c2,01d8ed72,(OK)]
POC    6 LId:  0 TId: 0 ( TRAIL, B-SLICE, QP 20 ) [DT  0.062] [L0 4c 0] [L1 8 16] [Checksum:0704bebd,01c36752,01deeb63,(OK)]
POC    5 LId:  0 TId: 0 ( TRAIL, B-SLICE, QP 24 ) [DT  0.031] [L0 4c 0] [L1 6 8 16] [Checksum:070268e5,01c64c84,01d8d861,(OK)]
POC    7 LId:  0 TId: 0 ( TRAIL, B-SLICE, QP 24 ) [DT  0.047] [L0 6c 4 0] [L1 8 16] [Checksum:07017439,01be99a4,01e3fe0d,(OK)]
POC   12 LId:  0 TId: 0 ( TRAIL, B-SLICE, QP 19 ) [DT  0.063] [L0 8c 0] [L1 16] [Checksum:0701b4fd,01b76ab0,01dd6bad,(***ERROR***)] [rxChecksum:0701b3d5,01b770c8,01dd7e39]
POC   10 LId:  0 TId: 0 ( TRAIL, B-SLICE, QP 20 ) [DT  0.063] [L0 8c 0] [L1 12 16] [Checksum:07055dcb,01bbc4c3,01ddba4e,(***ERROR***)] [rxChecksum:07055d28,01bbc986,01ddc73b]
POC    9 LId:  0 TId: 0 ( TRAIL, B-SLICE, QP 23 ) [DT  0.062] [L0 8c 0] [L1 10 12 16] [Checksum:07054d2b,01ba28da,01dcfa0d,(***ERROR***)] [rxChecksum:07054bec,01ba2c4e,01dd0371]
POC   11 LId:  0 TId: 0 ( TRAIL, B-SLICE, QP 20 ) [DT  0.078] [L0 10c 8 0] [L1 12 16] [Checksum:0705e90a,01b67b66,01dbcd29,(***ERROR***)] [rxChecksum:0705e20f,01b6803f,01dbd6ff]
POC   14 LId:  0 TId: 0 ( TRAIL, B-SLICE, QP 20 ) [DT  0.063] [L0 12c 8 0] [L1 16] [Checksum:07020553,01b8396f,01db7d04,(***ERROR***)] [rxChecksum:070201f4,01b83d73,01db8a1d]
POC   13 LId:  0 TId: 0 ( TRAIL, B-SLICE, QP 22 ) [DT  0.046] [L0 12c 8 0] [L1 14 16] [Checksum:0700430a,01b70dd0,01dcbe5d,(***ERROR***)] [rxChecksum:07003d89,01b71417,01dccb57]
POC   15 LId:  0 TId: 0 ( TRAIL, B-SLICE, QP 21 ) [DT  0.046] [L0 14c 12 8 0] [L1 16] [Checksum:06ff8988,01b72702,01dba108,(***ERROR***)] [rxChecksum:06ff9aed,01b72737,01dbb57f]

ERROR: In function "decompressSlice" in DecSlice.cpp:259: Expecting a terminating bit
adamjw24 commented 2 years ago

Compatibility of codecs from different developers. uvg266e 0.2.3-5ea9132 vvdec 1.5.0-c9fb1c5

I will give you some advice here, please follow it. If you encoded stuff with uvg266 and it doesn't decode with vvdec nor VTM, its a bug in uvg266 and should be posted there. If it decodes with VTM and doesn't with vvdec, its a bug with vvdec and should be reported in the vvdec repo. None of those cases should be reported in this repo. Enc/Dec for international standards don't have compatibilities. If a pair is incompatible, one of those has a bug.