4:4:4 inter- failure

[frankplow]

Conformance tests utilising 4:4:4 colour and inter-prediction are currently failing.

[frankplow]

CC https://github.com/ffvvc/FFmpeg/issues/83#issuecomment-1576105102

If you apply this patch. and run "ffmpeg -i tests/conformance/failed/v2/12b444vvc1_A_Sony_2.bit -vsync 0 -y -f rawvideo ffmpeg_patch.yuv" You will find the (64,68) cu on the frame 32 (poc is 32, decoder order is 1) is correct.

The main reason is we derive lumaIntraPredModeC based on "Table 20 – Specification of IntraPredModeC" in spec. It will return INTRA_VDIAG instead INTRA_PLANAR(the correct one)

It's wonderful if you can help double-check the spec and the code. see how to return INTRA_PLANAR

thank you.

git diff
diff --git a/libavcodec/vvc/vvc_ctu.c b/libavcodec/vvc/vvc_ctu.c
index a0a47cc819..c5810f7b0e 100644
--- a/libavcodec/vvc/vvc_ctu.c
+++ b/libavcodec/vvc/vvc_ctu.c
@@ -897,6 +897,11 @@ static void derive_chroma_intra_pred_mode(VVCLocalContext *lc,
                break;
        }
        cu->intra_pred_mode_c = pred_mode_c[intra_chroma_pred_mode][idx];
+
+        if (fc->decode_order == 1 && cu->x0 == 64 && cu->y0 == 68) {
+            cu->intra_pred_mode_c = INTRA_PLANAR;
+    }
+
    }
    if (sps->chroma_format_idc == CHROMA_FORMAT_422 && cu->intra_pred_mode_c <= INTRA_VDIAG) {

[frankplow]

I've been looking into this the last couple days but I'm not really getting anywhere. derive_chroma_intra_pred_mode appears to output the correct mode given the provided parameters. For both lumaIntraPredMode and IntraPredModeC to be INTRA_PLANAR, as we need here, intra_chroma_pred_mode must be non-zero (assuming ACT is disabled). intra_chroma_pred_mode is a part of the CABAC and has an atypical binarisation (H.266 Table 130), but the function for decoding this also seems to perform as expected.

@nuomi2021 How did you work out this was the issue? It appears that the intra prediction mode should be INTRA_PLANAR, as that eliminates the majority of errors in the entire video, but how did you work this out? I've been trying to debug VTM. I think the equivalent of derive_chroma_intra_pred_mode is PU::getFinalIntraMode, but am struggling to get a conditional breakpoint set up quite right.

[nuomi2021]

@frankplow , thank you for checking. We usually follow this process

1. Locate failed frames From YUView diff and VTM output, you know frame 1 (POC 32) is wrong.
2. Locate failed CTU We usually draw CTU as a grid on diff view. In our case, it's 64x64.
3. Locate failed CU You can print out all CU sizes under this CTU, see which is close to your failed block. In your case it's 64x68
4. Locate failed process Then you can compare the decoder output for each step. Inter, Recon, Filter. They all have an entry in vvc_thread.c. In our case it's intra, and it's related to chroma.

the vtm code returns the right chroma mode in https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/blob/master/source/Lib/CommonLib/UnitTools.cpp#L736 You can break at pu.cu->slice->m_iPOC == 32 && pu.blocks[0].x == 64 and pu.blocks[0].y == 68 to see what's happened.

thank you

[nuomi2021]

crated a spec issue at https://jvet.hhi.fraunhofer.de/trac/vvc/ticket/1602