mt_merge vs overlay chroma plane shift

TbtBI commented 3 years ago

Hi,

FFVideoSource(yv12_left_chroma)
Crop(0, 138, -0, -138)
m=mt_edge()
Overlay(BlankClip(last), Tweak(sat=10), mask=m, mode="chroma")

FFVideoSource(yv12_left_chroma)
Crop(0, 138, -0, -138)
m=mt_edge()
mt_merge(BlankClip(last), Tweak(sat=10), m, y=2, luma=true)

https://imgaa.com/gallery/lTRmOiv8NroJcje

You can see there is shift between both images. I guess mt_merge keep the original chroma position (left) and Overlay is using center chroma position?

Also not related to mt_merge but something about chroma position:

if using yv12 clip with left chroma position, extract every plane, add some filtering to them and then use combineplanes to receive yv12, does the chroma position of the resulted clip will be left?
if using yuv420p10 clip with top left chroma position, extract every plane, add some filtering to them and then use combineplanes to receive yuv420p10, does the chroma position of the resulted clip will be top left?

pinterf commented 3 years ago

I've put chroma placement support into mt_merge in 2.2.15. mt_merge new parameter hint when luma=true and 4:2:0/4:2:2 'cplace' (2.2.15-). Possible values "mpeg1" or "mpeg2" (default) It affects how non-subsampled full-resolution mask is handled there.

Then I've added 'placement' parameter to Avisynth 'Layer' as well. It works the same way as it does in masktools. http://avisynth.nl/index.php/Layer

Poor Overlay has no such parameter and due to the programming complexity, probably it won't get the featulre in the near future.

When you extract chroma planes, filter them and put it back as-is, everything is unaltered chroma-placement-wise. But chroma placement may become important when a non-subsampled mask should be somehow projected onto chroma plane. Or when a luma-resolution metrics from both luma and chroma are involved in a process.

TbtBI commented 3 years ago

Thanks.

I didn't notice parameter cplace. Is it possible to add top left chroma location too? It's common chroma location nowadays for UHD BD/HDR videos.

pinterf commented 3 years ago

Good request. Please test this 2.2.27 test build: https://drive.google.com/uc?export=download&id=1UzcSJf-w_96oJEYKl2U46GarB720QgOn

mt_merge now accepts cplace="topleft" (besides "mpeg1" or "mpeg2") valid only for 4:2:0 color spaces, otherwise ignored.

TbtBI commented 3 years ago

Using SetMaxCPU("none") is ok. Using SSE/AVX doesn't look correct.

https://i.imgaa.com/2021/05/13/609d29c09b628942077491.png (none) https://i.imgaa.com/2021/05/13/609d29c0b6538944511065.png (SSE/AVX)

pinterf commented 3 years ago

Thanks, next try please: -test2 https://drive.google.com/uc?export=download&id=1u_qFJHq7e47aBeL7Iow3MWhEYGM4pFVk

TbtBI commented 3 years ago

It's ok.

Thanks.

pinterf commented 3 years ago

Added in commit https://github.com/pinterf/masktools/commit/e41393577b605174bed4c7f37419b0d78e5ddc66

pinterf / masktools

mt_merge vs overlay chroma plane shift #17