search

CrendKing / avisynth_filter

DirectShow filters that put AviSynth and VapourSynth into video playing
MIT License
107 stars 8 forks source link

RGB48 doesn't work correctly #34

Closed chainikdn closed 3 years ago

chainikdn commented 3 years ago

Consider a 4:4:4 10-bit video (yuv444p10le). With RGB48 option checked (which is by default) the image is upside-down and with wrong colors. Pin info shows "Connection media type: Video: RGB0 1280x720 23.976fps". With RGB48 unchecked connection is made with "Video: RGB32 1280x720 23.976fps" and everything works just fine.

CrendKing commented 3 years ago

Can't reproduce with my own videos. I need more details. Can you actually use the "Bug report" issue template? Fill as much as you can. Also upload the video somewhere, if copyright is not a concern.

1

My video metadata:

Video
ID                             : 1
Format                         : AVC
Format/Info                    : Advanced Video Codec
Format profile                 : High 4:4:4 Predictive@L3.1
Format settings                : CABAC / 4 Ref Frames
Format settings, CABAC         : Yes
Format settings, Reference fra : 4 frames
Codec ID                       : avc1
Codec ID/Info                  : Advanced Video Coding
Duration                       : 5 s 0 ms
Bit rate                       : 1 680 kb/s
Width                          : 1 280 pixels
Height                         : 720 pixels
Display aspect ratio           : 16:9
Frame rate mode                : Constant
Frame rate                     : 25.000 FPS
Chroma subsampling             : 4:4:4
Bit depth                      : 10 bits
Scan type                      : Progressive
Bits/(Pixel*Frame)             : 0.073
Stream size                    : 1.00 MiB (82%)
Writing library                : x264 core 161
Encoding settings              : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=4 / threads=22 / lookahead_threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=81 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Codec configuration box        : avcC

created with this command:

ffmpeg -i .\25FPS.mp4 -t 5 -pix_fmt yuv444p10le out.mp4
chainikdn commented 3 years ago

https://drive.google.com/file/d/1E8A_3Qz29Rq1FLKGPITSJoPeHQlw1vsE/view?usp=sharing

in fact the image swaps upside-down after SVP kicks in

Avs in: RGB0 1280x720 23.976fps Avs out: P010 2048x720 (16:9) 85.249fps

the script does only the usual stuff

ConvertToYUV420()
ConvertBits(16)
CrendKing commented 3 years ago

I can reproduce now with the script (that's why I ask for the script in the issue template). In fact, the only part matters is ConvertToYUV420(). I suspect the upside down is due to the bitmap ordering in Windows API, and wrong color is due to https://github.com/AviSynth/AviSynthPlus/issues/146. I will take closer look at this, but don't have high hope for the wrong color problem since it is due to incompatibility of endian between DirectShow and AviSynth+.

chainikdn commented 3 years ago

well, it probably will be better to support yuv 4:4:4 directly

CrendKing commented 3 years ago

Like Y410? I could add that if there is demand. Please open an issue for tracking.

chainikdn commented 3 years ago

yeah, add a 4:4:4 checkbox, Y410 -> CS_YUV444P16 or just fix the RGB48 bytes order (if this is the reason for the "color problem") or both :)

CrendKing commented 3 years ago

OK. I took a look. Turns out, the "RGB48" in LAV filters is actually AV_PIX_FMT_RGB48LE, which in ffmpeg is defined as "packed RGB 16:16:16, 48bpp, 16R, 16G, 16B, the 2-byte value for each R/G/B component is stored as little-endian". The little endian part is fine, but the component ordering is R-G-B. Unfortunately, AviSynth+ only supports B-G-R ordering (aka BGR48).

I made this mistake because the "RGB32" and "RGB24" from LAV filters are both B-G-R. I blindly assumed their RGB48 would be B-G-R as well.

I created an issue on LAV filters as petition to support BGR48, but the chance of LAV supporting it is probably not high. As this point, I'll just remove RGB48 option from AVSF as I do not want to implement the costly pixel-by-pixel ordering conversion.

CrendKing commented 3 years ago

BTW, normally AVSF only supports formats that are both supported by LAV Filters and AviSynth+. Since Y410 and Y416 are not supported by AviSynth+, the only 4:4:4 format available is YV24, which is only 8 bit. It's not great, but better than nothing.

chainikdn commented 3 years ago

well, in this case I'm gonna uncheck everything except 4:2:0 formats at the installation time "yuv 4:4:4 p10 -> yuv 4:2:0 p16" looks better than "yuv 4:4:4 p10 -> rgb32 -> yuv 4:2:0 p8"

CrendKing commented 3 years ago

Or YUV 4:4:4 P10 -> YUV 4:4:4 P8 (YV24) with the latest snapshot. Or YUV 4:4:4 P10 -> YUV 4:2:2 P10 (P210). 1

BTW, Nevcairiel rejected the request to support BGR48, which is totally expected and respected.

chainikdn commented 3 years ago

since SVP plugins will only accept 4:2:0, loosing both color resolution and color depth is worse than loosing color resolution only

on the other side - "Do not rely on LAV to convert" (c) so the BEST possible solution would be implementing Y410 -> planar YUV444 conversion in AVSF, plus RGB48 -> BGR48 for compatibility ;)

CrendKing commented 3 years ago

on the other side - "Do not rely on LAV to convert" (c)

We already have LAV (basically ffmpeg) and AviSynth+ in the scene, both of which can do arbitrary conversion. I do not want to mingle another conversion layer in between. If I have to, I'll just convert with ffmpeg swscale.

If you really really want native 4:4:4 P10/16 support, maybe ask AviSynth+ to support Y410/Y416. It will be much easier for them to add one format (which is already similar to YUY2) than me introducing a whole ffmpeg pipeline.

CrendKing commented 3 years ago

Y410 -> planar YUV444 conversion in AVSF

How would this help you? You said SVP only support 4:2:0. So P010/016 is pretty much what you need, which are already supported.

Does SVP support RGB formats (including RGB48LE)? If so, then AviSynth+ supporting RGB48LE would be huge for you. But that's way less likely to happen (than Y410/416).

chainikdn commented 3 years ago

How would this help you?

I'm just saying how it could work in a perfect world. I'm good with "uncheck everything except 4:2:0 formats at the installation time", as long as this doesn't introduce some other compatibility problem(s). Are we sure LAV will convert everything to YUV4:2:0?

CrendKing commented 3 years ago

I'm just saying how it could work in a perfect world.

Well, in a perfect world, SVP should be able to handle 4:4:4 and 4:2:2 format, thus taking YV24 P210 natively without losing details, no?

Are we sure LAV will convert everything to YUV4:2:0?

That's question for Nevcairiel. Or you can read all the code (maybe starting from https://github.com/Nevcairiel/LAVFilters/blob/master/decoder/LAVVideo/pixconv/convert_generic.cpp) and judge by yourself ;-) Hail to open source!