AOM AV1 and SVT-AV1 Encoders Force 4:2:0 Chroma Subsampling (I420)

[WinterPhoenix]

Operating System Info

Windows 10

Other OS

No response

OBS Studio Version

30.1.1

OBS Studio Version (Other)

No response

OBS Studio Log URL

https://obsproject.com/logs/ygEQcKH0QiDfJy5p

OBS Studio Crash Log URL

No response

Expected Behavior

Setting the Color Format in OBS to I444 and recording using AOM AV1 or SVT-AV1 should produce a video file with 4:4:4 Chroma Subsampling.

Current Behavior

The output video file has 4:2:0 Chroma Subsampling. Example video: https://winteris.moe/share/2024-04-01_00-30-33.mkv MediaInfo output of Example video:

General
Unique ID                                : 67236476256232977644775950872213691578 (0x329546385FB1E5387F1F6985A5A39CBA)
Complete name                            : D:\Dev Showcases\2024-04-01 00-30-18.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 313 KiB
Duration                                 : 5 s 909 ms
Overall bit rate                         : 434 kb/s
Frame rate                               : 60.000 FPS
Writing application                      : Lavf60.16.100
Writing library                          : Lavf60.16.100
ErrorDetectionType                       : Per level 1

Video
ID                                       : 1
Format                                   : AV1
Format/Info                              : AOMedia Video 1
Format profile                           : Main@L5.0
Codec ID                                 : V_AV1
Duration                                 : 5 s 900 ms
Width                                    : 2 304 pixels
Height                                   : 1 296 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 60.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Default                                  : No
Forced                                   : No
Color range                              : Full
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : A_AAC-2
Duration                                 : 5 s 909 ms
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Title                                    : Track1
Default                                  : No
Forced                                   : No

Steps to Reproduce

1. Set Advanced > Video > Color Format to I444 (8-bit, 4:4:4, 3 planes)
2. Change Output > Output Mode to Advanced
3. Set Output > Recording > Video Encoder to AOM AV1 or SVT-AV1
4. Press Start Recording
5. Observe the output video file has 4:2:0 Chroma Subsampling upon Stopping Recording

Anything else we should know?

I think the relevant code is here, but I'm not massively familiar with OBS's codebase: https://github.com/obsproject/obs-studio/blob/3a1035576eb6a7c60c1757b3498d97da5d7bfb3b/plugins/obs-ffmpeg/obs-ffmpeg-av1.c#L53-L65

Also, setting -profile high or -pix_fmt yuv444p has no effect. While AV1's Main Profile doesn't support 4:4:4, its High and Professional Profiles do:

Is this just a relic of the AV1 ecosystem not really supporting High/Professional Profiles until relatively recently?

[flaeri]

svt-av1 only does yuv420 and yuv420p10le, so that is expected.

As for AOM, it does have support, and I can confirm your findings. It will be converted to 4:2:0 chroma subsampling.

[flaeri]

Simply adding a case for I444 like suggested does give you 4:4:4 with AOM, but throws on svt-av1 (cus it of course cannot handle it), so it requires a bit more than that.

[WinterPhoenix]

Damn, didn't know that about SVT-AV1...it's much more usable performance-wise than AOM in my experience so far.

Relevant issue: https://gitlab.com/AOMediaCodec/SVT-AV1/-/issues/1433

[flaeri]

It is what it is :) I experimented a bit, but its very annoying to deal with to be honest. There a multiple formats that do work, and multiple that dont.

Adding cases/ifs for essentially all formats, for each encoder is pretty meh...

If just yuv444 gets added, then it would set the precedent, and I would presume requests would come for yuv422, 10 bit and friends, rgb, what have you. I don't really feel like opening a PR I personally would not merge.

If you or someone would make a very compelling argument for just adding a specific one (because it is very useful in x scenario), then maybe :) I personally can't.

av1 is a highly compressed format, and aom is not exactly well suited for realtime encoding (OBS use cases), so it makes sense that it would primarily be used with yuv420. At least that is my take.

[WinterPhoenix]

So here's my use case. When I stream, I record the VOD separately in significantly higher quality for platforms that support higher quality video than x264 @ 6Mbps w/ 4:2:0. So basically everything that's not Twitch (barring the Enhanced Broadcasting Beta).

Up until now, I've just been doing this using x264 with the software encoder because it's fast, compatible, and good quality...but only if you crank CRF/bitrate/whatever. It's not remotely efficient. A recent 3 hour stream I had recorded a VOD with a size of 21GB (CRF=14). Re-encoding that to AV1 after the fact, with roughly the same amount of perceived visual quality (CRF=28), reduces that video's size to ~11GB.

Storage, how long it takes to upload to other platforms, and how long it takes to process on those platforms are the key concerns for me here, and re-encoding to AV1 locally takes a while too so I'd prefer to just have OBS record straight to a video file I can use.

All of this said...AOM is definitely not fast enough for realtime encoding on CPU right now unless you've got a Threadripper or something. I sort of went into this believing SVT had support for 4:4:4 chroma subsampling because why wouldn't it at this point. So, I'm not sure what the solution is for my use case right now at least.

OBS should definitely support 4:4:4 chroma subsampling for everything possible if it's supposed to be the king of streaming/recording software though. Especially if OBS has an option that says "turn this on for 4:4:4" and FFmpeg/the codec supports it.

There's not even anything put in the logs about trying to use a Color Format that's not supported by FFmpeg/codec/etc right now.

[JCBuck]

I have previously played around with the custom (FFMPEG) recording setting and was able to use AOM AV1 to encode with i444 selected, but it would crash since it would randomly drop frames as I've learned in the following issue from @flaeri. https://github.com/obsproject/obs-studio/issues/7472

Thank you for pointing out the cause in the code @WinterPhoenix. I tested the following modification and it seems to be able to produce a video file with 4:4:4 encoded in AV1:

static void av1_video_info(void *data, struct video_scale_info *info)
{
    UNUSED_PARAMETER(data);

    switch (info->format) {
    case VIDEO_FORMAT_I010:
    case VIDEO_FORMAT_P010:
        info->format = VIDEO_FORMAT_I010;
        break;
    /* Allow 444 for av1 encoders in Standard recording settings */
    case VIDEO_FORMAT_I444: 
        info->format = VIDEO_FORMAT_I444;
        break;
    default:
        info->format = VIDEO_FORMAT_I420;
    }
}

I've made a fork with this modification in this commit here: https://github.com/JCBuck/obs-studio/commit/70ebc1ffb810eb109ad118360c227436ce9b3168

Do note when compiling this on Windows 10, the build guide is slightly out of date, and actually requires an even later version of windows SDK and CMake then what Visual Studio may be installed with ( I installed 3.3 and put it in the path with VS developer command prompt, but the cmake config suggested >= 2.8 and the build guide was suggesting >= 2.4). When I built it, OBS seems to be at 30.2.2. Visual Studio may throw linker errors but rebuilding the solution several times seems to work.

This is the properties of a sample av1 encoded video file with a 1440x1080@30 i444 source that primarily was just screen recording a code editor. (I also had 10s for I frames, CRF at 1, with qp=0 crf=0 and lossless=1 just in case to try lossless encoding but not sure if it's working or not)

Video
ID                             : 1
Format                         : AV1
Format/Info                    : AOMedia Video 1
Format profile                 : High@L4.0
Codec ID                       : av01
Duration                       : 37 s 67 ms
Bit rate                       : 365 kb/s
Maximum bit rate               : 2 500 kb/s
Width                          : 1 440 pixels
Height                         : 1 080 pixels
Display aspect ratio           : 4:3
Frame rate mode                : Constant
Frame rate                     : 30.000 FPS
Color space                    : YUV
Chroma subsampling             : 4:4:4
Bit depth                      : 8 bits
Bits/(Pixel*Frame)             : 0.008
Stream size                    : 1.61 MiB (97%)
Color range                    : Full
Color primaries                : BT.709
Transfer characteristics       : sRGB/sYCC
Matrix coefficients            : BT.709
Codec configuration box        : av1C