Full/partial color range switch is broken when using custom output through FFmpeg

[miabaka]

Operating System Info

Windows 11

Other OS

No response

OBS Studio Version

Other

OBS Studio Version (Other)

29.0.1

OBS Studio Log URL

https://obsproject.com/logs/BgY3fRZrYgONbSLt

OBS Studio Crash Log URL

No response

Expected Behavior

Full/partial color range switch is working properly with custom FFmpeg output

Current Behavior

If you select full color range in advanced settings and try to record video through custom FFmpeg output, OBS marks output file as full range in metadata (you can see it with MediaInfo, for example), but actually encodes it always as partial range, so colors get washed out

Steps to Reproduce

1. Launch OBS
2. Setup custom FFmpeg output (for example, mkv container with plain H.264 through libx264 or h264_nvenc)
3. Set color range to full
4. Record short video
5. Look at washed out result

Anything else we should know?

It worked properly until release 28.x.x

Broken full range example

OBS settings

MediaInfo

Reference image

Frame from video (screenshot of clsid2/mpc-hc with madVR)

[flaeri]

I am a little bit confused about this report. In the title, and in several other places, you type "RGB color format", but in the settings and screenshots, you show i444/yuv444.

Additionally, full range seem to work fine (is full range, and tagged as such) for me with BGRA for me in custom ffmpeg output, both with cineform and utVideo. The issue posted as is makes it sound like its a core issue with custom ffmpeg output, but that has not been my experience with these other codecs.

If I set i444 color format, full range color, and the libx264, I can reproduce the color range mismatch you're describing. I still think its important that the issues be as specific as possible.

Is this issue multiple issue? If yes, I personally think they should be split into seperate issue, and each have their own reproduction steps.

[miabaka]

I am a little bit confused about this report. In the title, and in several other places, you type "RGB color format", but in the settings and screenshots, you show i444/yuv444.

Oh, sorry. I originally wanted to report only color range issue, but then I decided that this problems may be caused by the same thing and mentioned problem with RGB output too

Additionally, full range seem to work fine (is full range, and tagged as such) for me with BGRA for me in custom ffmpeg output, both with cineform and utVideo. The issue posted as is makes it sound like its a core issue with custom ffmpeg output, but that has not been my experience with these other codecs.

Oh, interesting. In code both problems look like a core issue:

In libobs/media-io/video-io.c color conversion output isn't configured properly, so wrong color range used even before encoding and full range must be broken with any codec and encoder if output format doesn't matches input

in plugins/obs-ffmpeg/obs-ffmpeg-output.c config.colorspace is set like its YUV/YCbCr even if actually RGB color space is used, so encoders that rely on this must not encode RGB properly too. Maybe cineform and utVideo just ignores this and always assumes that input is RGB?

If I set i444 color format, full range color, and the libx264, I can reproduce the color range mismatch you're describing. I still think its important that the issues be as specific as possible.

Is this issue multiple issue? If yes, I personally think they should be split into seperate issue, and each have their own reproduction steps.

Oh, it's a multiple issue I guess. I agree with you, how can I split it?

[flaeri]

You may be right. RGB formats in partial do not make much sense, so it plausible that certain codecs are implied to be full range, even if its not said explicitly.

I think I would keep this issue for the yuv/i444 problems, since it seems the most data and information is regarding that. Then I'd move all mentions/issues/information regarding RGB over to a new issue.

I saw your PR, and while it may fix both scenarios, I still think you'll have more luck getting it merged when you have clear and concise information, and since it is two seperate problems, I think its better to split them, so that seperate information, repro steps etc can easily be viewed.

I'll test your PR later today, thank you 👍

[miabaka]

I think I would keep this issue for the yuv/i444 problems, since it seems the most data and information is regarding that. Then I'd move all mentions/issues/information regarding RGB over to a new issue.

Thanks! I separated RGB issue as #8209

[TheDude53]

Also having this issue. Using the NV12 format with sRGB space and full range still produces a partial range output (16-235). Interestingly, this only seems to occur (for me at least) with libx264; other encoders such as libvpx-vp9 work fine.

[Kobi-Blade]

This was reported months ago and brushed off by OBS team, glad you stepped up to fix this issue.

[JonasCz]

I can also confirm this (I think), to reproduce:

1. Drag the following PNG file onto OBS to add it as an image source:

2. Advanced -> Colour Format: I444, Colour Space: 709, Range: Full

3. Output: Advanced, Recording: custom FFMPEG, Container: matroska, Encoder: ffv1.

4. All other settings per OP (defaults)

5. Record video.

6. Looking at resulting video in a player / editor / Histogram tool, it's not full range:

For example ffplay "test.mkv" -vf "histogram", (with correct full range video it should extend all the way to the left and right):

As a result in other tools and players (e.g. VirtualDub2, or MPC-HC as per OP), it's washed out.

VLC seems to be OK and not washed out, perhaps because of this VLC bug: Full/Limited Video Levels incorrectly displayed.

OBS 28.1.2 also has this problem for me, unlike OP.

In the real world: I have a capture card which can output full range, OBS set to full range everywhere seems to scale to 16-235 and flag the file as full range anyway.

Edit: with https://github.com/obsproject/obs-studio/pull/8374 it's fixed, histogram: