Decoding full-range yuv444p to RGB differs from ffmpeg

Overview

When decoding a full-range yuv444p video stream and converting to RGB like so

import av
from PIL import Image
with open(video_path, 'rb') as file:
    with av.open(file) as container:
        for pack in container.demux(0):
            for frame in pack.decode():
                rgb = frame.to_ndarray(format='rgb24')
                Image.fromarray(rgb, mode='RGB').show()

I found that the resulting images are different than what I get when extracting images with ffmpeg:

ffmpeg -I video.mov video%06d.png

Expected behavior

The expected behaviour is that the resulting RGB is very close to what is generated by running the above ffmpeg command. For other formats (e.g. limited range yuv420p) this is the case.

Actual behavior

The actual behaviour is that the images differ significantly. It seems like the RGB created with PyAV is scaled incorrectly, using more range than the reference:

Figure_1

import matplotlib.pyplot as plt
plt.figure()
plt.scatter(rgb_ffmpeg[::10, ::10, :].ravel(), rgb_pyav[::10, ::10, :].ravel(), s=1, marker='.')
plt.xlabel('RGB from ffmpeg')
plt.ylabel('RGB from pyav')

Investigation

I found that specifying the src_color_range or dst_color_range fixes the scaling:

Figure_2

from av.video.reformatter import ColorRange
rgb = frame.to_ndarray(format='rgb24', src_color_range=ColorRange.MPEG)   # works
rgb = frame.to_ndarray(format='rgb24', dst_color_range=ColorRange.MPEG)   # works, too

Reproduction

To create a video with yuv444p full-range I used the following command

fmpeg -i yuv420p.mov -vf scale=in_range=limited:out_range=full -color_range 2 -pix_fmt yuv444p -c:v hevc -tag:v hvc1 yuv444p_full_range.mov

This is the video that I tested with:

https://github.com/PyAV-Org/PyAV/assets/12171321/3a934bab-5c78-4286-82fb-9b8a94edf54b

Versions

OS: MacOS 14.4
Pyton 3.11.5 (macOS-14.4-arm64-arm-64bit)

PyAV runtime:

PyAV v12.0.0
library configuration: --disable-static --enable-shared --libdir=/tmp/vendor/lib --prefix=/tmp/vendor --arch=arm64 --enable-cross-compile --disable-alsa --disable-doc --disable-libtheora --disable-mediafoundation --disable-videotoolbox --enable-fontconfig --enable-gmp --disable-gnutls --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libspeex --enable-libtwolame --enable-libvorbis --enable-libvpx --disable-libxcb --enable-libxml2 --enable-lzma --enable-zlib --enable-version3 --enable-libx264 --disable-libopenh264 --enable-libx265 --enable-libxvid --enable-gpl
library license: GPL version 3 or later
libavcodec     60. 31.102
libavdevice    60.  3.100
libavfilter     9. 12.100
libavformat    60. 16.1900
libavutil      58. 29.100
libswresample   4. 12.100
libswscale      7.  5.100

Research

I have done the following:

[x] Checked the PyAV documentation
[x] Searched on Google
[x] Searched on Stack Overflow
[x] Looked through old GitHub issues

Additional context

Related Issue#1378

PyAV-Org / PyAV