search

intel / cartwheel-ffmpeg

Intel developer staging area for unmerged upstream patch contributions to FFmpeg
GNU Lesser General Public License v2.1
84 stars 31 forks source link

MJPEG_QSV and BGRA #337

Open VD-NVencC opened 6 days ago

VD-NVencC commented 6 days ago

environment - intel i5-13500, Win10 21H2, graphics driver 101.5593, full-ffmpeg-7.0.1 from gyan.dev, and IrfanView v4.67 x64

I'm interested in capturing 8bit RGB losslessly, is that within the scope of this igpus hw encoder?

In order to verify an encoder works for this purpose I have an 4096x4096 16,777,216 unique color image that encompasses the entire colorspace and have it sent to be encoded and then check the decoded output. My preference is having ffmpeg convert it to the bgr24 pixel format then runs it through framemd5, but IrfanView can also give us a unique color count by Image, Information... dialog box when the hash doesn't match.

ff701 -hide_banner -i 16777216.png -vf format=bgr24 -f framemd5 - will output stream, dts, pts, duration, size, hash 0, 0, 0, 1, 50331648, 69fc85794c528f9192c1143cd535c4ca

a more elaborate example of transferring the colors to the 10b YCbCr colorspace, pipe it to another instance, and converting it back again ff701 -hide_banner -i 16777216.png -vf zscale=matrix=bt709:primariesin=bt709:primaries=bt709:transferin=iec61966-2-1:transfer=iec61966-2-1:in_range=full:out_range=limited,format=yuv444p10le -f rawvideo - | ff701 -hide_banner -f rawvideo -pix_fmt yuv444p10le -s 4096x4096 -i - -vf scale=in_color_matrix=bt709,format=bgr24 -f framemd5 - 0, 0, 0, 1, 50331648, 69fc85794c528f9192c1143cd535c4ca the hash matches

now let's do a lossy step on purpose ff701 -hide_banner -i 16777216.png -vf zscale=matrix=bt709:primariesin=bt709:primaries=bt709:transferin=iec61966-2-1:transfer=iec61966-2-1:in_range=full:out_range=full,format=yuv444p,format=bgr24 -f framemd5 - 0, 0, 0, 1, 50331648, ef5971224818b5668cd7c6205a58c40a it's a hash mismatch when converted back from 8bit full YCbCr, let's generate an image to compare against ff701 -hide_banner -i 16777216.png -vf zscale=matrix=bt709:primariesin=bt709:primaries=bt709:transferin=iec61966-2-1:transfer=iec61966-2-1:in_range=full:out_range=full,format=yuv444p,format=bgr24 -c:v bmp yuv444.bmp IrfanView reports 4,146,765 unique colors

let's try using the mjpeg_qsv encoder now ff701 -hide_banner -i 16777216.png -vf format=bgra -c:v mjpeg_qsv mjpeg_qsv.mov ff701 -hide_banner -i mjpeg_qsv.mov -vf format=bgr24 -f framemd5 - 0, 0, 0, 1, 50331648, acd048d28903161c2e1c11a1e7b4508f it's a hash mismatch, let's review it ff701 -hide_banner -i mjpeg_qsv.mov -vf format=bgr24 -c:v bmp mjpeg.bmp IrfranView reports 11,490,560 unique colors, so it doesn't appear to convert to the YCbCr colorspace

I've placed both 16777216.png and mjpeg.bmp into an image editor [Krita], with them being filtered by difference, levels, and threshold, and then the resulting image was sent to a Pixel Color Counter: 9,368,064 were the correct color, but 7,409,152 were not.

Perhaps this is just a decoding issue with ffmpeg, I'd like to hear a workaround if you've got one since it doesn't appear to be intended behavior to me. 16777216 mjpeg_krita

VD-NVencC commented 4 days ago

I'd also like to note ff701 -y -hide_banner -i 16777216.png -vf format=bgra -colorspace 1 -c:v mjpeg_qsv mjpeg_qsv.mkv it reported the output stream... Stream #0:0: Video: mjpeg (MJPG / 0x47504A4D), bgra(pc, bt709/unknown/unknown, progressive), 4096x4096 .. but when parsed ff701 -i mjpeg_qsv.mkv -vf showinfo -f null - color_range:pc color_space:bt470bg color_primaries:unknown color_trc:unknown however when opened in a tool like MKVToolNIX GUI v85 Color information > color matrix coefficients has a value of 1 as expected, a value of 5 would be bt470bg

odd, let's change the containers colorspace another way.. ff701 -i mjpeg_qsv.mkv -colorspace 15 -c:v copy colorspace.mkv outputstream says Stream #0:0: Video: mjpeg (Baseline) (MJPG / 0x47504A4D), gbrp(pc, unknown, progressive), 4096x4096 .. however another msg came up during the process saying... Codec AVOption colorspace (color space) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream. I'm not an expert, but I suspect it's not using the containers provided colorspace, during other testing color_trc and primaries changed just fine but colorspace seemed hardcoded.

I'm unaware of any -bsf options for mjpeg to denote what colorspace it uses, but I've had to use both -color_trc # -bsf *_metadata=transfer_characteristics=# options to change the transfer_characteristics of other containers like mp4 in a similar situation to get a different decoded output.