[ffmpeg][vaapi][decode][mjpeg 422H]422 mjpeg decode has garbage

[fulinjie]

[Cmdline] ffmpeg -hwaccel vaapi -init_hw_device vaapi=hw:/dev/dri/renderD128 -filter_hw_device hw -v debug -i /root/media/testing/assets/jpeg/canoneos450d_img_3719.jpg -f rawvideo -vsync passthrough -vframes 1 -y test.yuv

[Result] Reslult has garbage.

[Analysis] It could be root caused the decode result before VP process is correct in yuy2 format.

Got result through debug tool: surfdump_loc[preALL]_lyr[0]_f[000]_w[4272]_h[2848]_p[8576].yuy2 -----> correct surfdump_loc[postALL]_lyr[0]_f[000]_w[4272]_h[2848]_p[4352].422h -----> incorrect (in UV)

[XinfengZhang]

@fulinjie , is it a common issue or issue with the dedicated jpeg files? and how to get the file?

[fulinjie]

Clips shared internally through mail.

[uartie]

@fulinjie I think this could be a ffmpeg bug. It works fine with gstreamer-vaapi:

gst-launch-1.0 -vf filesrc location=canoneos450d_img_3719.jpg \
 ! jpegparse ! vaapijpegdec \
 ! checksumsink2 dump-output=true dump-location=testgst.yuv

vaapijpegdec will output as YUY2 by default, but it can be converted to 422H output correctly too (i.e. with vaapijpegdec ! videoconvert ! video/x-raw,format=Y42B)

Can you please double-check?

[feiwan1]

@uartie , gstreamer vaapijpegdec will decode the jpeg to YUY2 format, then use videoconvert do a software conversation to Y42B.

While this ffmpeg case, surface created without specific format, which in media-driver it will choose YUY2 as default. When get 422H(Y42B) image from the YUY2 surface, there will be a VP process which convert YUY2 surface into 422H surface inside vaGetImage. But the data from 422H Image seems lost some U/V which height is any half of Y. Take 4x2 resolution for example: The expected result should be: YYYY YYYY UU UU VV VV

But actually we get: YYYY YYYY UU VV

So this should be still a issue from media-driver, @XinfengZhang Could you help to check this? Thanks.

[XinfengZhang]

@Jister could you help to check it

[Jister]

@feiwan1 For this case, if ffmpeg want to decode to get 422H as output, why not directly create a 422H surface during decoding, rather than call VP to do a conversion in GetImage? Currently VP can support 422H/422V as input, but cannot support them as output.

[feiwan1]

Thanks @Jister for your clarification, if media-driver doesn't support YUY2->422H, we can close this issue. We already submitted the patch to create 422H surface patch to ffmpeg which hasn't been accepted. But it already land in our cartwheel repo: https://github.com/intel-media-ci/ffmpeg-cartwheel/blob/master/patchset/0009-avcodec-vaapi-set-more-flags-for-VASurfaceAttrib.patch

Hi @fulinjie, would you help to close this issue? Thanks.

CC @FocusLuo.

[fulinjie]

closed, thanks.