[Question]: On CaptureVideo sample, who decode MJPEG to NV12

[toruishihara]

I run CaptureVideo sample with VisualStudio2019 on Windows 10. Camera is Logitech C270 USB 2.0 CPU is AMD Ryzen 3 3250U with Radeon Graphics. I unselected all "Options" and click "Run", camera frame is displayed.

In MFCaptureImpl.cpp, SelectSource(), 1280x720@30fps NV12 is selected. USB 2.0 isochronous mode does not allow to send 1280x720x30x2 byte in 1 sec.

At same time, I captured USB traffic by WireShark + USBPcap. C270 sends compressed 30Kbyte every 15ms. I assume C270 send as MJPEG.

On lower layer someone decodes MJPEG to NV12 and passing to CaptureVideo app.

Do you know who decode MJPEG to NV12? Is this decoding done by AMD hardware or software?

Thanks

[MikhailAMD]

IMFSourceReader is a mini-pipeline built from elements provided by the sample and from added by MF. in your setup it probably includes MJPEG Software Decoder MFT provided by Microsoft.

[toruishihara]

I am looking for AMD Hardware MJPEG decoder. Software decoder is too slow for handling 4K x 30 fps.

[MikhailAMD]

AMF SDK supports MJPEG HW decoding if HW supports it. Not all GPUs have it.

[MikhailAMD]

Correction: AMD also installs MJPEG HW decoder MFT and it will be used if HW is capable.

[toruishihara]

I cannot find sample code to run MJPEG HW decoder MFT. Do you know some sample code?

[MikhailAMD]

AMF doesn't provide MFT samples. The use of the decoder is no different from any other HW decoder MFT, given that video format subtype is MFVideoFormat_MJPG passed to MFTEnumEx.

[toruishihara]

Do you know AMF provides MJPEG or JPEG HW decoder sample other than MFT?

[MikhailAMD]

Find MP4 MJPEG clip and play it with PlaybackHW sample or transcode with TranscodeHW sample.

[toruishihara]

.mp4 file cannot have Motion JPEG in contents. Do you have sample file URL for TranscodeHW?

[MikhailAMD]

Try any other file format : MKV, MOV, whatever is supported by ffmpeg. TranscodeHW is a sample from this SDK. https://github.com/GPUOpen-LibrariesAndSDKs/AMF/tree/master/amf/public/samples/CPPSamples/TranscodeHW

[toruishihara]

I run sample MFT handling code with "AMD MFT MJPEG Decoder", output is MFVideoFormat_NV12. It shows mostly correct frame from Logitech C270. Decoding is twice faster than software decoder. But top 10% of frame is green. On this portion UV plane is all zero. How should I report a bug of "AMD MFT MJPEG Decoder"?

[MikhailAMD]

Top 10% of the frame is green can be caused by decoder failed to provide crop parameter or by next transform to use the crop parameter, or parameter is wrong. Crop is used to remove with vertical alignment. MF_MT_MINIMUM_DISPLAY_APERTURE. To deal with it, you can:

• Check the latest public driver
  • Check how output sample is consumed - if crop is used.
  • Use Windows "Camera" application to record from this camera and check if HW decoder is used in Task Manager or GPUVIEW log. If it works, it is likely sample issue.
  • Provide your sample app so developer could repro it. I think we have C270 or similar.
  • If all points to Decoder problem I will notify developer.

[toruishihara]

I put sample code on this Github repository. https://github.com/toruishihara/amf_mjpeg_decoder

AMF MJPEG decoded N12 data has all zeros area 0x12c00 - 0x14000 on 320x240. That all zeros area cause green gap.

By calling memcpy(), I can fix this problem. But memcpy() makes slower than software decoder.

[toruishihara]

AMF MJPEG decoder change output size to 320x256 on 320x240 input case.