[Bug]: <HEVC decoding fails after encoding with windows media foundation>

[ksmuthucv]

Describe the bug We have been using wcap (wmf, https://github.com/mmozeiko/wcap) to record the desktop using h264/hevc codecs. It was working fine till last week, prior to driver crash. Post that crash, the radeon rx vega m gh gpu was not recognized under device manager. To recover, we have installed the latest driver (selected factory reset while the installation) as mentioned below. From then the decoding of recorded file based on hevc codec has failed i.e. played with noise (PFA of screen mockup attached).

To Reproduce Steps to reproduce the behavior:

1. Initialize the IMFSinkWriter with hw capability.
2. Associate the output type with hevc codec, frame size, frame rate and bit rate, etc.
3. Setup the input type with frame size, frame rate, etc.
4. Configure the codec api with rate control, bit rate, low latency, etc.
5. Create the IMFSample from the texture capture and write it to the file via imf-sink-writer instance.

Reference https://github.com/mmozeiko/wcap/blob/main/wcap_encoder.c, line# 285 onwards.

Setup:

• OS: Windows 11 23H2
• Driver Version: 31.0.21001.46001
• GPU: Radeon RX Vega M GH Graphics
• Encoder (invoked via IMFSinkWriter)

Debug Log: 2023-12-31 11:13:55.612 1688 [AMFAsyncMFTBase] Error: ..........\runtime\src\mft\mft-framework\AsyncMFTBase.cpp(852):AsyncMFTBase(4)::GetOutputCurrentType(StreamID=0): MF_E_TRANSFORM_TYPE_NOT_SET 2023-12-31 11:13:55.612 1688 [AMFAsyncEncoderMF] Error: ..........\runtime\src\mft\hevcencoder\HevcAsyncEncoderMFT.cpp(217):OnCheckInputType(4): Output type must be set first. 2023-12-31 11:13:55.612 1688 [AMFAsyncMFTBase] Error: ..........\runtime\src\mft\mft-framework\AsyncMFTBase.cpp(709):AsyncMFTBase(4)::OnCheckInputType(StreamID=0) - failed with HRESULT(0xC00D6D60) 2023-12-31 11:13:55.612 1688 [AMFAsyncEncoderMF] Error: ..........\runtime\src\mft\hevcencoder\HevcAsyncEncoderMFT.cpp(539):OnSetOutputType(4): failed to set MF_MT_INTERLACE_MODE set to 2, HRESULT(0x80004001) 2023-12-31 11:13:55.612 1688 [AMFCodecAPIImpl] Error: ..........\runtime\src\mft\encoder-common\CodecAPIImpl.cpp(95):ICODECAPI(4)::SetValue(AVEncMPVDefaultBPictureCount, VT_UI4(2)) failed hr=0x80070057 2023-12-31 11:13:55.613 1688 [AMFCodecAPIImpl] Error: ..........\runtime\src\mft\encoder-common\CodecAPIImpl.cpp(95):ICODECAPI(4)::SetValue({143A0FF6-A131-43DA-B81E-98FBB8EC378E}, VT_BOOL(VARIANT_TRUE)) failed hr=0x80004001 The thread 0x1f24 has exited with code 0 (0x0). The thread 0x2d08 has exited with code 0 (0x0). The thread 0x1b84 has exited with code 0 (0x0). The thread '.NET ThreadPool Gate' (0x4b98) has exited with code 0 (0x0). The thread '.NET ThreadPool Gate' (0x3bac) has exited with code 0 (0x0). The thread '.NET ThreadPool Gate' (0x21f8) has exited with code 0 (0x0). The thread 0x1884 has exited with code 0 (0x0). The thread 0x4db8 has exited with code 0 (0x0). The thread '.NET ThreadPool Gate' (0x2314) has exited with code 0 (0x0). 2023-12-31 11:14:03.511 3B64 [DriverParameterUVEHEVCAdapter] Error: ..........\runtime\src\components\EncoderUVE\DriverParameterUVEHEVCAdapter.cpp(216):AMF_ERROR 1 : AMF_FAIL: Failed to set UVEHEVC param(17). err 0x80000009 2023-12-31 11:14:03.511 3B64 [DriverParameterUVEHEVCAdapter] Error: ..........\runtime\src\components\EncoderUVE\DriverParameterUVEHEVCAdapter.cpp(216):AMF_ERROR 1 : AMF_FAIL: Failed to set UVEHEVC param(39). err 0x80000009 The thread 0x33c0 has exited with code 0 (0x0). The thread 0x3928 has exited with code 0 (0x0).

Expected behavior The recorded file should be decoded without any noise/corruption

Screenshots

Recorded File: https://github.com/GPUOpen-LibrariesAndSDKs/AMF/assets/16358707/1737391b-fa23-43fd-ab6a-3b3085a93f38

[MikhailAMD]

The clip shows synchronization issue. The driver you installed is not the latest for Vega. But I cannot find RX Vega M GH or similar on www.amd.com. This is seems a mobile GPU. Where did you get the driver? Is it embedded GPU like Ryzen or Athlon? If so, driver should be 31.0.21905.1001 from November 2023. Can you find your APU here: https://www.amd.com/en/support? You can use factory reset in the installed or a special utility to remove the older driver: https://www.amd.com/en/support/kb/faq/gpu-601

[ksmuthucv]

Hello @MikhailAMD,

Wishing you a very happy new year! Thanks for helping us over the holidays, highly appreciated. Please find the machine details below.

Machine: Intel® NUC Kit NUC8i7HVK Driver: https://www.intel.com/content/www/us/en/download/19269/radeon-rx-vega-m-graphics-driver-for-windows-10-64-bit-for-nuc8i7hnk-nuc8i7hvk.html

We couldn't find the match for radeon-rx-vega-m-graphics under https://www.amd.com/en/support.

Also, If possible, can you please advise if the parameter issues [UVEHEVC param(17) and UVEHEVC param(39)] reported can be configured from code, so that synchronization issue can be fixed?

Thanks, Muthu

[MikhailAMD]

Ah, this is very special GPU included into Intel's CPU HW package. Unfortunately, for this part drivers are provided by Intel only. I doubt that regular Vega driver will recognize it. 17 corresponds to AMF parameter AMF_VIDEO_ENCODER_HEVC_GOP_SIZE 39 corresponds to AMF parameter AMF_VIDEO_ENCODER_HEVC_RATE_CONTROL_METHOD 0x80000009 internally means "access denied". The app probably sets these parameters after initialization of AMF encoder (AMFComponent::Init call) and these parameters are "static" meaning they cannot be changed after the Init call (for this GPU). For later GPUs GOP_SIZE can be changed dynamically. Init is called in MFT_MESSAGE_NOTIFY_BEGIN_STREAMING message handler I doubt that these errors cause synchronization issue.

Regarding the synchronization issue: I looked into the code. I would try three things, one by one in this order:

1. Insert ID3D11DeviceContext::Flush after ID3D11DeviceContext_Dispatch - this should be always done
2. Currently encoder input is 8 NV12 textures used in round-robin mode. There is no checking if texture is actually released by the encoder. You can attach a private GUID property with texture index to IMFsample. Reading this property in Encoder__VideoInvoke you will know that the encoder released the input texture and it can be reused. or you can use IMFsample pointer to find it in VideoSample array.
3. Add CPU level sync after color conversion using ID3D11Query created as D3D11_QUERY_EVENT. Submit query with ID3D11DeviceContext::End and loop polling ID3D11DeviceContext::GetData and wait for completion of the query .

[ksmuthucv]

Hello @MikhailAMD, Thanks for your feedback. Invoking ID3D11DeviceContext::Flush after ID3D11DeviceContext_Dispatch didn't the synchronization issue. Will try 2 & 3 recommendations or will check with the author (i.e. i'm relatively new to this section of programming). Mean while, just curious to know, how does the same utility would work fine with h264 codec? Do you think, rolling back to earlier version of the driver (if i could fine one) will resolve this issue?

-Muthu

[ksmuthucv]

Hello @MikhailAMD, When the same utility was tested against igpu [Intel(R) HD Graphics 630), the recorded file played without synchronization issue.

https://github.com/GPUOpen-LibrariesAndSDKs/AMF/assets/16358707/d5dfde10-585f-4a5f-96a8-0f0bd9522e4d

Thanks, Muthu

[MikhailAMD]

Except Flush, I asked for the changes not as real fixes but as way to sort out the problem. Though, timing on different platforms may be different. Also related to #2: Intel may behave differently.