MediaTranscoding sample "Unknown failure"

[Zofware]

The MediaTranscoding sample shows a red "unknown failure" error when I try to transcode to UHD 2160p60 on an Intel machine without Quick Sync Video and with an Nvidia GPU. UHD 2160p60 transcoding succeeds on Intel hardware with Quick Sync Video.

Repro steps:

1. Build the MediaTranscoding sample (I built the C# version).
2. Run the sample on a machine with an Intel CPU without Quick Sync Video. I used a Core i7-5960X with an Nvidia GeForce GTX 980 Ti.
3. Select scenario #2 (Transcode with custom settings).
4. Select any source video file.
5. Enter Width=3840, Height=2160, Frame Rate=60
6. Select an output file.
7. Click the Transcode button.

You'll get a red "unknown failure" error.

If you change the target frame rate from 60 to 30, this scenario will succeed.

If you run this scenario on a Surface Pro it will succeed at both 60 and 30 frames per second. I used a Surface Pro LTE from mid 2018 that has an Intel Core i5-7300U with Quick Sync Video.

Perhaps this is some kind of platform bug. Or perhaps it's some kind of Intel and/or Nvidia driver bug. For what it's worth, I installed the latest driver from Nvidia (445.87). I am running Windows 10.0.18363.815.

[Zofware]

I captured a trace with WPR and it showed one thing that might be a clue:

EventName: Microsoft-Windows-Runtime-Media/WinRTTranscode_PrepareTranscode_Task/Stop
Payload: ErrorCode="-1,072,861,856" FormattedMessage="PrepareTranscode operation 0x29ed6283fe0{Object} ended. -1,072,861,856{ErrorCode} "

That error code happens to be MF_E_TRANSFORM_TYPE_NOT_SET. The Media Foundation doc page for the H.264 encoder says this:

The output type must be set before the input type. Until the output type is set, the encoder's IMFTransform::SetInputType method returns MF_E_TRANSFORM_TYPE_NOT_SET.

So maybe there's a bug in how the OS sets up the graph when transcoding to 59.94 fps.

[oldnewthing]

I have filed a ticket against the feature team to investigate this.

[tahnik]

@oldnewthing Hey, is there an update on this? The media composition API is incredibly useful. But this bug is making it impossible to create the video editing app that I am trying to work on. I really don't want to switch QT as I simply love the way UWP and MediaComposition API works.

[tahnik]

Just to add to this, I am using a mp4 video exported from GeForce Experience. When I convert to 1080p 30fps mp4 using the media composition API, I get this weird washed out colours. You can see an example here:

I should mentioned that none of these issues exists (including the 60fps issue mentioned by @Zofware) if I use VC-1/WMV as the output.

[oldnewthing]

I sent a reminder to the feature team.

[Zofware]

@tahnik Just a few days ago the team communicated a workaround to me for the unknown failure when transcoding to 2160p60 in the scenario I described in the original post. Fortunately the workaround is fairly simple.

The issue is that some video encoders (e.g. Nvidia) have to be configured with a slightly higher encoding "level" to be able to process high frame rate UHD 2160p video. This Wikipedia article explains the various encoding levels required for H.264 video. Windows doesn't automatically set the higher level for you so you have to set it yourself.

When you set up the video portion of the media encoding profile for high frame rate UHD 2160p H.264, you need to set MF_MT_VIDEO_LEVEL to eAVEncH264VLevel.eAVEncH264VLevel5_2. In scenario 2 of the C# version of the the sample, I believe you can do something like this on or around line 158 of Scenario2_Custom.xaml.cs:

// Workaround: set MF_MT_VIDEO_LEVEL to 52 (XAVC level 5.2 which supports 4k60)
_Profile.Video.Properties[MF_MT_VIDEO_LEVEL] = (UInt32)eAVEncH264VLevel.eAVEncH264VLevel5_2;

where MF_MT_VIDEO_LEVEL and eAVEncH264VLevel.eAVEncH264VLevel5_2 are defined to match the definitions from the Windows SDK. Like this:

// %ProgramFiles(x86)%\Windows Kits\10\Include\10.0.19041.0\um\mfapi.h
static readonly Guid MF_MT_VIDEO_LEVEL = new Guid("{96F66574-11C5-4015-8666-BFF516436DA7}");

// %ProgramFiles(x86)%\Windows Kits\10\Include\10.0.19041.0\um\codecapi.h
enum eAVEncH264VLevel : UInt32
{
    eAVEncH264VLevel1         = 10,
    eAVEncH264VLevel1_b       = 11,    
    eAVEncH264VLevel1_1       = 11,
    eAVEncH264VLevel1_2       = 12,
    eAVEncH264VLevel1_3       = 13,
    eAVEncH264VLevel2         = 20,
    eAVEncH264VLevel2_1       = 21,
    eAVEncH264VLevel2_2       = 22,
    eAVEncH264VLevel3         = 30,
    eAVEncH264VLevel3_1       = 31,
    eAVEncH264VLevel3_2       = 32,
    eAVEncH264VLevel4         = 40,
    eAVEncH264VLevel4_1       = 41,
    eAVEncH264VLevel4_2       = 42,
    eAVEncH264VLevel5         = 50,
    eAVEncH264VLevel5_1       = 51,
    eAVEncH264VLevel5_2       = 52
};

[* Full disclosure: Since making the original post I started working at Microsoft but I am not commenting here in that capacity. ]

[tahnik]

@Zofware You sir, are a wonderful person for taking your time to reply. And yes, it worked! So I could not find the MF_MT_VIDEO_LEVEL and eAVEncH264VLevel.eAVEncH264VLevel5_2 in the UWP project that I am working on. So I directly used the GUID you wrote in your comment. Here is the line of code that I added if anyone is looking at this issue in future:

profile.Video.Properties[Guid.Parse("96f66574-11c5-4015-8666-bff516436da7")] = (UInt32)52;

However, the colours in the converted video is still washed out. Do you have any idea why that might happen?

Also, congratulations on getting a job at MS!

[tahnik]

Never mind, I should probably create a different issue as that is not directly not related to this issue. I think it has some thing to do with the colour profile Nvidia is using when recording the screen. Thank you for your help.

Tracking the washed out color issue here: #1257

[Zofware]

Thanks @tahnik!

That's right, the GUID and the encoding levels aren't defined by default in the UWP C# environment. I pulled those definitions out of two of the C header files in the Windows SDK. The specific files are noted in the comments above.

Be aware there are encoding levels defined for HEVC/H.265 video that I didn't include above but they are in the SDK header files also. This is speculation, but I wonder if certain HEVC encoding settings require a level to be set? It's something to be aware of if you run into problems with HEVC in the future.

Cheers, Adam

P.S. I don't know the cause of the washed out colors you're seeing but your theory sounds reasonable to me. Good luck!

[Zofware]

I will work with @oldnewthing to add this workaround to the MediaTranscoding sample when I have some time.

[oldnewthing]

Fixed in v8.0.12. Thanks, @Zofware

[Zofware]

Thanks for seeing this through @oldnewthing!