Windows Media Foundation Hook

[grill2010]

Hi, I found your project and it is really helpful so far. I found out that the application which I'm trying to intercept is using some Windows Media Foundation functions like the IDirectXVideoProcessor::VideoProcessBlt method. The idea which I have is basically the same as you did in your Direct3DHook project. I would like to create a managed VideoProcessor object, get the virtual method table addresses and hook into some calls. Unfortunately I'm not able to create the VideoProcessor object, I've even tried it on several PCs. The error message which I receive is this:

{"HRESULT: [0x80070057], Module: [General], ApiCode: [E_INVALIDARG/Invalid Arguments], Message: Falscher Parameter.\r\n"}

I could not find a single example of SharpDX which uses the VideoProcessor interface so I really hope you can help me out. Below you can find some code of my example project. I use SharpDX version v4.0.1.

using (Direct3D d3d = new Direct3D())
{
    using (var renderForm = new System.Windows.Forms.Form())
    {
        SharpDX.Direct3D9.Device device;
        //using (device = new Device(d3d, 0, SharpDX.Direct3D9.DeviceType.NullReference, IntPtr.Zero, CreateFlags.HardwareVertexProcessing, new PresentParameters() { BackBufferWidth = 1, BackBufferHeight = 1, DeviceWindowHandle = renderForm.Handle, PresentFlags = PresentFlags.Video }))
        using (device = new Device(d3d, 0, SharpDX.Direct3D9.DeviceType.Hardware, renderForm.Handle, CreateFlags.HardwareVertexProcessing, new SharpDX.Direct3D9.PresentParameters(1, 1) /*new SharpDX.Direct3D9.PresentParameters(this.ClientSize.Width, this.ClientSize.Height, Format.A8R8G8B8, 1) { BackBufferWidth = 1, BackBufferHeight = 1, DeviceWindowHandle = renderForm.Handle, PresentFlags = PresentFlags.None, FullScreenRefreshRateInHz = 60, SwapEffect = SwapEffect.Copy, AutoDepthStencilFormat = Format.A8R8G8B8, EnableAutoDepthStencil= false }*/))
        {

            using (VideoProcessorService videoProcessorService = new VideoProcessorService(device))
            {

                int count = 5;
                Guid[] guid = new Guid[10];
                // https://msdn.microsoft.com/de-de/library/windows/desktop/ms695370(v=vs.85).aspx
                VideoDesc videoDesc = new VideoDesc()
                {
                    Format = SharpDX.Direct3D9.Format.A8R8G8B8,
                    InputSampleFreq = new Frequency()
                    {
                        Denominator = 1,
                        Numerator = 60
                    },
                    OutputFrameFreq = new Frequency()
                    {
                        Denominator = 1,
                        Numerator = 60
                    },
                    SampleFormat = new ExtendedFormat()
                    {
                        NominalRange = (int)NominalRange.Range16_235,
                        SampleFormat = (int)SampleFormat.ProgressiveFrame,
                        VideoChromaSubsampling = (int)VideoChromaSubSampling.SamplingVerticallyAlignedChromaPlanes,
                        VideoLighting = (int)VideoLighting.Dim,
                        VideoPrimaries = (int)VideoPrimaries.Bt709,
                        VideoTransferFunction = (int)VideoTransferFunction.Func709,
                        VideoTransferMatrix = (int)VideoTransferMatrix.Bt709,
                        Value = 0
                    },
                    SampleWidth = 640,
                    SampleHeight = 480,
                    UABProtectionLevel = 0
                };

                // On my PC count is 5 but only 1 guid is present in the array???
                videoProcessorService.GetVideoProcessorDeviceGuids(ref videoDesc, out count, guid);

                if (count > 0)
                {
                    try
                    {
                        SharpDX.MediaFoundation.DirectX.VideoProcessor videoProcessor;
                        // always fails
                        videoProcessorService.CreateVideoProcessor(guid[0], ref videoDesc, SharpDX.Direct3D9.Format.A8R8G8B8, 0, out videoProcessor);
                    }
                    catch (Exception ex)
                    {
                    }
                }
            }
        }
    }
}

I seems in regard to the error message that some parameters are invalid or not correct but as I'm a beginner in DirectX I don't know what I should do. Sorry if it is maybe obvious but I'm really lost now. Thank you for your help.

EDIT: I've also posted an issue in the SharpDX repo which you can find here:

https://github.com/sharpdx/SharpDX/issues/955

It seems that you have to manually handle the marshalling of arguments. I hope this will solve the problem. Do you have any experience with this issue?

[justinstenning]

@grill2010 might be worth seeing if the COMClassInfo helper is of any use here: http://easyhook.github.io/api/html/T_EasyHook_COMClassInfo.htm

[grill2010]

Thank you, I will check if it helps.

[grill2010]

In general, I wonder if there is any "better" way to capture the screen for example as a video stream and not as single image frames. Although it works very well in combination with your SharedMemory project but as I also need to stream the captured images over the network it would be better to stream it in a more compressed way like as a video stream. I also tried FFmpeg to convert the bitmap sequence in a video stream but it looks like it is to slow as I need to receive the content of the hooked application in real time (but could also be my fault due miss configuration of FFmpeg). Do you know any other possibilities or is this already the recommended way to capture the screen? Thank you anyway for your help during the development of my app.

[justinstenning]

@grill2010 yes, you can look at using Desktop Duplication API to grab the frame and feed into MediaFoundation. Either way you need to first grab the individual frames and add them to your stream.