search

wang-bin / mdk-sdk

multimedia development kit. download:
https://sourceforge.net/projects/mdk-sdk/files/
291 stars 33 forks source link

仿照WindowsStore Example在WinUI3上初始化SwapChain失败 #183

Closed axojhf closed 8 months ago

axojhf commented 9 months ago

Describe the bug 在仿照Example里的UWP C++代码 https://github.com/wang-bin/mdk-examples/blob/master/WindowsStore/XamlCx/OpenGLESPage.xaml.cpp#L49 初始化C# WinUI3的SwapChainPanel时，APP会崩溃，VS报告异常：

引发了异常: 读取访问权限冲突。
this->d._Mypair._Myval2->swapchain.**ptr_** 是 nullptr。

image

To Reproduce WinUI3 C#代码下（C++下没有试过） ① 使用setRenderAPI设置API和SwapChainPanel的指针 ② 使用updateNativeSurface再设置SwapChainPanel的指针 ③ 设置播放状态为Playing

Expected behavior 无错误发生

Environment:

Log 日志关键部分：

### Level: Debug Msg: 0000026F39D7A770->PlatformSurface::Event::NativeHandle: 0000000000000000>>>0000026F318A2608

### Level: Debug Msg: auto __cdecl mdk::abi::VideoPresenter::{ctor}::<lambda_3>::operator ()(class UGS::PlatformSurface *) const: 0000026F318A2608=>0000026F39D61450 RenderAPI.type: 4

### Level: Debug Msg: createRenderContext of RenderAPI::Type 4 from surface 0000026F39D7A770 with extra native res 0000000000000000

### Level: Debug Msg: dxgi adapter 0: vendor 8086, device 3ea0, revision 0, Intel(R) UHD Graphics 620

### Level: Debug Msg: d3d11.1 device feature level: 0xc100, requested: 0xc100.

### Level: Debug Msg: hr = factory2->CreateSwapChainForHwnd(dev, hWnd, &sd, nullptr, nullptr, &sc1)  ERROR@110DXGI::RenderContext::createSwapChain: (80070005) �ܾ����ʡ�

### Level: Debug Msg: 
### Level: Debug Msg: factory2->CreateSwapChainForHwnd(dev, hWnd, &sd, nullptr, nullptr, &sc1)  ERROR@114DXGI::RenderContext::createSwapChain: (80070005) �ܾ����ʡ�

### Level: Debug Msg: 
### Level: Debug Msg: ++++++++++onContextCreated(surface 0000026F39D7A770, ctx 0000026F3A697600)+++++++++++

### Level: Debug Msg: set d3d11 device. Feature level: 0XC100, create flags: 0

### Level: Debug Msg: d3d11: 0000026F39B1BB08, dxgi adapter vendor 8086, device 3ea0, revision 0, Intel(R) UHD Graphics 620. UMA

### Level: Debug Msg: set d3d11 device. Feature level: 0XC100, create flags: 0

### Level: Debug Msg: d3d11: 0000026F39B1BB08, dxgi adapter vendor 8086, device 3ea0, revision 0, Intel(R) UHD Graphics 620. UMA

### Level: Debug Msg: PlatformSurface::Event::Resize 0x0
axojhf commented 9 months ago

根据日志上来看，在CreateSwapChainForHwnd这一步的时候就报错80070005（E_ACCESSDENIED ？）

从微软文档链接的例子来看，WinUI3（可能UWP也是？）的swapchain的创建可能需要通过dxgiFactory->CreateSwapChainForComposition()方法才能正常使用 https://github.com/microsoftarchive/msdn-code-gallery-microsoft/blob/master/Official%20Windows%20Platform%20Sample/XAML%20SwapChainPanel%20DirectX%20interop%20sample/%5BC%23%5D%5BC%2B%2B%5D-XAML%20SwapChainPanel%20DirectX%20interop%20sample/C%23%20and%20C%2B%2B/DirectXPanels/DirectXPanelBase.cpp#L223

在StackOverflow也找到了相似的UWP上的问题（同样出现E_ACCESSDENIED）：https://stackoverflow.com/questions/48991029

cnblog也找到一篇WinUI3的博客同样采用了CreateSwapChainForComposition方法来创建SwapChain：https://www.cnblogs.com/xymfblogs/p/17218256.html

大佬可否添加一个接口或者选项来让开发者提供是否在WinUI3或者UWP环境下

wang-bin commented 9 months ago

CreateSwapChainForComposition相关代码目前只在uwp编译了，win32我试了下也能编译，我看看接口要怎么调整，也可能不要调整

wang-bin commented 9 months ago

有个问题还不确定，uwp上ISwapChainPanelNative 设置 swapchain要在ui线程上，winui3不知道有没有这种限制

axojhf commented 9 months ago

有个问题还不确定，uwp上ISwapChainPanelNative 设置 swapchain要在ui线程上，winui3不知道有没有这种限制

这个不晓得，可能和uwp行为是差不多的

wang-bin commented 9 months ago

最新的包win32上也支持了SwapChainPanel，但是目前还有一部分uwp代码还没改好，导致 updateNativeSurface最后一个参数不能SurfaceType::Auto，你设成其他值先，然后你的代码里要监听SwapChainPanel尺寸变化事件，并再次调用updateNativeSurface(panel, newWidth, newHeight, SurfaceType::X11) ， 估计能跑。后面我把uwp相关代码都编译到win32里就不用这么复杂了，和uwp例子一样就行

axojhf commented 9 months ago

最新的包win32上也支持了SwapChainPanel，但是目前还有一部分uwp代码还没改好，导致 updateNativeSurface最后一个参数不能SurfaceType::Auto，你设成其他值先，然后你的代码里要监听SwapChainPanel尺寸变化事件，并再次调用updateNativeSurface(panel, newWidth, newHeight, SurfaceType::X11) ， 估计能跑。后面我把uwp相关代码都编译到win32里就不用这么复杂了，和uwp例子一样就行

SurfaceType好像没有对的上的，日志说unsupported native surface to create d3d render context，Auto还是老样子 SurfaceType其他几项貌似都是Linux上的来着

wang-bin commented 9 months ago

unsupported native surface to create d3d render context

传的指针没有 ISwapChainPanel 接口

axojhf commented 9 months ago

行，明天我再试试

wang-bin commented 9 months ago

最新的包日志里会输出传入指针支持的接口的uuid，可以看看有没有ISwapChainPanel的，可以贴下日志

IInspectable::GetIids[0]: {a uuid}
IInspectable::GetIids[1]: {a uuid}
axojhf commented 9 months ago 
00000311C8CAD390->PlatformSurface::Event::NativeHandle: 0000000000000000>>>00000311C0AECD88

auto __cdecl mdk::abi::VideoPresenter::{ctor}::<lambda_3>::operator ()(class UGS::PlatformSurface *) const: 00000311C0AECD88=>00000311C8A9A710 RenderAPI.type: 4

createRenderContext of RenderAPI::Type 4 from surface 00000311C8CAD390 with extra native res 0000000000000000

dxgi adapter 0: vendor 8086, device 3ea0, revision 0, Intel(R) UHD Graphics 620

d3d11.1 device feature level: 0xc100, requested: 0xc100.

00000311C0AECD88 IInspectable::GetIids count: 20

5IInspectable::GetIids[0]: {63aad0b8-7c24-40ff-85a8-640d944cc325}

5IInspectable::GetIids[1]: {88fd8248-10da-4810-bb4c-010dd27faea9}

5IInspectable::GetIids[2]: {08844f85-aa1b-540d-bef2-b2bb7b257f8c}

5IInspectable::GetIids[3]: {c4496219-9014-58a1-b4ad-c5044913a5bb}

5IInspectable::GetIids[4]: {27a1b418-56f3-525e-b883-cefed905eed3}

5IInspectable::GetIids[5]: {fe08f13d-dc6a-5495-ad44-c2d8d21863b0}

5IInspectable::GetIids[6]: {e59a3db0-91e5-5903-9caf-d1bb9f458bf2}

5IInspectable::GetIids[7]: {ffc6fd98-f38c-5904-9ce4-97a3427cf4ba}

5IInspectable::GetIids[8]: {c3c01020-320c-5cf6-9d24-d396bbfa4d8b}

5IInspectable::GetIids[9]: {8f69b9e9-1f00-5834-9bf1-a9257bed39f0}

5IInspectable::GetIids[10]: {9034f41e-ab7b-59e7-8168-50de6b689dde}

5IInspectable::GetIids[11]: {8f56119d-b96d-58d0-9916-d1c5e390f890}

5IInspectable::GetIids[12]: {2180f1f5-b5d8-4bf6-920a-12006e63efef}

5IInspectable::GetIids[13]: {bc950c8d-1db0-53aa-9dee-34271cd18ce6}

5IInspectable::GetIids[14]: {e7beaee7-160e-50f7-8789-d63463f979fa}

5IInspectable::GetIids[15]: {eb24c20b-9816-4ac7-8cff-36f67a118f4e}

5IInspectable::GetIids[16]: {00000038-0000-0000-c000-000000000046}

5IInspectable::GetIids[17]: {df0b3d60-548f-101b-8e65-08002b2bd119}

5IInspectable::GetIids[18]: {00000000-0000-0000-c000-000000000046}

5IInspectable::GetIids[19]: {af86e2e0-b12d-4c6a-9c5a-d7aa65101e90}

unsupported native surface 00000311C0AECD88 to create d3d render context

++++++++++onContextCreated(surface 00000311C8CAD390, ctx 00000311C0B1E330)+++++++++++

set d3d11 device. Feature level: 0XC100, create flags: 0

d3d11: 00000311C8151718, dxgi adapter vendor 8086, device 3ea0, revision 0, Intel(R) UHD Graphics 620. UMA

set d3d11 device. Feature level: 0XC100, create flags: 0

d3d11: 00000311C8151718, dxgi adapter vendor 8086, device 3ea0, revision 0, Intel(R) UHD Graphics 620. UMA

PlatformSurface::Event::Resize 0x0

应该是有的，这第一个貌似就是

axojhf commented 9 months ago

63aad0b8-7c24-40ff-85a8-640d944cc325 这个是ISwapChainPanelNative

axojhf commented 9 months ago

WinUI3的idl文件里是这么描述ISwapChainPanelNative的

[
    object,
    uuid( 63aad0b8-7c24-40ff-85a8-640d944cc325 ),
    local,
    pointer_default(unique)
]
interface ISwapChainPanelNative: IUnknown
{
    HRESULT SetSwapChain([in, annotation("_In_")] IDXGISwapChain *swapChain);
}

cpp_quote("#endif // NTDDI_VERSION >= NTDDI_WINBLUE")

cpp_quote("#if (NTDDI_VERSION >= NTDDI_WINTHRESHOLD)")
[
    object,
    uuid( 88fd8248-10da-4810-bb4c-010dd27faea9 ),
    local,
    pointer_default(unique)
]
interface ISwapChainPanelNative2: ISwapChainPanelNative
{
    HRESULT SetSwapChainHandle([in, annotation("_In_")] HANDLE swapChainHandle);
}
cpp_quote("#endif // NTDDI_VERSION >= NTDDI_WINTHRESHOLD")

相关头文件在这个nuget里 https://www.nuget.org/packages/Microsoft.WindowsAppSDK (microsoft.ui.xaml.media.dxinterop.h)

wang-bin commented 9 months ago

有没有ISwapChainPanel的uuid？我这边需要ISwapChainPanel。 你对比下uwp的日志看看

axojhf commented 9 months ago

有没有ISwapChainPanel的uuid？我这边需要ISwapChainPanel。 你对比下uwp的日志看看

ISwapChainPanel的是08844F85-AA1B-540D-BEF2-B2BB7B257F8C，UWP我没办法安装环境，不好测试

wang-bin commented 9 months ago

有没有ISwapChainPanel的uuid？我这边需要ISwapChainPanel。 你对比下uwp的日志看看

ISwapChainPanel的是08844F85-AA1B-540D-BEF2-B2BB7B257F8C，UWP我没办法安装环境，不好测试

这个uuid哪来的？我搜windows sdk没有。windows sdk里是这样的

// windows.ui.xaml.controls.h
namespace ABI {
    namespace Windows {
        namespace UI {
            namespace Xaml {
                namespace Controls {
                    MIDL_INTERFACE("c589644f-eba8-427a-b75a-9f1f93a11ae9")
                    ISwapChainPanel : public IInspectable
                    {
                    public:
axojhf commented 9 months ago

好像不是直接定义在某个头文件里，我创建了一个WinUI3 C++项目，在xaml里用了之后，它似乎是在编译的时候被生成出来的

在Microsoft.UI.Xaml.Controls.0.h文件里：

    template <> inline constexpr guid guid_v<winrt::Microsoft::UI::Xaml::Controls::ISwapChainPanel>{ 0x08844F85,0xAA1B,0x540D,{ 0xBE,0xF2,0xB2,0xBB,0x7B,0x25,0x7F,0x8C } }; // 08844F85-AA1B-540D-BEF2-B2BB7B257F8C
    template <> inline constexpr guid guid_v<winrt::Microsoft::UI::Xaml::Controls::ISwapChainPanelFactory>{ 0x38D00B69,0x5759,0x5F37,{ 0x9E,0x1C,0x3A,0xE0,0xD4,0x28,0x8B,0x26 } }; // 38D00B69-5759-5F37-9E1C-3AE0D4288B26
    template <> inline constexpr guid guid_v<winrt::Microsoft::UI::Xaml::Controls::ISwapChainPanelStatics>{ 0xD83B0621,0xAACA,0x534B,{ 0xAF,0xDE,0x53,0x70,0x16,0x71,0x76,0xB5 } }; // D83B0621-AACA-534B-AFDE-5370167176B5

以及在C#里，靠VS的反编译查看也能看到 image

wang-bin commented 9 months ago

windows sdk里也有同样的文件，uuid也和你的不一样，奇怪了。WinUI3 C++工程怎么创建？

// cppwinrt/winrt/impl/microsoft.ui.xaml.controls.0.h
    template <> inline constexpr guid guid_v<winrt::Windows::UI::Xaml::Controls::ISwapChainPanel>{ 0xC589644F,0xEBA8,0x427A,{ 0xB7,0x5A,0x9F,0x1F,0x93,0xA1,0x1A,0xE9 } }; // C589644F-EBA8-427A-B75A-9F1F93A11AE9

namespace不一样，一个Windows一个Microsoft

axojhf commented 9 months ago

按照这两个，在VS里创建项目的时候应该就有模板了 https://learn.microsoft.com/windows/apps/windows-app-sdk/set-up-your-development-environment?tabs=cs-vs-community%2Ccpp-vs-community%2Cvs-2022-17-1-a%2Cvs-2022-17-1-b#required-workloads-and-components https://learn.microsoft.com/windows/apps/winui/winui3/create-your-first-winui3-app#packaged-create-a-new-project-for-a-packaged-c-or-c-winui-3-desktop-app

然后我是在MainWindow里加上

<SwapChainPanel x:Name="swapChainPanel" HorizontalAlignment="Stretch" VerticalAlignment="Stretch"/>

再编译的

wang-bin commented 9 months ago

你把所有生成的文件打包一份给我，我这边复制些相关接口的定义估计估计就好操作了

axojhf commented 9 months ago

Generated Files.zip

wang-bin commented 9 months ago

WinUI3的idl文件里是这么描述ISwapChainPanelNative的

[
    object,
    uuid( 63aad0b8-7c24-40ff-85a8-640d944cc325 ),
    local,
    pointer_default(unique)
]
interface ISwapChainPanelNative: IUnknown
{
    HRESULT SetSwapChain([in, annotation("_In_")] IDXGISwapChain *swapChain);
}

cpp_quote("#endif // NTDDI_VERSION >= NTDDI_WINBLUE")

cpp_quote("#if (NTDDI_VERSION >= NTDDI_WINTHRESHOLD)")
[
    object,
    uuid( 88fd8248-10da-4810-bb4c-010dd27faea9 ),
    local,
    pointer_default(unique)
]
interface ISwapChainPanelNative2: ISwapChainPanelNative
{
    HRESULT SetSwapChainHandle([in, annotation("_In_")] HANDLE swapChainHandle);
}
cpp_quote("#endif // NTDDI_VERSION >= NTDDI_WINTHRESHOLD")

相关头文件在这个nuget里 https://www.nuget.org/packages/Microsoft.WindowsAppSDK (microsoft.ui.xaml.media.dxinterop.h)

这idl文件在哪里？打包的头文件里没找到相关接口的定义。用 vs创建的模板只有 c# 的？

axojhf commented 9 months ago

microsoft.ui.xaml.media.dxinterop.idl在这个nuget包（https://www.nuget.org/packages/Microsoft.WindowsAppSDK ）的include文件夹下，不是后来生成的 image

wang-bin commented 9 months ago

cppwinrt看懂了些，加了几个接口定义。 试下最新的包，swapchain这边支持了winui3

axojhf commented 9 months ago

Crash了，不知道会不会是UI线程相关的问题

0x00007FF82318E1B3 (mdk.dll)处(位于 Example.exe 中)引发的异常: 0xC0000005: 读取位置 0x0000000000000000 时发生访问冲突。 image

SwapChainPanel应该是识别成功了

日志尾部：

native surface class name: Microsoft.UI.Xaml.Controls.SwapChainPanel
native surface is a WinUI SwapChainPanel
axojhf commented 9 months ago

如果和DependencyObject有关，可能是WinUI3这边使用的是winrt::Microsoft::UI::Xaml::DependencyObject

wang-bin commented 9 months ago

DependencyObject处理过了。你等会试下 https://github.com/wang-bin/mdk-sdk/actions/runs/8094410448

axojhf commented 9 months ago

好吧，日志说dispatcher是空的，会不会是WinUI3和UWP的API差异导致的 https://learn.microsoft.com/en-us/windows/windows-app-sdk/api/winrt/microsoft.ui.xaml.controls.swapchainpanel?view=windows-app-sdk-1.4 image

这个是官方的迁移提示： https://learn.microsoft.com/windows/apps/windows-app-sdk/migrate-to-windows-app-sdk/guides/threading#change-coredispatcherrunasync-to-dispatcherqueuetryenqueue

wang-bin commented 9 months ago

试下最新包

axojhf commented 9 months ago

还是不太正常，日志说dispatcher queue是空的，在C#里验证过DispatcherQueue.TryEnqueue是可用的

Level: Debug Msg: native surface class name: Microsoft.UI.Xaml.Controls.SwapChainPanel

Level: Debug Msg: native surface is a WinUI SwapChainPanel

Level: All Msg: dispatcher queue: 0000000000000000 Microsoft.UI.Dispatching.DispatcherQueue

Level: Debug Msg: dispatcher->TryEnqueueWithPriority((int32_t)WinUI::DispatcherQueuePriority::Normal, Make<WinUiDispatcherQueueHandler>(std::move(f), std::move(p)).Get(), &enqueued)  ERROR@87RunOnUIThread: (8000001c) �������֧�� IAgileObject �ӿ�

Level: Debug Msg: 
Level: Debug Msg: GetOnUIThread(dp, std::move(f))  ERROR@186DXGI::RenderContext::createSwapChain: (8cfcdde0) unknown error

Level: Debug Msg: 
Level: Debug Msg: ++++++++++onContextCreated(surface 0000022598A4EE00, ctx 000002258CFCDDE0)+++++++++++

Level: Debug Msg: set d3d11 device. Feature level: 0XC100, create flags: 0

Level: Debug Msg: d3d11: 000002259868CDF8, dxgi adapter vendor 8086, device 3ea0, revision 0, Intel(R) UHD Graphics 620. UMA

Level: Debug Msg: set d3d11 device. Feature level: 0XC100, create flags: 0

Level: Debug Msg: d3d11: 000002259868CDF8, dxgi adapter vendor 8086, device 3ea0, revision 0, Intel(R) UHD Graphics 620. UMA

Level: Debug Msg: PlatformSurface::Event::Resize 0x0
wang-bin commented 9 months ago

不是空的，日志打错了。报错是TryEnqueueWithPriority的handler必须是Agile object，没看到文档有描述, ICoreDispatcher倒是有说要agile

wang-bin commented 9 months ago

https://github.com/wang-bin/mdk-sdk/actions/runs/8106009351 试下这个

axojhf commented 9 months ago

这个包好像编译有问题

wang-bin commented 9 months ago

https://github.com/wang-bin/mdk-sdk/actions/runs/8107608935

axojhf commented 9 months ago

没崩溃了，但是可能还是有一定问题 灰色部分应该是SwapChainPanel的区域： image 设置updateNativeSurface后（设置宽高也没用）： image

日志好像也有个错误在

Level: All Msg: dispatcher queue: 00000162F9F74460 Microsoft.UI.Dispatching.DispatcherQueue

Level: Debug Msg: ++++++++++onContextCreated(surface 000001A39E82DB00, ctx 000001A3962CDFE0)+++++++++++

Level: Debug Msg: set d3d11 device. Feature level: 0XC100, create flags: 0

Level: Debug Msg: d3d11: 000001A39C266A68, dxgi adapter vendor 8086, device 3ea0, revision 0, Intel(R) UHD Graphics 620. UMA

Level: Debug Msg: set d3d11 device. Feature level: 0XC100, create flags: 0

Level: Debug Msg: d3d11: 000001A39C266A68, dxgi adapter vendor 8086, device 3ea0, revision 0, Intel(R) UHD Graphics 620. UMA

Level: Debug Msg: PlatformSurface::Event::Resize 0x0

Level: Debug Msg: hr = d->swapchain->ResizeBuffers(sd.BufferCount, w, h, d->newFormat, sd.Flags)  ERROR@239DXGI::RenderContext::resizeSwapChainBuffers: (80070057) ��������

Level: Debug Msg: 
Level: Debug Msg: 000001A395F99170 setVideoSurfaceSize(0x0, 000001A3961114D8)
wang-bin commented 9 months ago

resize 0x0, 你宽高设了多少？有相关日志出现吗？

axojhf commented 9 months ago

updateNativeSurface里设置的是720x400，其他日志没有这两个数存在 尝试在调用updateNativeSurface后马上调用setVideoSurfaceSize，有日志000002E7B3FAF0E0 setVideoSurfaceSize(720x400, 0000000000000000)，但是这个时候还没有创建swapchain，之后被自动设置为0x0了

axojhf commented 9 months ago

MainWindow.xaml是这样的

<?xml version="1.0" encoding="utf-8"?>
<Window
    x:Class="MDKWinUI3Example.MainWindow"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d">
    <StackPanel>
        <Button x:Name="myButton" Click="myButton_Click">Click Me</Button>
        <VariableSizedWrapGrid Background="Gray" ItemWidth="720" ItemHeight="400" Orientation="Horizontal">
            <SwapChainPanel x:Name="swapChainPanel" HorizontalAlignment="Stretch" VerticalAlignment="Stretch"/>
        </VariableSizedWrapGrid>
    </StackPanel>
</Window>

我是通过swapChainPanel的ActualWidth和ActualHeight字段获取宽高的

wang-bin commented 9 months ago

要在swapchain创建后update一次，看来外面调的时机不太好控制。最新包加了检测大小变化了（dpi还没处理），使用方法和uwp版本完全一样了

axojhf commented 9 months ago

OK，能正常显示了，感谢，辛苦了

axojhf commented 9 months ago

GPU资源还占用还算挺低的，虽然比Qt RHI的要高一些，但是比Avaloina那边好多了

Avalonia 11.0那边自定义渲染还是有挺多性能问题的，目前Avaloina在Win上结合自定义渲染器（DX，Vk）的方式貌似是用D3D共享纹理做的，用空载的纹理也经常占用20~30%左右的GPU（UHD 620） 换来换去可能还只有Avalonia用的angle直接提供的opengl es性能最好