Better document (and/or support?) consuming the Windows App SDK from C++, without Visual Studio

[Donpedro13]

Describe the bug

Consuming/using the Windows App SDK from C++ using Visual Studio is well documented. Visual Studio does all the heavy lifting (restoring nuget packages, invoking cppwinrt, etc.), so the whole process is quite hassle-free, it works out of the box. However, not everyone is using Visual Studio or MSBuild as build tools, especially in the C++ world.

Consuming “regular” 3rd party libraries from C++ is not a big deal (usually): there are header files, static libraries and/or import libraries and DLL files. Add the header files’ location to your include directory, link the given libraries, and you are good to go.

Consuming the Windows App SDK seems to be a whole lot more complex. In addition to the the usual files, there are also .winmd files, and a tool - cppwinrt.exe - to generate header files from winmd files for the WinRT C++ language projection. Some header files come pre-generated (like Microsoft.Xaml.UI.Window.h), while others don’t (like Microsoft.UI.Xaml.Controls.h). Visual Studio seems to generate these header files on the fly, as needed (Processing WinMD lines in the build output). There’s also some extra machinery in the background, like bootstrapping. If you try and use certain functions without bootstrapping, you will get runtime errors. Good luck debugging and root causing those to combase.dll not being runtime patched in your application.

Just to give you some context: I was happy to see that using Win32 controls and WinUI 3 in the same (unpackaged) C++ application is finally supported. The Simple Island App sample shows the necessary steps to achieve interoperability between the two worlds. I thought I’d give this a try in my day job: a large, multi-platform, enterprise application with a Win32 UI on Windows. Said project - of course - does not use Visual Studio as a build tool, so my first step was to try and consume the Windows App SDK in this environment.

The end result? Countless hours of trying to “reverse engineer” what, how, and when Visual Studio does to make it all work. Initially, this method seemed to work, fixing error after error, however, I gave up after a few days. Many times errors manifested in deeper layers of the implementation, diagnosing them would have required either in-depth knowledge of the implementation details, or access to the source code - I have neither.

All in all, the introduction of Win32-WinUI 3 interop is a good sign: maybe Microsoft does want legacy applications to be modernized with the Windows App SDK and WinUI 3, which sounds great! However, probably not many of these kinds of applications use Visual Studio as a build tool, so improvements in this space (such as explicit support and documentation for this use case) would be very welcome.

Steps to reproduce the bug

Try to consume the Windows App SDK without Visual Studio and MSBuild, from a C++ project.

Expected behavior

The described scenario is supported, the necessary build steps are documented.

Screenshots

No response

NuGet package version

Windows App SDK 1.4.1: 1.4.230913002

Packaging type

Unpackaged

Windows version

Windows 11 version 22H2 (22621, 2022 Update)

IDE

Visual Studio 2022, Other

Additional context

No response

[DarranRowe]

Preparation is the worst of this, since you need to generate the WinRT component headers from the .winmd files. There are a couple of extra annoyances, like how the Windows App SDK wants Windows 10 compatibility marked in the application manifest file as well as DPI awareness.

If you are not attempting to build a full Xaml type application, it isn't too bad to use the Windows App SDK from the command line. Wanting to use it self contained makes it a bit more tricky, but it is still pretty easy. Wanting to use the WinUI 3 type of application is just an outright pain because of the Xaml compiler. There is an executable Xaml compiler, but it is so obtuse and undocumented that it is near impossible to figure out how to use without really digging into the build files.

I would have to prepare a little to write a little guide, but it is just getting things to fit into the right places. The only suggestion I have is to familiarise yourself with the concept of registration free WinRT. If you wish to emulate this, you have to know how it works.

[DarranRowe]

As a bit of a quick sample, just to show it working. This is built using VC2022 17.7.4 using the Windows SDK version 10.0.22621 (the Windows 11 22H2 SDK). The preparation for this is simply grab all of the NuGet packages required and extract them. The only thing to note about my setup is that I don't use the default VS/Windows SDK install directories. The NuGet packages that I used for this are: Windows App SDK C++/WinRT AbiWinRT

This is the directory layout. Ignore the extra packages for now. Since the Windows SDK is 22621, the build tools aren't needed. Windows Implementation Library is useful, but since this is just a sample then it is unneeded. The only preparation done is to extract the NuGet packages and then renamed them to something more manageable. These packages are just .zip files.

The Visual Studio command line that I am using is targeting x64 and is using the Windows 11 22H2 Windows SDK. The only other modification that I did was to add things to the paths.

C:\Users\Darran\cmdlinebuild\testprj>set OLD_PATH=%PATH%
C:\Users\Darran\cmdlinebuild\testprj>set PATH=C:\Users\Darran\cmdlinebuild\testprj\cppwinrt\bin;C:\Users\Darran\cmdlinebuild\testprj\abiwinrt\bin;%OLD_PATH%

This backs up the path and then adds cppwinrt.exe and abi.exe to the path. The backup was done to allow easy removal.

C:\Users\Darran\cmdlinebuild\testprj>set OLD_INCLUDE=%INCLUDE%
C:\Users\Darran\cmdlinebuild\testprj>set INCLUDE=C:\Users\Darran\cmdlinebuild\testprj\include;C:\Users\Darran\cmdlinebuild\testprj\windowsappsdk\include;%OLD_INCLUDE%

This backs up the include environment and adds the generated include directory and the Windows App SDK include directory to the include environment.

C:\Users\Darran\cmdlinebuild\testprj>set OLD_LIB=%LIB%
C:\Users\Darran\cmdlinebuild\testprj>set LIB=C:\Users\Darran\cmdlinebuild\testprj\windowsappsdk\lib\win10-x64;C:\Users\Darran\cmdlinebuild\testprj\windowsappsdk\lib\native\win10-x64;C:\Users\Darran\cmdlinebuild\testprj\cppwinrt\build\native\lib\x64;%OLD_LIB%

Finally, this does the same with the library paths. Anyway, now that is done, the preparation instructions are next.

For C++/WinRT, you need to generate the platform projection first. These headers are versioned, so they really like it if you do this.

C:\Users\Darran\cmdlinebuild\testprj>cppwinrt.exe -input 10.0.22621.0+ -reference 10.0.22621.0+ -output include -optimize -base

Next, generate the Windows App SDK projection. If you notice in my screenshot above, I have a winmd directory. This is there to copy the Windows App SDK .winmd files to. There are reasons behind it, just having the .winmd files in a directory makes things easier when using the command line. Also, since I am using the Windows 11 SDK and really targeting Windows 11, I am using the .winmd files from the 10.0.18362 directory. You should only use the 17763 directory if you are targeting this version of Windows. There are a few things that are unavailable.

C:\Users\Darran\cmdlinebuild\testprj>copy windowsappsdk\lib\uap10.0.18362*.winmd winmd C:\Users\Darran\cmdlinebuild\testprj>copy windowsappsdk\lib\uap10.0*.winmd winmd C:\Users\Darran\cmdlinebuild\testprj>for %i in (winmd*.winmd) do cppwinrt.exe -input %i -reference 10.0.22621.0+ -reference winmd -output include -optimize -base

This generates the C++/WinRT projection for the Windows App SDK. With this done, here is the code that I am using for the C++ source.

#include <Windows.h>
#include <winrt/Windows.Foundation.h>
#include <winrt/Windows.Foundation.Collections.h>
#include <winrt/Microsoft.UI.Dispatching.h>

int wmain()
{
    auto dqc = winrt::Microsoft::UI::Dispatching::DispatcherQueueController::CreateOnCurrentThread();

    dqc.ShutdownQueue();
    return 0;
}

This activates a dispatcher queue on the current thread and then shuts it down. The application manifest that I am using is a modification of a standard manifest file that I use for my projects.

<?xml version="1.0" encoding="utf-8"?>
<assembly manifestVersion="1.0" xmlns="urn:schemas-microsoft-com:asm.v1" xmlns:ws="http://schemas.microsoft.com/SMI/2005/WindowsSettings" xmlns:ws4="http://schemas.microsoft.com/SMI/2016/WindowsSettings" xmlns:ws6="http://schemas.microsoft.com/SMI/2019/WindowsSettings">
  <assemblyIdentity version="1.0.0.0" name="test.App"/>

  <compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1">
    <application>
      <!-- Windows 10-->
      <supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}" />
      <maxversiontested Id="10.0.22621.0" />
    </application>
  </compatibility>
  <application xmlns="urn:schemas-microsoft-com:asm.v3">
    <windowsSettings>
      <ws:dpiAware>true/PM</ws:dpiAware>
      <ws4:dpiAwareness>PerMonitorV2, PerMonitor</ws4:dpiAwareness>
      <ws6:activeCodePage>UTF-8</ws6:activeCodePage>
    </windowsSettings>
  </application>
</assembly>

It declares Windows 10 compatibility, DPI awareness and sets the codepage to UTF-8. These files are named testwrt.cpp and settings.manifest. They are placed in the testprj directory.

The compiler/linker command line used is as follows. C:\Users\Darran\cmdlinebuild\testprj>cl /c /EHsc /O2 /sdl /utf-8 /std:c++17 testwrt.cpp C:\Users\Darran\cmdlinebuild\testprj>cl /c /EHsc /O2 /sdl /utf-8 /std:c++17 windowsappsdk\include\MddBootstrapAutoInitializer.cpp C:\Users\Darran\cmdlinebuild\testprj>link /out:testwrt.exe testwrt.obj MddBootstrapAutoInitializer.obj Microsoft.WindowsAppRuntime.Bootstrap.lib windowsapp.lib C:\Users\Darran\cmdlinebuild\testprj>mt -manifest settings.manifest -outputresource:testwrt.exe;#1 C:\Users\Darran\cmdlinebuild\testprj>copy windowsappsdk\runtimes\win10-x64\native\Microsoft.WindowsAppRuntime.Bootstrap.dll .

This compiles the source file and the auto initialiser. The auto initialiser initialises the Windows App SDK automatically. These are linked together with the import library for the bootstrapper and windowsapp.lib. The windowsapp.lib library is required to resolve some of the WinRT dependencies. The manifest file is then embedded into the executable using mt.exe which is part of the Windows SDK. Finally, the bootstrap library is copied into the same directory as the executable, since it is needed for the bootstrap auto initialiser.

[DarranRowe]

Using the ABI has a similar setup to using C++/WinRT outlined above. That had already added abi.exe to the path, so it was almost set up for using the ABI in the first place, but there is one more useful thing to add to this. One more useful library to use when using the ABI is Windows Implementation Library. For this, I grabbed the NuGet package and extracted it in the testprj directory and renamed the NuGet package directory to windowsimplementationlibrary.

This means that using the C++/WinRT instructions as a base, the only real modification to the environment is:

set INCLUDE=C:\Users\Darran\cmdlinebuild\testprj\include;C:\Users\Darran\cmdlinebuild\testprj\windowsappsdk\include;C:\Users\Darran\cmdlinebuild\testprj\windowsimplementationlibrary\include;%OLD_INCLUDE%

This adds the Windows Implementation Library includes to the path. From this, generating the ABI files is next. This happens after the .winmd files are copied to the winmd directory.

C:\Users\Darran\cmdlinebuild\testprj>for %i in (winmd*.winmd) do abi.exe -input %i -reference 10.0.22621.0+ -reference winmd -output include -ns-prefix -lowercase-include-guard -enable-header-deprecation C:\Users\Darran\cmdlinebuild\testprj>for %i in (winmd*.winmd) do winmdidl /outdir:include /metadata_dir:"D:\Programs32\Windows Kits\10\UnionMetadata\10.0.22621.0" /metadata_dir:winmd %i

The options provided for abi.exe makes the generated files close to the Windows SDK ABI headers. I also generate the .idl files too, but these are only really useful when you are generating components. Unlike C++/WinRT, the Windows SDK ABI headers are fine for usage here. This is why I don't generate them. For the files that I am going to use to build, the same application manifest that I used for C++/WinRT is used here.

The source file. named testwrl.cpp, is:

#include <Windows.h>
#include <wrl.h>
#include <windows.foundation.h>
#include <windows.foundation.collections.h>
#include <microsoft.ui.dispatching.h>

#include <cstdio>

int wmain()
{
    HRESULT hr = S_OK;

    Microsoft::WRL::Wrappers::HString dqc_name;
    hr = dqc_name.Set(RuntimeClass_Microsoft_UI_Dispatching_DispatcherQueueController);

    if (FAILED(hr))
    {
        wprintf(L"HString set failed\n");
        return hr;
    }

    Microsoft::WRL::ComPtr<ABI::Microsoft::UI::Dispatching::IDispatcherQueueControllerStatics> dqc_statics;
    Microsoft::WRL::ComPtr<ABI::Microsoft::UI::Dispatching::IDispatcherQueueController> dqc_ptr;

    hr = Windows::Foundation::GetActivationFactory(dqc_name, &dqc_statics);
    if (FAILED(hr))
    {
        wprintf(L"GetActivationFactory failed failed\n");
        return hr;
    }

    hr = dqc_statics->CreateOnCurrentThread(dqc_ptr.ReleaseAndGetAddressOf());
    if (FAILED(hr))
    {
        wprintf(L"CreateOnCurrentThread failed\n");
        return hr;
    }

    wprintf(L"DispatcherQueue creation succeeded\n");

    Microsoft::WRL::ComPtr<ABI::Microsoft::UI::Dispatching::IDispatcherQueueController2> dqc_ptr2;
    hr = dqc_ptr.As(&dqc_ptr2);
    if(FAILED(hr))
    {
        wprintf(L"QI failed\n");
        return hr;
    }

    hr = dqc_ptr2->ShutdownQueue();
    if (FAILED(hr))
    {
        wprintf(L"ShutdownQueue failed\n");
        return hr;
    }

    return 0;
}

This is using the ABI and WRL, which is part of the Windows SDK. I don't use the Windows Implementation Library here since it is unneeded for the sample. I build it using:

C:\Users\Darran\cmdlinebuild\testprj>cl /c /EHsc /O2 /sdl /utf-8 /std:c++17 testwrl.cpp C:\Users\Darran\cmdlinebuild\testprj>cl /c /EHsc /O2 /sdl /utf-8 /std:c++17 windowsappsdk\include\MddBootstrapAutoInitializer.cpp C:\Users\Darran\cmdlinebuild\testprj>link /out:testwrl.exe testwrl.obj MddBootstrapAutoInitializer.obj Microsoft.WindowsAppRuntime.Bootstrap.lib windowsapp.lib C:\Users\Darran\cmdlinebuild\testprj>mt -manifest settings.manifest -outputresource:testwrl.exe;#1

It builds successfully. Again, you have to copy Microsoft.WindowsAppRuntime.Bootstrap.dll to the same path as the executable. Also, while I use std:c++17 in the command line, WRL works with C++11/14.

For the simpler framework dependant Windows App SDK usage, this is all that is needed.

[fredemmott]

https://github.com/fredemmott/cmake-cpp-winrt-winui3 is a stale minimal example of using WindowsAppSDK including WinUI3 with Xaml from CMake

I'm no longer maintaining that, but https://github.com/OpenKneeboard/OpenKneeboard is an up-to-date real world example.

[DarranRowe]

Self contained can be not much worse than already stated, if you don't want to use Xaml in any form. If you want to use Xaml, even using the hosting API, then it is a really good idea to get the resources into place.

For self contained without Xaml, based upon the layout above, there are very few changes that you need to make. First, you don't use MddBootstrapAutoInitializer.cpp, instead you would use UndockedRegFreeWinRT-AutoInitializer.cpp for the Windows App SDK auto initialiser. This is a direct replacement and can compile with the same command line options. Secondly, you would need to handle the WinRT component registration. When you are dependant on the Windows App Runtime .msix package, the component registration is in the package itself. But when you are not depending on the .msix package, the WinRT component registration doesn't exist. You need to provide it in another way in order for the application to work. For this, the Windows App SDK uses Registration Free WinRT, and the intent is that the registration will go into the application manifest file. The registration information is available in the Windows App SDK NuGet package itself. There are 3 manifest files in the NuGet package, so you can mostly just use these and merge them with your own manifest. I use mostly because one of the manifests is bad in the 1.4.1 NuGet package. If you use it as is then it will cause the application to fail to start due to a side by side manifest error. Thankfully, the merged manifest is a build artefact that only needs to be generated once per major version of the Windows App SDK. The manifest file Microsoft.WindowsAppSdk.Foundation.manifest has an empty file element, <file name="" />, at the bottom. Windows doesn't like this. You should remove this line. Once you do this, you just use mt.exe to merge the manifests together.

So from layout above, if you add a directory named manifests to testprj, copy the Windows App SDK manifests from windowsappsdk\manifests and make any changes needed, you can then merge the manifests into one.

C:\Users\Darran\cmdlinebuild\testprj>mt -manifest manifests\Microsoft.WindowsAppSdk.Foundation.manifest manifests\Microsoft.InteractiveExperiences.manifest manifests\Microsoft.WindowsAppSdk.WinUI.manifest -out:manifests\winappsdkmerged.manifest

When you build, you can then merge this with your application manifest and embed it into the executable.

C:\Users\Darran\cmdlinebuild\testprj>mt -manifest settings.manifest manifests\winappsdkmerged.manifest -outputresource:testwrt.exe;#1

As a reminder, the Microsoft.WindowsAppSdk.Foundation.manifest file was modified to remove a bad element, the other manifests are used as is. To set up the self contained directory, you would then find the Windows App Runtime .msix package extract the contents and place it in the same directory as the executable you just built. Really, you should rename resources.pri to Microsoft.UI.Xaml.Controls.pri too, but this isn't important in an unpackaged application. That file name is what the Windows App SDK build process uses. The Windows App Runtime package can be found in the Windows App SDK NuGet package. You can find them under the tools\msix directory. The package that you want to use is the platform correct Microsoft.WindowsAppRuntime.[version].msix. So for 1.4 it would be Microsoft.WindowsAppRuntime.1.4.msix. This file, like the NuGet package, is just a renamed .zip file. There are some things that you can strip out of the extracted package too. The AppxMetadata directory, [Content_Types].xml and all of the Appx* files. The language directories can also be trimmed down a little, but you really should have the languages that you expect to deploy available in the directory.

[DarranRowe]

If you want to use Xaml, then there is extra work that needs to be done to get the resources into place. This means battling with makepri.exe. I didn't touch on makepri.exe when working with the framework dependent version because you don't really need to do it unless you use MRT Core for your own resources. The Xaml resources would be obtained from the Windows App Runtime package. When you make the application self contained, you can think of it along the lines of your application becomes the package. This means you have to provide the Xaml resources in your application. Thankfully, the build artefact for makepri is something that can be cached in this case. Working with makepri also ends up being necessary if you intend to use custom controls too. Currently, the Xaml runtime only looks at one pri file, so all of your resources need to be in that file. You have some control over the name in 1.4, but the name it looks for by default is either resources.pri if the application is packaged, or [executable base name].pri if the application is unpackaged. To potentially answer a question in advance, your application is unpackaged unless you do some extra steps with makeappx.exe.

So, to give steps on putting everything in place for working with resources. Let's assume that I extract the Windows App Runtime .msix file to a directory named windowsappruntime in the testprj directory as mentioned above. The only modifications made to this package is the file named resources.pri has been renamed to Microsoft.UI.Xaml.Controls.pri. The first file you need to create is a file that tells makepri where the extra resource files that you want it to use are. The build process names this pri.resfiles. It also only contains the absolute paths of all of the .pri files you want to merge. So for this little example, it would contain:

C:\Users\Darran\cmdlinebuild\testprj\windowsappruntime\Microsoft.UI.Xaml.Controls.pri

Any extra .pri files, for example custom control resources, would need to be added to this file. One .pri file per line. The Visual Studio generated file also has an extra blank line at the end. The next thing that you need is a configuration file for makepri. This tells makepri what to look for and how to interpret things. The default name used is priconfig.xml. If you are only merging .pri files, then the config file is pretty simple.

<?xml version="1.0" encoding="utf-8"?>
<resources targetOsVersion="10.0.0" majorVersion="1">
    <index root="\" startIndexAt="pri.resfiles">
        <default>
            <qualifier name="Language" value="en-GB" />
            <qualifier name="Contrast" value="standard" />
            <qualifier name="Scale" value="200" />
            <qualifier name="HomeRegion" value="001" />
            <qualifier name="TargetSize" value="256" />
            <qualifier name="LayoutDirection" value="LTR" />
            <qualifier name="DXFeatureLevel" value="DX9" />
            <qualifier name="Configuration" value="" />
            <qualifier name="AlternateForm" value="" />
            <qualifier name="Platform" value="UAP" />
        </default>
        <indexer-config type="PRI" />
        <indexer-config type="RESFILES" qualifierDelimiter="." />
    </index>
</resources>

Things like the language are per project configuration. The startIndexAt is basically the relative path of the file in relation to the project root passed into makepri. Using the files above, with both pri.resfiles and priconfig.xml in the testprj directory, if you use the command line:

C:\Users\Darran\cmdlinebuild\testprj>makepri new /pr C:\Users\Darran\cmdlinebuild\testprj /cf C:\Users\Darran\cmdlinebuild\testprj\priconfig.xml /of testwrt.pri /in testwrt /o /v

Then this will generate the file correctly. The new option tells makepri to make a new .pri file. The /pr option tells makepri how to interpret paths in the priconfig.xml file. The /of option gives the name of output .pri file. As you can see, I named it testwrt.pri. This is to match the testwrt.exe name for the C++/WinRT test application. The /in gives the index name. This is the root index for the project and really should match the base executable name. While I placed all of the files in the testprj directory, you can specify different directories for various options. What's more the startIndexAt attribute in the priconfig file is a relative path. This can be in a sub directory. If you place this .pri file next to the .exe file and then place the Windows App Runtime files then this should be a complete self contained layout that works with Xaml. This was spread out over a few posts, but this is should be the basics of consuming the Windows App SDK from outside of Visual Studio.

[driver1998]

If you are not just talking about different build system, and also different compiler (so basically MinGW), you may want to take a look at https://github.com/driver1998/winui-mingw.

Mind that XAML will require a lot of boilerplate code to get working, also not everything works.

[Donpedro13]

Thank you everyone for commenting. Thanks @DarranRowe for the very detailed descriptions.

One last thing that nobody mentioned is that in the Simple Island App, Microsoft.UI.Windowing.Core.dll is delay loaded. I wonder if this is also a requirement for applications consuming the Windows App SDK, or is it just a WinUI 3 (/Win32) thing?

[DarranRowe]

That is a Xaml Island specific thing. To properly handle the islands, you need to call ContentPreTranslateMessage in your main message pump. This function is exported from Microsoft.UI.Windowing.Core.dll. This leaves a pretty big problem, when you are framework dependant, that is using a means like MddBootstrapInitialize to get access to the Windows App SDK, Microsoft.UI.Windowing.Core.dll will not be visible until after the process bootstraps the Windows App SDK. This is an operation that must occur during the process' execution, however, the process can only execute code if it loads. Normally, Windows has to resolve all DLL references before it allows code execution to start. Setting Microsoft.UI.Windowing.Core.dll as delay loaded means that Windows doesn't need to resolve this DLL until it is first used, thus breaking the need the DLL to load and need to load to access the DLL cycle.

[DarranRowe]

As one final follow up to this. With the release of the WinUI 3 source on the WinUI repository, the Xaml compiler source is also available. There is enough information in this to piece together the format of the .json input file. While it is true that this is still not the nicest command line tool to use, it is still possible to use it. There is still some things which aren't explicitly called out, like the generation of the metadata provider. Figuring out what Visual Studio does isn't that bad. Knowing what to do and in what order would be nice to have documented though.

--Edit-- Using the diagnostic build log from Visual Studio, the intermediate files for reference and information found in the source code, I was able to build a WinUI 3 completely using the command line tools and no tricks using MSBuild.