search

microsoft / microsoft-ui-xaml

Windows UI Library: the latest Windows 10 native controls and Fluent styles for your applications
MIT License
6.36k stars 679 forks source link

Can't get error output from XamlCompiler.exe #10027

Open res2k opened 1 month ago

res2k commented 1 month ago

Describe the bug

I'm trying to experiment with using the standalone XamlCompiler.exe. One thing I noticed is that it doesn't seem to really report any errors, it just fails with exit code 1. Upon closer inspection, it seems any entry added to the ConsoleLogger used by the standalone program isn't really output in the case of error. The only use of ConsoleLogger.Entries is in SaveResults(). That, however, is only executed in the case of success: https://github.com/microsoft/microsoft-ui-xaml/blob/35c590bb28841eb9d466624bb828c78b939d4312/src/src/XamlCompiler/BuildTasks/ConsoleCompileXaml.cs#L77 So it seems if there's an error, the actual message isn't actually output anywhere...

Steps to reproduce the bug

  1. Craft an input JSON file for XamlCompiler.exe that successfully parses: 
    {
"Language" : "CppWinRT",
"LanguageSourceExtension" : ".cpp",
"OutputPath" : "D:/sources/xamlc_standalone_test/build/generated",
"ProjectPath" : "D:/sources/xamlc_standalone_test",
"ReferenceAssemblies" : [],
"ReferenceAssemblyPaths" : [],
"TargetPlatformMinVersion" : "10.0.17763.0",
"XamlPages" : 
[
{
  "FullPath" : "D:/sources/xamlc_standalone_test/MainWindow.xaml",
  "ItemSpec" : "D:/sources/xamlc_standalone_test/MainWindow.xaml"
}
]
}
  2. Run XamlCompiler.exe manually
  3. Notice the error return, but don't see any error output (or output json)

Expected behavior

No response

Screenshots

The standalone compiler should report errors, at least to the console, if it doesn't produce an output json.

NuGet package version

WinUI 3 - Windows App SDK 1.6.0: 1.6.240829007

Windows version

Windows 11 (22H2): Build 22621

Additional context

No response

github-actions[bot] commented 1 month ago

Hi I'm an AI powered bot that finds similar issues based off the issue title.

Please view the issues below to see if they solve your problem, and if the issue describes your problem please consider closing this one. Thank you!

Open similar issues:

Note: You can give me feedback by thumbs upping or thumbs downing this comment.

DarranRowe commented 1 month ago

One of the big issues here is that the only supported way of using the Xaml compiler is through MSBuild. So there really should be a feature request to provide a supported version of the Xaml compiler that can be run from the command line.

I will need to check my notes, because I did do a lot of work to figure out how to use the Xaml compiler from the command line. But be aware, this is going to be unsupported by Microsoft. I'll either edit this post or do another reply when I have figured everything out.

DarranRowe commented 1 month ago

Right, this will be a long post. It will also include some steps from other tools since they are required for everything to work correctly when you get to the compilation stage. I will list things in the order that Visual Studio did them at 17.7, since that was when I did this work. This was also using the Windows App SDK 1.4.2. Again, some of these can be done slightly out of order.

  1. XamlMetaDataProvider.idl is generated.
  2. Use midl to generate the .winmd files for the project, including the generated XamlMetaDataProvider.idl.
  3. Use mdmerge to merge the project .winmd files into a single .winmd file. This is named after the project's root namespace.
  4. The Xaml compiler is run for the first time.
  5. Generate XamlMetaDataProvider.cpp.
  6. Generate the platform projection using C++/WinRT. This first step only generates the headers for the platform. This doesn't use Windows.winmd, it uses the contracts listed under <Windows Kits Directory>\10\Platforms\UAP\<Windows SDK Version>\Platform.xml.
  7. Generate the projection for the extra references, this is where the projection for the Windows App SDK is generated.
  8. Generate the projection for the components in your project.
  9. The Xaml compiler is run for the second time.

At this point, the source and header files for the project have been generated. It then does the precompiled header compile, compile the rest of the source files for the project and then links it. The last step is to make the .pri file for the project. I glossed over some of this, but remember if you set Visual Studio to diagnostic logging, then you can get all of these steps.

Now, out of all those steps, the important thing for the Xaml compiler is the generation of the merged .winmd for your project for the second pass.

The contents of the generated XamlMetaDataProvider.idl is:

//XamlMetaDataProvider.idl
//The namespace must match your application's root namespace.
namespace xcmanualtest
{
    runtimeclass XamlMetaDataProvider : [default] Microsoft.UI.Xaml.Markup.IXamlMetadataProvider
    {
        XamlMetaDataProvider();
    }
}

The namespace I have there is just the namespace that I was using for the manual project. This name isn't special, but you must make it match everything in your project. The contents of XamlMetaDataProvider.cpp is:

#include "pch.h"
#include "XamlMetaDataProvider.h"
#include "XamlMetaDataProvider.g.cpp"

Basically, the contents of the file is generated in XamlMetaDataProvider.g.cpp and XamlMetaDataProvider.cpp only exists to pull the real contents in. Now for the not very fun part, the Xaml compiler input itself.

The main .json format follows what is in ICompileXamlInputs region of CompileXaml.cs. But the actual accepted values are not really documented there, I had to really dig through the source code and use diagnostic Visual Studio output to get anywhere. Any references follow the public interface of MSBuildItem.cs.

Now, the reference assemblies ends up being a pretty large list. It lists all of the .winmd files from the Windows App SDK, the WebView2 .winmd file, any other .winmd from NuGet packages or projects, and any referenced platform .winmd file by contract. It does not reference the full Windows.winmd. Two examples of the referenced .winmd files are

    {
      "DependentUpon": "",
      "FullPath": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\was\\lib\\uap10.0.18362\\Microsoft.Foundation.winmd",
      "ItemSpec": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\was\\lib\\uap10.0.18362\\Microsoft.Foundation.winmd",
      "IsSystemReference": false,
      "IsNuGetReference": false,
      "IsStaticLibraryReference": false,
      "MSBuild_Link": "",
      "MSBuild_TargetPath": "",
      "MSBuild_XamlResourceMapName": "",
      "MSBuild_XamlComponentResourceLocation": ""
    }
    {
      "DependentUpon": "",
      "FullPath": "D:\\Programs32\\Windows Kits\\10\\References\\10.0.22621.0\\Windows.Foundation.UniversalApiContract\\15.0.0.0\\Windows.Foundation.UniversalApiContract.winmd",
      "ItemSpec": "D:\\Programs32\\Windows Kits\\10\\References\\10.0.22621.0\\Windows.Foundation.UniversalApiContract\\15.0.0.0\\Windows.Foundation.UniversalApiContract.winmd",
      "IsSystemReference": true,
      "IsNuGetReference": false,
      "IsStaticLibraryReference": false,
      "MSBuild_Link": "",
      "MSBuild_TargetPath": "",
      "MSBuild_XamlResourceMapName": "",
      "MSBuild_XamlComponentResourceLocation": ""
    }

The paths are specific to my system and the fact that the project was a manual use of the Xaml compiler. But for the ReferenceAssemblies lst, the FullPath and ItemSpec values are always equal. IsSystemReference is true for anything in the Windows SDK, otherwise the remaining values are false or empty strings. But it is important that you set these paths to their location on your system. The list of reference assemblies took up 1213 lines of my input.json files.

To give an example of the input.json files I used, the pass 1 file is:

{
  "ProjectPath": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\dummy.txt",
  "Language": "CppWinRT",
  "LanguageSourceExtension": ".cpp",
  "OutputPath": "generated\\",
  "ReferenceAssemblies": [...
  ],
  "TargetPlatformMinVersion": "10.0.17763.0",
  "ReferenceAssemblyPaths": [
  ],
  "BuildConfiguration": null,
  "ForceSharedStateShutdown": false,
  "DisableXbfGeneration": false,
  "DisableXbfLineInfo": false,
  "EnableXBindDiagnostics": false,
  "ClIncludeFiles": [
    {
      "DependentUpon": "",
      "FullPath": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\pch.h",
      "ItemSpec": "pch.h",
      "IsSystemReference": false,
      "IsNuGetReference": false,
      "IsStaticLibraryReference": false,
      "MSBuild_Link": "",
      "MSBuild_TargetPath": "",
      "MSBuild_XamlResourceMapName": "",
      "MSBuild_XamlComponentResourceLocation": ""
    },
    {
      "DependentUpon": "App.xaml",
      "FullPath": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\App.xaml.h",
      "ItemSpec": "App.xaml.h",
      "IsSystemReference": false,
      "IsNuGetReference": false,
      "IsStaticLibraryReference": false,
      "MSBuild_Link": "",
      "MSBuild_TargetPath": "",
      "MSBuild_XamlResourceMapName": "",
      "MSBuild_XamlComponentResourceLocation": ""
    },
    {
      "DependentUpon": "MainWindow.xaml",
      "FullPath": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\MainWindow.xaml.h",
      "ItemSpec": "MainWindow.xaml.h",
      "IsSystemReference": false,
      "IsNuGetReference": false,
      "IsStaticLibraryReference": false,
      "MSBuild_Link": "",
      "MSBuild_TargetPath": "",
      "MSBuild_XamlResourceMapName": "",
      "MSBuild_XamlComponentResourceLocation": ""
    }
  ],
  "CIncludeDirectories": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\was\\include\\;C:\\Users\\Darran\\source\\repos\\xcmanualtest\\wil\\include\\;C:\\Users\\Darran\\source\\repos\\xcmanualtest\\;generated\\;output\\;",
  "XamlApplications": [
    {
      "DependentUpon": "",
      "FullPath": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\App.xaml",
      "ItemSpec": "App.xaml",
      "IsSystemReference": false,
      "IsNuGetReference": false,
      "IsStaticLibraryReference": false,
      "MSBuild_Link": "",
      "MSBuild_TargetPath": "",
      "MSBuild_XamlResourceMapName": "",
      "MSBuild_XamlComponentResourceLocation": ""
    }
  ],
  "XamlPages": [
    {
      "DependentUpon": "",
      "FullPath": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\MainWindow.xaml",
      "ItemSpec": "MainWindow.xaml",
      "IsSystemReference": false,
      "IsNuGetReference": false,
      "IsStaticLibraryReference": false,
      "MSBuild_Link": "",
      "MSBuild_TargetPath": "",
      "MSBuild_XamlResourceMapName": "",
      "MSBuild_XamlComponentResourceLocation": ""
    }
  ],
  "LocalAssembly": null,
  "SdkXamlPages": null,
  "ProjectName": "xcmanualtest",
  "IsPass1": true,
  "RootNamespace": "xcmanualtest",
  "OutputType": "exe",
  "PriIndexName": null,
  "CodeGenerationControlFlags": null,
  "FeatureControlFlags": "EnableXBindDiagnostics;EnableDefaultValidationContextGeneration;EnableWin32Codegen",
  "XamlFingerprint": true,
  "UseVCMetaManaged": true,
  "FingerprintIgnorePaths": [
    "D:\\Programs32\\Windows Kits\\10\\"
  ],
  "VCInstallDir": "D:\\Programs64\\Microsoft Visual Studio\\2022\\Community\\VC\\",
  "VCInstallPath32": "D:\\Programs64\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.37.32822\\bin\\Hostx64\\x86\\vcmeta.dll",
  "VCInstallPath64": "D:\\Programs64\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.37.32822\\bin\\Hostx64\\x64\\vcmeta.dll",
  "WindowsSdkPath": "D:\\Programs32\\Windows Kits\\10\\",
  "CompileMode": "RealBuildPass1",
  "SavedStateFile": "int\\XamlSaveStateFile.xml",
  "RootsLog": null,
  "SuppressWarnings": null,
  "GenXbfPath": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\was\\tools\\",
  "PrecompiledHeaderFile": "pch.h",
  "XamlResourceMapName": null,
  "XamlComponentResourceLocation": null,
  "XamlPlatform": null,
  "TargetFileName": null,
  "IgnoreSpecifiedTargetPlatformMinVersion": false
}

The reference assemblies have been stripped to save a lot of space. Pass 2 is:

{
  "ProjectPath": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\dummy.txt",
  "Language": "CppWinRT",
  "LanguageSourceExtension": ".cpp",
  "OutputPath": "generated\\",
  "ReferenceAssemblies": [...
  ],
  "TargetPlatformMinVersion": "10.0.17763.0",
  "ReferenceAssemblyPaths": [
  ],
  "BuildConfiguration": null,
  "ForceSharedStateShutdown": false,
  "DisableXbfGeneration": false,
  "DisableXbfLineInfo": false,
  "EnableXBindDiagnostics": false,
  "ClIncludeFiles": [
    {
      "DependentUpon": "",
      "FullPath": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\pch.h",
      "ItemSpec": "pch.h",
      "IsSystemReference": false,
      "IsNuGetReference": false,
      "IsStaticLibraryReference": false,
      "MSBuild_Link": "",
      "MSBuild_TargetPath": "",
      "MSBuild_XamlResourceMapName": "",
      "MSBuild_XamlComponentResourceLocation": ""
    },
    {
      "DependentUpon": "App.xaml",
      "FullPath": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\App.xaml.h",
      "ItemSpec": "App.xaml.h",
      "IsSystemReference": false,
      "IsNuGetReference": false,
      "IsStaticLibraryReference": false,
      "MSBuild_Link": "",
      "MSBuild_TargetPath": "",
      "MSBuild_XamlResourceMapName": "",
      "MSBuild_XamlComponentResourceLocation": ""
    },
    {
      "DependentUpon": "MainWindow.xaml",
      "FullPath": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\MainWindow.xaml.h",
      "ItemSpec": "MainWindow.xaml.h",
      "IsSystemReference": false,
      "IsNuGetReference": false,
      "IsStaticLibraryReference": false,
      "MSBuild_Link": "",
      "MSBuild_TargetPath": "",
      "MSBuild_XamlResourceMapName": "",
      "MSBuild_XamlComponentResourceLocation": ""
    }
  ],
  "CIncludeDirectories": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\was\\include\\;C:\\Users\\Darran\\source\\repos\\xcmanualtest\\wil\\include\\;C:\\Users\\Darran\\source\\repos\\xcmanualtest\\;generated\\;output\\;",
  "XamlApplications": [
    {
      "DependentUpon": "",
      "FullPath": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\App.xaml",
      "ItemSpec": "App.xaml",
      "IsSystemReference": false,
      "IsNuGetReference": false,
      "IsStaticLibraryReference": false,
      "MSBuild_Link": "",
      "MSBuild_TargetPath": "",
      "MSBuild_XamlResourceMapName": "",
      "MSBuild_XamlComponentResourceLocation": ""
    }
  ],
  "XamlPages": [
    {
      "DependentUpon": "",
      "FullPath": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\MainWindow.xaml",
      "ItemSpec": "MainWindow.xaml",
      "IsSystemReference": false,
      "IsNuGetReference": false,
      "IsStaticLibraryReference": false,
      "MSBuild_Link": "",
      "MSBuild_TargetPath": "",
      "MSBuild_XamlResourceMapName": "",
      "MSBuild_XamlComponentResourceLocation": ""
    }
  ],
  "LocalAssembly": [
    {
      "DependentUpon": "",
      "FullPath": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\output\\xcmanualtest.winmd",
      "ItemSpec": "output\\xcmanualtest.winmd",
      "IsSystemReference": false,
      "IsNuGetReference": false,
      "IsStaticLibraryReference": false,
      "MSBuild_Link": "",
      "MSBuild_TargetPath": "",
      "MSBuild_XamlResourceMapName": "",
      "MSBuild_XamlComponentResourceLocation": ""
    }
  ],
  "SdkXamlPages": null,
  "ProjectName": "xcmanualtest",
  "IsPass1": false,
  "RootNamespace": "xcmanualtest",
  "OutputType": "exe",
  "PriIndexName": null,
  "CodeGenerationControlFlags": null,
  "FeatureControlFlags": "EnableXBindDiagnostics;EnableDefaultValidationContextGeneration;EnableWin32Codegen",
  "XamlFingerprint": true,
  "UseVCMetaManaged": true,
  "FingerprintIgnorePaths": [
    "D:\\Programs32\\Windows Kits\\10\\"
  ],
  "VCInstallDir": "D:\\Programs64\\Microsoft Visual Studio\\2022\\Community\\VC\\",
  "VCInstallPath32": "D:\\Programs64\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.37.32822\\bin\\Hostx64\\x86\\vcmeta.dll",
  "VCInstallPath64": "D:\\Programs64\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.37.32822\\bin\\Hostx64\\x64\\vcmeta.dll",
  "WindowsSdkPath": "D:\\Programs32\\Windows Kits\\10\\",
  "CompileMode": "RealBuildPass2",
  "SavedStateFile": "int\\XamlSaveStateFile.xml",
  "RootsLog": null,
  "SuppressWarnings": null,
  "GenXbfPath": "C:\\Users\\Darran\\source\\repos\\xcmanualtest\\was\\tools\\",
  "PrecompiledHeaderFile": "pch.h",
  "XamlResourceMapName": null,
  "XamlComponentResourceLocation": null,
  "XamlPlatform": null,
  "TargetFileName": null,
  "IgnoreSpecifiedTargetPlatformMinVersion": false
}

There is very little difference between them, but there are some differences. First, CompileMode is set to RealBuildPass1 for pass 1 and RealBuildPass2 for pass 2. There is also the IsPass1 property that is true for pass 1 and false for pass2. LocalAssembly is null for pass 1, but must be a reference to your application's .winmd for pass 2. Other things to note are as follows. ProjectPath must be a path to a project file. For Visual C++ projects, this is the path to the .vcxproj file. When I did this investigation, the Xaml compiler didn't read the file, but just used the containing directory as the root path for any relative paths. This is why I used an empty text file. This is also what OutputPath and the ItemSpec and DependentUpon properties are relative to. ReferenceAssemblyPaths must be empty, but it has to be an empty array and not null. CIncludeDirectories are any directories in addition to the INCLUDE environment variable that will be used. Finally, the GenXbfPath doesn't include the platform, the Xaml compiler will detect the one that it needs automatically.

These should be the most important points, but if you want, I can double check what I have with the latest version of the tools to be sure that things still work. I can get things packaged up into a small .zip file with more instructions. I will remind you at this point that this is unsupported, so do not take this as Microsoft approved.

res2k commented 1 month ago

Right, this will be a long post.

...which looks extremely helpful! Lots of things I would otherwise have to have puzzled out myself. I'll try to make good use of your info dump. (Fingers crossed I don't make too much mistakes... after all, won't be able to get decent error output right now 😅)

DarranRowe commented 1 month ago

Well, I double checked with the Windows App SDK 1.6 and the latest VC, and it still works. The only real change I had to do was to change where the WebView2 metadata was coming from.