Proposal: Add alternate tasks to compile XBF files

[wjk]

Proposal: Add alternate tasks to compile and package XBF files

Summary

One of the most important features in building a WinUI Desktop app is proper compilation of XAML files into XBF, and packaging them into PRI files. This process is convoluted and buggy when using the first-party UWP targets. I aim to fix this.

Rationale

• The UWP targets shipped with Visual Studio are designed to build and package full UWP apps that run in the AppContainer. Attempting to use them to build a standalone DLL that can be referenced from a desktop app is a complicated process, and I have not been able to get it fully working. Attempting to place the resulting app into a wapproj for Desktop Bridge packaging introduces even more problems.
• The Windows Community toolkit has attempted to make it possible to use WinUI XAML in a non-UWP project in https://github.com/windows-toolkit/Microsoft.Toolkit.Win32/pull/147, but this implementation does not function. My main complaint with that PR is that it does not package the XBF files into a PRI file like is done when building a full UWP app or library. My goal is to have one PRI file for each DLL named after that DLL, plus (if needed) a resources.pri with resources for the main app. No amount of modification to the targets in the above PR causes this to happen. The XBF files are copied as-is into the bin directory, and removing them causes the app to crash. This is because the generated .g.i.cs files are hardcoded to look for them directly in the filesystem.
• Additionally, the tasks that perform the actual XAML compilation are (a) proprietary, and (b) do not invoke an EXE. Due to this second fact, it is impossible to compile XBF files any way other than using the UWP toolchain, which is IMO undesirable for the above reasons. However, there is the Windows.UI.Xaml.Markup.XamlBinaryWriter class, which does something similar to the XBF compilation tasks. However, those tasks use a different API (genxbf.dll). Presumably this is to ensure 100% backwards compatibility when targeting downlevel versions of Windows 10, but there could be other reasons that I do not know about.

Scope

Capability	Priority
Create open-source MSBuild tasks (in this repo) that compile XAML to XBF files	Must
Create open-source MSBuild tasks (in this repo) that invoke makepri.exe	Must
Create open-source MSBuild tasks (in this repo) that generate the .g.i.cs code-behind files	Must
The tasks will support C# apps	Must
The tasks will support C++ apps	Must
The tasks will support packaging XBF files into PRI files, and loading them at runtime directly from there	Should

Open Questions

• Will the open-sourcing of the Windows code as part of WinUI 3 include the code for XamlBinaryWriter? I hope it does, but there has been no confirmation from MS one way or the other as to the scope of the planned code drop.
• The Microsoft.UI.Xaml NuGet package, when referenced from a .NET Core/.NET 5 project, is not copied into the root directory properly. The PRI file from the NuGet package is copied into directory with the EXE, but the DLL is copied into the runtimes/win10-x64 subdirectory. This breaks use of Registration-Free WinRT manifest code. I would appreciate it if this could be fixed, although it may be done in a different PR.
• I would very much appreciate it if the Visual Studio project system could be updated to treat XAML files in a project that references these tasks as UWP XAML files instead of WPF XAML files. (Currently, when I try to use https://github.com/windows-toolkit/Microsoft.Toolkit.Win32/pull/147, VS insists the XAML files are of the WPF flavor, despite project capabilities being included that specify otherwise. This results in types not being resolved properly in the designer.) This may well be outside the scope of this repository, however.

[huoyaoyuan]

The {x:Bind} compiler is even more buggy than xaml that I don't have patience to open separated issue for each bug I found. Please modernize both the experience and infrastructure like Blazor.

[stevenbrix]

@wjk @huoyaoyuan thank you for the awesome feedback! I couldn't agree more, this needs to be addressed. I just created the area-Tooling label so that we can track tooling related improvements.

I don't have anything concrete to share right now, but I am working on a proposal that will move us in the right direction. Once we have something ready to share we can start to get everyone's input. If you have (or see) any tooling related issues please tag or tweet me.

/cc @pag3

[sylveon]

"The tasks will support C++ apps" needs to be a must - C++ developers should not be forced to the inferior experience, especially considering it's the same XAML compiler ultimately.

[pjmlp]

"The tasks will support C++ apps" needs to be a must - C++ developers should not be forced to the inferior experience, especially considering it's the same XAML compiler ultimately.

This includes a proper experience for the IDL files required to bind into XAML, with syntax highlighting and code completion, and no manual copying of generated files.

[wjk]

Updated OP. The only real reason I put that as a "should" was to indicate that (to me) it was of lower priority, and that I was OK if only the C# version shipped first. Now I see I was in error. I absolutely agree with all of the complaints about the C++/XAML experience.

[sylveon]

@pjmlp IDL is the responsibility of cppwinrt, this issue is about the XAML compiler. Currently, outside C++-specific codegen bugs, the XAML compiler experience is on par with C#.

[pjmlp]

@sylveon After killing the productivity we had with C++/CX the responsibility to fix it seems to belong always to someone else.

From C++/WinRT team apparently ISO C++ is to blame and we should now just sit and wait for the day that ISO C++ might support C++/CX features, nice service.

[stevenbrix]

@wjk we have started work on a xaml compiler exe. If you are using Preview2, you can set UseXamlCompilerExecutable in your project file, this will be used instead of the MSBuild task dll. It's still early on (and why we haven't made it the default yet), so we'd appreciate getting your feedback on it. While the code isn't open source yet, you can probably look through our props/targets in the nuget package to figure out how it's used :)

@sylveon and @pjmlp I think I've said this on a few other threads, but we don't want us to wait for metaclasses support to drop IDL. We can and do want to create a better experience today, we just aren't quite ready to start that work today.

/cc @RealTommyKlein, @Scottj1s, @kennykerr, @JeanRoca

[pjmlp]

@stevenbrix Thanks for confirming that the productivity loss due to withdrawing C++/CX is taking care of.

If you allow me just a little bit of feedback, C++/CX is almost perfect, finally Microsoft seemed to be getting what it meant to have a C++ RAD offering, which while not yet C++ Builder like, it is almost like there.

Instead of improving what was still missing, versus the .NET experience for UWP applications (C++/CX still required a bit more boilerplate), everything was dropped in name of a ISO C++ compatibility that doesn't matter, when the applications are anyway not portable to anything other than Windows/UWP environment.

Instead we got a developer experience that feels worse than just doing plain old MFC applications, and it is good that the message is no longer that we sit and wait for ISO C++ to someday catch up with C++/CX capabilities from 2012.

Now I have something to look for, and will correct my message in regards to C++/WinRT current state.

It would be nice that whatever is being planned eventually gets announced in some Windows or Visual Studio related blog, so that all that feel the same way see that "developers, developers, developers" also includes WinUI/C++ tooling.

[stevenbrix]

@pjmlp we're so early in discussion, it's mostly just hallway talk at this point (if that were a thing anymore). Once there is something to share, we will. But I wouldn't expect anything in 2020.

I'm glad you are happy with C++/Cx, but for everyone who feels the way you do, someone says they hate it.

[riverar]

Bad news: Project Reunion/WinUI 3 no longer supports the xaml compiler executable.

.nuget\packages\microsoft.projectreunion.winui\0.8.0-preview\buildTransitive\Microsoft.UI.Xaml.Markup.Compiler.interop.targets(458,9): error : The executable Xaml compiler is no longer supported. Please set UseXamlCompilerExecutable=false.

[sylveon]

What a shame :(

I had hopes XAML would finally stop being bound to MSBuild.

[wjk]

To be fair, it is possible to write your own XAML compiler using the XamlBinaryWriter class. This API has existed as far back as Windows 8, believe it or not, and I have finally been able to confirm that it does the exact same thing as genxbf.dll, just with a different API surface. However, since you can (currently) only access WinUI in an MSIX-packaged app, creating a command-line build tool that calls this API is difficult. But it’s most certainly not impossible.

[RealTommyKlein]

Hi folks - apologies the executable compiler removal wasn't in the Reunion 0.8 notes. We've removed the compiler executable from Project Reunion since its current form is still tied to MSBuild (the json input file it requires is produced from an MSBuild task, and contains type information only producible from a .NET Framework context). For some background on that choice - the WinUI/Reunion Xaml compiler ships as an MSBuild task built against the .NET Framework (as you're all aware :smile: ), and we had planned to integrate it with the .NET CLI as well. However, the .NET CLI wouldn't be able to load a .NET Framework library, and converting the Xaml compiler to .NET Standard or updating the source code to dual-build for both .NET Framework/.NET Core were prohibitively expensive. So we created the .NET Framework executable Xaml compiler that shipped in earlier versions of WinUI 3/Reunion, as a low-cost way of running the .NET Framework Xaml compiler in other contexts like the .NET CLI, albeit still with MSBuild dependencies to generate the input JSON file.

We're aware of the demand for a compiler executable that is truly separate from MSBuild, but did not think the compiler executable that shipped up until Reunion 0.8 met that bar, nor that it provided value to developers in its current form. We can bring it back in its pre-0.8 form, but would like to better understand its current use case from developers leveraging it (@riverar - your insight would be greatly appreciated!). We also can't make any promises when a truly MSBuild-independent executable compiler would ship, but not in the near future (would happen after Reunion 1.0). Thanks!