Build time of UWP C# projects takes too long

This issue has been moved from a ticket on Developer Community.

Hi,

I'm using the latest Visual Studio 2019 (16.2.1) with everything update to the latest version. My machine is a I9-9900K 3.60Ghz CPU with 16 logical processors. My disk drive is a very fast Western Digital WD Black SN750, 1 To, M.2 (Type 2280) SSD. All programs are on this SSD drives.

My test is the following:

1) create a brand new C# Universal Windows app, and build it. Everything works fine. 2) create a brand new C++/WinRT Universal Windows app, and build it. Everything works fine.

I'm always using Debug configuration, not using native toolchain.

Now, when I change something in the C# project and I build again, I noticed it always take like 4 seconds. When I change something in the C++/WinRT C# project it takes like 2 seconds, 2 seconds less.

I think there is an issue with the C# build tool chain as when I look at the build (for example using this tool: https://github.com/KirillOsenkov/MSBuildStructuredLog) I see the MarkupCompilePass1 / CompileXaml MSBuild task takes more than 2 seconds for the C# project and almost nothing for the C++/WinRt project although both are using the same .xaml files.

See the first screenshot that demonstrates 2 seconds difference between C# and C++/WinRT.

To go a bit further, I have spied on MsBuild using procmon from sysinternals (with all filters File, Registry, Network, Threads, etc.), and whatever C# UWP application I try, I always see around 2 seconds are lost somewhere in the .XAML compilation, which is consistent with my impressions and my other findings. I have attached the result in another screenshot. I also have a .CSV that I can send if needed.

This is really a pain to wait for 5 second for a blank project and of course it doesn't get better when the project gets bigger. Is there anything that can be done. I'm pretty sure there's some problem with the Xaml compiler. Simon.

Original Comments

Visual Studio Feedback System on 8/13/2019, 10:55 PM:

We have directed your feedback to the appropriate engineering team for further evaluation. The team will review the feedback and notify you about the next steps.

Svet Bonev [MSFT] on 8/15/2019, 11:33 AM:

Hello! Thank you for your feedback!

We have addressed a performance issue with XAML compilation in the latest Preview of Visual Studio 2019 (Visual Studio 2019 version 16.3 Preview 2).
Would you be able to install that version and check if your scenario has improved?

Thanks!

Simon Mourier on 8/16/2019, 00:09 AM:

Hi,

I've tested the new 16.2.2 and the 16.3 Preview 2 but it doesn't change anything with regards to this MarkupCompilePass1 / CompileXaml time MsBuild target that takes more than 2 seconds for C# projects.

Simon.

David Beavon on 8/18/2019, 03:02 PM:

I don't want to muddy the water here... hopefully the following is helpful. I've been following UWP/xaml project compile performance in c# projects for a long time. It is quite a lot slower than WPF/xaml for starters. But even if we set that information aside, I had noticed that the compile performance became dramatically worse for UWP/xaml after changing "TargetPlatformMinVersion" to something greater than 10.0.15063.0. I think the performance drop coincided with the fall creators update of 2017 (and the added support for .net standard 2.0). In order to confirm this observation, you need to go back to using visual studio 2017. (They won't allow us to target the platform 10.0.15063 anymore in vs 2019). More info here:

https://developercommunity.visualstudio.com/content/problem/376045/compilation-of-uwp-xaml-project-is-slower-when-tar.html

Essentially my observation was the same as yours. A new c# project with an empty user control takes about two seconds longer than it should. Compile time of c# projects quickly grows to 10 seconds (my personal limit) after just one or two dozen user controls. I opened a premier support case with MS once I realized that I could prove a regression bug in visual studio 2017 (because the behavior of the builds was so much slower when targeting platforms above 10.0.15063 ).

To their credit, Microsoft added a statement to my developercommunity report to try to explain the regression: "This performance issue you that identified was introduced in an effort to address build consistency issues, especially in incremental compiler situations. With that in mind, we are looking at ways to make this better so the compile time impact is less."

My interpretation is that the current compilation process is "playing it safe"... it is probably doing redundant work on the c# side of things. I really think UWP/xaml compile performance in c# deserves a LOT more discussion, and perhaps a blog post or two. I believe I've heard Microsoft say that UWP/xaml is ready for line-of-business apps but that can't happen if developer productivity (inner loop) is multiple times slower than for a similar WPF app (or winforms). I suspect this is one of the top reasons why the UWP/xaml platform is not catching up very quickly to WPF. I really like UWP aside from this complaint and a few other things (lack of a ribbon control, outlook bar, adorner layer, etc).

David Beavon on 8/18/2019, 03:45 PM:

Simon, you may also want to start a new discussion on stackoverflow with a specific example of your problem. Your comparison between .Net and C++ is something that I hadn't seen before, and it may be very valuable to others who do UWP development (especially those who have the luxury to pick between c# and c++ for any given project).

If I had to guess, there is a very high likelihood that this problem reported here in the visualstudio developercommunity will be soon be closed; but the underlying compilation performance in UWP/xaml will remain. This is not a new problem and it will probably take a long time before we see improvements. I don't get a sense of urgency in this area (at least it isn't urgent compared to some other initiatives like migrating WPF/winforms to Core. ... but that initiative might end up working in favor of UWP one day as well....)

If it helps, here is a place in stackoverflow where I posted my findings about UWP/xaml compile performance:

https://stackoverflow.com/questions/52919086/speeding-up-compilation-in-uwp

Simon Mourier on 8/18/2019, 10:52 PM:

Hi David,

Thanks for your comments. I have seen the discussions on SO but it didn't help much (no offense :-). UWP compilation is, overall, a huge loss of time (not even mentioning the Deploy phase which adds its share). I think only Microsoft can fix this, and beyond all other UWP issues, this one is a showstopper for me.

If you use procmon on msbuild.exe during a compilation it shows crazy things (scanning of full directories, opening .zip files), every single time. The compilation of a hello world project shows 30,000 file access in procmon, for a total of 5 seconds (again w/o deploy) on a 16 logical core machine with a very fast SSD disk. A Winforms hello world project takes ... 0 second.

I've been writing code with Microsoft technologies for 30 years and this is very disappointing.

But beyond that, and beyond all other UWP issues (I'm about to abandon my project actually) , I'm pretty sure there is a very specific issue with the C# project case where time seems suspended during almost 2 seconds.

David Beavon on 8/19/2019, 05:49 AM:

Yes, I agree that the performance was a disappointment and I also put my UWP/XAML/c# projects on hold as well, for the same reasons. I've spent the last year watching things improve. It is a very slow process. Sometimes I will find a demo on Channel 9 where a Microsoft developer is showing a UWP/xaml solution in c#. The developer will eventually need to compile a relatively small csproj during the presentation, but they have to find a way to fill in the awkward gap of time (like 10 seconds) while the compiler is processing a couple of very small xaml files. At least it makes me feel better that I am not the only one affected.

It was interesting to hear from you that the c++ side behaves in a more "acceptable" way when working with UWP/xaml. Based on that, it is possible that c# (.net) developer is not (yet) the main target audience for the UWP/xaml platform. Microsoft may not (yet) be investing heavily in the developer experience on the c# side of things yet. This may change after WPF goes to Core, and WINUI 3.0 is rolled, out and Microsoft tries to start encouraging the use of their xaml islands.

I should point out that I was eventually able to get a single control library to compile to a *shared* output directory, and start debugging immediately (so you don't have to wait to build *all* the remaining projects that are *dependent* on that library). It was a bit tricky to get that working, but it is worth doing if you want to avoid excessive build times.

Also I should point out that Microsoft created "XAML Studio" earlier this year to allow UWP/xaml to be created and modified without waiting on VS builds. But this seems like more of a workaround than a solution.

Thanks for the pointer to MSBuildStructuredLog. That is something that I've not come across but it seems very helpful. I was just thinking the other day how it might be nice to see a "compiler plan" diagram with estimated costs, similar to how SQL Server can generate "query plan" diagrams.

Daniel Z [MSFT] on 8/26/2019, 04:27 PM:

Thanks all for reporting these performance issues! We were not aware of how wide spread they were until just recently. We've just started looking into fixing them.

We already patched the "vcmeta.dll" issue and shipped it with Visual Studio 16.3 and 15.9.15. This should fix a problem where the Xaml Compiler was looking for vcmeta.dll in a different folder than where it actually was. The dll was moved in Visual Studio 2017 without detecting its dependency for Xaml compilation.
Two other fixes, one to start supporting .Net standard and another one to fix inconsistency in incremental builds, also regressed Xaml compilation times. We're working on addressing those in our upcoming WinUI 3.0 release that we announced at //build.
We're also looking into other performance optimizations we could do in that same timeframe to make Xaml compilation faster.
We want to assure you that we're taking this matter very seriously and plan to address it in our next release.

David Beavon on 8/28/2019, 11:32 AM:

It might be nice if you could help set expectations for UWP/xaml compilation times. It seems like the language is quite similar to WPF/xaml so I would expect similar compile performance. If anything, it seems to me that UWP/xaml is quite a bit more (1) primitive and (2) native. And together those should result in even faster compilation than WPF.

It would be nice if we had a "baseline" to use when measuring future compiler improvements. For example, many developers use similar hardware: SSD and ~4GHz CPU. From one developer to another, a difference in compiler performance is not very likely due to different hardware, but rather due to misconfiguration of csproj, or due to the misconfiguration of VS, or to the configuration of something on the machine (eg. defender).

Going forward, compiler performance would (hopefully) improve based on the updates to Visual Studio. That is where a "baseline" would become relevant. Assuming no misconfigurations, you can tell us how much of an improvement a developer can expect to see in compiler times for each update.

Baseline measurements could be done with reference to a sample project, eg. the time it takes to compile a certain version of the vanarsdel UWP app, eg. https://github.com/microsoft/vanarsdel

If it helps I had taken measurements of the time it takes to compile a variety of projects in github (community toolkit, inventory sample, vanarsdel sample) These measurements were taken about a year ago in VS 2017 and I would bet I'd get similar results today in VS 2019 as well. See

https://stackoverflow.com/questions/52919086/speeding-up-compilation-in-uwp

Daniel Z [MSFT] on 8/28/2019, 00:57 PM:

The WPF and UWP Xaml Compilers deal with very similar languages, they both compile Xaml to a binary format (XBF or BAML), both generate code in 2 passes, and both take reference assemblies as input. The UWP compiler generates code for 4 languages, supports x:Bind (which generates code orders of magnitude larger and more complex than regular Xaml compilations), and generally has a few other features that the WPF compiler does not. I'm sure the oposite can be said about the PBT. What I'm trying to say is they're similar and the features they uniquely support could benefit the other one, too. I wish they were one, not two pieces of software, but they're not, and to merge them would be a monumental effort and won't guarantee that UWP compilation will magically become faster without a good architecture to support its more complex features. We are considering all the feedback here and will improve whet we believe are the performance bottlenecks in currentt UWP compilation, primarly reflection-based type lookup, Xaml parsing, and change detection. I agree that having baselines is good and we do have them: they're the time it took to build a scenario with the previous released build. I agree that, while adding new features, aiming to not exceed these baselines and hopefully improve upon them in newer versions is what we should be aiming for. Baselines are not enough though. We're planning to use several scenarios to measure perf against those baselines, such as:
- full build of a standard project (for all languages),
- incremental build when code changes,
- incremental build when headers change,
- incremental build when small impact Xaml changes (Xaml with no impact on generated code),
- incremental build when high impact Xaml changes such as x:Bind, x:Name, events). The Xaml Compilation time alone should also not be looked at in isolation. Based on Xaml Compilation, the generated code influences how the language compiler behaves and its performance.
Speaking of that, we've already introduced and still tinking about other improvements in that spece, such as using a pure reflection-based metadata provider in debug builds, or using incrementally focused C++ headers and type activation.

We'll report back numbers on all those scenarios when we ship Win UI 3.0. I use Xaml a lot too, and man, I hate wasting time waiting for a build as mush as anybody else. :-) #### David Beavon on 8/28/2019, 02:10 PM:

Sounds like you are on top of this. I appreciate the open communication. There may be tips/tricks/workarounds that we are overlooking, and those would be helpful to hear about as well. For example, compiling-and-debugging a single control assembly (by sending it to a shared output directory) was particularly tricky to do with a UWP solution. I spent many days working with MSDN tech support to find an obscure property ("TargetedSDKConfiguration") in the csproj that was disallowing the streamlining of my compile-and-debug workflows. I still won't find much information about this property if I googled it right now.

Given that you already have scenarios for measuring performance, it is likely that Microsoft was already aware of the dramatic drop in performance for any UWP SDK that followed 10.0.15063.0. It would have been nice if there was a line-of-communication for letting us know about that. I would suspect that you have more open lines of communication with your third party control vendors (who seem to be pretty far along with their UWP offerings). Or maybe they just call the MSDN tech support team, the same way that I do?

I don't mean to sound negative. I still think UWP/xaml is pretty great... although it moves forward quite slowly (some bloggers like Paul Thurrott are trying to convince us that it is no longer alive). Of course he should know better. I suspect things will improve with WINUI 3.0 when the pace of change is no longer bound to Windows platform updates.

It seems apparent that there is a large burden on UWP/xaml to support xbox, hololens, surfacehub, in addition to Windows 10. If the ongoing improvements to UWP/xaml must be made in unmanaged code, and have to be coordinated across many hardware platforms, then it is very obvious why the pace of change is slower (compared to WPF/xaml which is predominantly managed code and it targets only the Windows desktop).

#### Daniel Z [MSFT] on 8/28/2019, 03:10 PM:

Thanks for understanding! Unfortunately, we were not testing those scenarios I mentioned on a performance bench to be able to catch this regression, so we were not aware just how much we regressed or how wide-spread the problem was. We knew about the vcmeta.dll and some of the other individual problems in isolation, but never connected them together to see their overall impact until just recently. I'm sure that will change with WinUI 3 where we're expecting the pace of changes and delivery to accelerate. WinUI 3 is shaping up to be awesome! :-) P.S I don't know anything at all about "TargetedSDKConfiguration"

#### Svet Bonev [MSFT] on 8/30/2019, 01:11 PM:

Hi everyone!
Thanks to @Daniel Z [MSFT]'s input, I believe we have a much better understanding of what is happening with UWP XAML build performance and the road map of the team.

It is unfortunate that the vcmeta.dll fix that we shipped recently will not address the full breadth of the problem, but I am hopeful that the WinUI 3 decoupling strategy would allow fixes to ship with much higher velocity.

Would you mind if I closed this feedback ticket? I feel that we have exhausted our options for a fix on the Visual Studio side.

I would recommend submitting an issue at https://github.com/microsoft/microsoft-ui-xaml where WinUI work is happening.

Thanks,

Svet Bonev

Visual Studio Client Tools

#### Simon Mourier on 8/30/2019, 01:46 PM:

Well, nothing is fixed, but you can close it if you feel this is the right thing to do. I stand on my (pretty bad) impression till further notice.

Thanks

#### David Beavon on 8/30/2019, 02:24 PM:

Simon, I stand by my suggestion to put your research and questions on stackoverflow. ;-) The same thing happened to me when I reported this previously.

I may be cynical, but I suspect it is all about locking the votes. It is an inconvenient problem if the votes grow out of control. At the end of the day there is only one vote that matters!

#### Svet Bonev [MSFT] on 9/3/2019, 08:58 AM:

Let me address @Simon Mourier and @David Beavon's comments - this is about who is ultimately going to implement a fix for the issue.

Visual Studio is a conglomerate of multiple toolchains that are developed by their respective owners and some of these owners are teams outside the Visual Studio organization. The XAML compiler team owns the particular piece of technology that we are discussing here and that team is not part of the Visual Studio organization. They use https://github.com/microsoft/microsoft-ui-xaml to track issues which is why I am recommending to submit your findings and questions there.

In short, keeping the ticket open here will not help address the issue directly as any input on this thread will need to be transferred over to the github project creating an unnecessary level of indirection.

#### Simon Mourier on 9/3/2019, 09:29 AM:

Hi Svet,

Thanks. I perfectly understand what you mean, and it's obvious when you look at the thousands of file actions that are generated by a UWP compilation that responsibilities are completely fragmented, but from my perspective, understanding who's responsible from what when building a UWP app (which is not only made of XAML files) is totally beyond my reach, technically and organizationally.

In this particular case, I had (before you mentioned it) absolutely no idea that this github repository even existed. When I consult it, I don't see anywhere a mention that this is a place to go for XAML compilation issues, cross-project type.

For me, this site here "https://developercommunity.visualstudio.com" was the one-stop place to ask questions and report issues. I though these issues would be sorted and triaged and routed nad linked to the proper "Microsoft Development Area" (whatever that mean) place, internal or external. It seems I was wrong since from what you say, it's up to *us* (we, non owner of this site) to determine who to send the request to, when we face this kind of problem (and then follow it?). I can do (and I sometimes do when I'm blocked) that when I'm pretty sure who to target, but certainly not in the general case.

I was not suggesting anything, nor asking any question, I was reporting a real, reproductible, problem. Anyway, I understand noone is concerned on this "https://developercommunity.visualstudio.com" site about XAML compilation issues, so, yes sure, close the ticket as I said earlier, UWP is not a concern to me anymore (you can guess why).

#### Svet Bonev [MSFT] on 9/3/2019, 03:43 PM:

We plan to address the performance issues we identified with the recent versions of the UWP XAML Compiler with the release of WinUI 3. Any additional feedback is welcomed and appreciated at https://github.com/microsoft/microsoft-ui-xaml/.

--- ### Original Solutions (no solutions)

microsoft / microsoft-ui-xaml