Creating UWP AppxBundle works in VS Store Wizard but fails in msbuild

[martinsuchan]

We're developing UWP app - main app is built using C#/XAML plus it's referencing several our C++ libs. Building this app for Store works when using the Store -> Create App Packages Wizard in Visual Studio 2017.

The problem is that we're unable to build our app into AppxBundle using just PowerShell and msbuild. We'd like to be able to build our app automatically on our build/CI server. We've read thoroughly this article but with no success: https://docs.microsoft.com/en-us/windows/uwp/packaging/auto-build-package-uwp-apps

The build fails because msbuild is ignoring custom OutDir properties when building the second 'arm' package part for the AppxBundle.

Here is a minimal zip repro for our problem: https://1drv.ms/u/s!AlURSa6JiyiVo5FJuYVe4v0k7FSjxQ

If you open it VS2017 and try to build it using the Store -> Create App Packages wizard, it works as expected.

If you try to run the build.ps1 script, it fails. Note that path to nuget.exe is expected in the PATH env var. So our question is, how to replicate in PowerShell the identical build process that is used in Visual Studio Create App Packages Wizard? If the project can be build in VS, it should be possible to build it in PowerShell as well.

As far as I know the UapAppxPackageBuildMode=StoreUpload msbuild mode should work exactly like the VS Store Wizard, but it is behaving differently now.

[karelz]

@vitek-karas @MattWhilden how best to route this issue? It feel more related to UWP targets than to msbuild itself.

[vitek-karas]

I've sent this internally to a team in VS to take look...

[vitek-karas]

@diverdan92 and his team will look into this.

[martinsuchan]

Just to summarize, the goal is to have a possibility to build UWP .appxupload package from PowerShell in the exactly same way it is created in the VS Store Wizard, ideally just by providing the same data (project, configuration, platform). Right now msbuild uses different build workflow when building for Store and it fails for our project.

[mirkomuggli]

There's one more required parameter. For non-x86 builds the AppxBundlePlatforms value must be passed. In this case: AppxBundlePlatforms="ARM" (the quotes around ARM are important).

What was happening here is that the main package would build with ARM, but dependent packages build using the default platform (this is by design). In the end, this will end up dropping Utils.lib in the x86 path while the main project continues to try to build for ARM and winds up searching for the dependency in the wrong place.

My commandline looked like this: msbuild MainApp.sln /p:UapAppxPackageBuildMode=StoreUpload;AppxBundlePlatforms="ARM"

[martinsuchan]

@mirkomuggli Hi, as you can see in the repro I provided above, I pass the proper AppxBundlePlatforms parameter and it still does not work:

msbuild $sln /p:Configuration=$config /p:Platform=x86 /p:AppxBundlePlatforms="x86|arm" /p:AppxBundle=Always /p:UapAppxPackageBuildMode=StoreUpload

Also note that the goal here is to build AppxBundle with both x86 and ARM package - this does not work. If I try to build AppxBundle with just ARM or just x86, it works as expected.

[mirkomuggli]

Very curious. The commandline you shared successfully builds an Arm & x86 .appxUpload with the test app you've provided.

If you can, please try this again in the latest VS Preview just to rule out any issues we've recently addressed.

Also, please run the same commandline with the following at the end. /v:diag > buildlog.log That'll create a diagnostic log and allow me to investigate further.

[martinsuchan]

Have you tried the actual sample in my first post? There is nothing wrong with the command, but it does not work with a specific project configuration we have, even thought this project configuration works in the Visual Studio Build for Store wizard.

[martinsuchan]

I've just tested the msbuild from latest Visual Studio Preview with my repro -> same results, same error.
Here's the zipped log you requested. buildlog.zip

[martinsuchan]

@mirkomuggli Hi Mirko, any update on this issue? Have you tried the minimal zip repro for this problem?

[mirkomuggli]

@martinsuchan Hi Martin. I was finally able to track down the issue and am working on a fix. The plan is to roll it out with 15.6. I'll let you know when it's available - in a preview release or otherwise.

[martinsuchan]

@mirkomuggli Hi, thanks for the update! It's great to hear you were able to reproduce it and it will be fixed.
As I mentioned, the goal is simple - we should be able to build any UWP solution (C#/VB/C++/cppwinrt) using the msbuild.exe in the exactly same way it is built in the "Create App Packages" wizard.
Looking forward to test it in VS 15.6 Preview!

[martinsuchan]

Note I've created tracking ticket on the Visual Studio Developer Community web.
Based on VS 15.6 Preview 1 changelog, it's not fixed yet.

[icodingforfood]

encounter the same issue

[martinsuchan]

Updated to Visual Studio 2017 15.6 Preview 5 and this issue is still not fixed. cc @mirkomuggli

[CZEMacLeod]

There also seems to be an issue here that when I pass in /p:Configuration="Release.UWP" to prevent it building my other platforms, this is ignored in subsequent the x64 and arm builds (it works for the first x86 build) and so it tries to build the iOS and Android projects which are selected in Release build and then fails. This is with the VSTS build task created by the UWP build template.

[CZEMacLeod]

For clarification, it appears that the Solution configuration passed in is ignored when building other platforms, and the project configuration is then used as the Solution configuration.

E.g. When calling MSBuild with the arguments /p:Configuration="XYZ" /p:Platform=x86 /p:AppxBundlePlatforms="x86|x64|arm" and the solution has the UWP project mapped in Configuration Manager with XYZ|x86 -> ABC|x86, it then runs inner builds of the solution with /p:Configuration="ABC" /p:Platform=x64 and /p:Configuration="ABC" /p:Platform=arm which may have different projects selected to build etc.

I worked around my issue for now by copying the project configuration ABC as XYZ for each platform in the UWP project and setting the solution configuration to XYZ|x86 -> XYZ|x86; unfortunately this causes some other unwanted behaviours because (in my case) the configuration is no longer Release but is Release.UWP.

[mirkomuggli]

The above is, essentially, correct. Because of the way we do multi-architecture builds from the command line the fix is proving exceedingly tricky. It IS on my plate (and keeping me awake at night) and it will get fixed but, unfortunately, I can't make any promises around timing at this point.

[CZEMacLeod]

@mirkomuggli Would a (temporary) solution be to have a property such as SolutionConfiguration which would be set to Configuration if it is not set in Microsoft.AppxPackage.Targets and then use that variable instead of Configuration in the various MSBuild tasks and other places? I would be happy with using something like /p:Configuration="XYZ" /p:SolutionConfiguration="XYZ" /p:Platform=x86 /p:AppxBundlePlatforms="x86|x64|arm". Although it seems to me that this property should be set in the solution 'metaproj'. I see in the template that there appears to already be something similar in place for AspNet

  <PropertyGroup Condition=" ('$(AspNetConfiguration)' == '') ">
    <AspNetConfiguration>$(Configuration)</AspNetConfiguration>
  </PropertyGroup>

Would it be possible to use

  <Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\SolutionFile\ImportBefore\*" Condition="'$(ImportByWildcardBeforeSolution)' != 'false' and exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\SolutionFile\ImportBefore')" />

to implement this at the solution level, although you would probably have to do something before build in order to add SolutionConfiguration to AdditionalProperties or modify CurrentSolutionConfigurationContents to add the configuration to the SolutionConfiguration XML

[CZEMacLeod]

@mirkomuggli I did a quick trial of the Importxxx* trick for a solution and my (not very elegant) files are attached. Place them in C:\Program Files (x86)\Microsoft Visual Studio\Preview\Professional\MSBuild\15.0\SolutionFile under ImportBefore and ImportAfter folders respectively (these folders may not exist) and you should see it work. In any project file or imported targets file you can then access both $(Configuration) and $(SolutionConfiguration) which may (or may not) be the same.

SolutionFile.zip

You could then add something like

<PropertyGroup>
  <AppxConfiguration Condition="'$(AppxConfiguration)'=='' and '$(SolutionConfiguration)' != ''">$(SolutionConfiguration)</AppxConfiguration>
  <AppxConfiguration Condition="'$(AppxConfiguration)'==''">$(Configuration)</AppxConfiguration>
</PropertyGroup>

to Microsoft.AppxPackage.Targets and swap all references to $(Configuration) to $(AppxConfiguration)

[mirkomuggli]

Hm. that should work. I'll dive into this early next week and report my findings. Thanks!

[mirkomuggli]

Just a scheduling update, I'm afraid: had a hectic week followed by one spent mostly bedridden. I have an idea for a real fix here, just need to put out some fires first.

[DominikMe]

I worked around this issue by passing in /p:BuildPlatform="$(BuildPlatform)" as another MSBuild argument in the VSTS Build task. BuildPlatform is a VSTS variable fixed to x64 that will not change for the dependent builds, while the Platform property gets changed by MSBuild for each build.

Then I added conditional Content tags in my UWP App project's csproj which compare $(BuildPlatform) with $(Platform) and either reference a path like \x64\release or x64\release\x86. The $(BuildPlatform) == '' checks are to leave the local VisualStudio "Create Package" build unmodified.

<Content Include="$(SolutionDir)MyCppProject\bin\$(Platform)\$(Configuration)\MyCpp.exe" Condition="('$(BuildPlatform)' == '') Or ('$(BuildPlatform)' == '$(Platform)')">
  <CopyToOutputDirectory>Always</CopyToOutputDirectory>
</Content>
<Content Include="$(SolutionDir)MyCppProject\bin\$(BuildPlatform)\$(Configuration)\$(Platform)\MyCpp.exe" Condition="('$(BuildPlatform)' != '') And ('$(BuildPlatform)' != '$(Platform)')">
  <CopyToOutputDirectory>Always</CopyToOutputDirectory>
</Content>

I have not submitted to the Store yet, but the created AppxBundle looks as expected and contains the native CPP assembly for each platform.

@mirkomuggli

[FurryX]

Hi @mirkomuggli , have you found time to solve this issue?

[mirkomuggli]

Potentially. This issue's been stumping a few folks on the team until someone stumbled on an old flag that alters the output folder structure due to a TeamBuild requirement from way back when.

The flag is: UseSubFolderForOutputDirDuringMultiPlatformBuild I just tried setting this to false in the quickbuild script included in the sample project, and the build succeeded.

Can you give that a try on your end? Would be awesome (and incredibly frustrating) if it's that easy.

[martinsuchan]

I can confirm setting this flag to 'false' fixed the problem for us.
It would be better though to have working behavior in msbuild without need to search for hidden flags, any chance to make the 'false' behavior the default one?

[mirkomuggli]

I've left more details on the Developer Community ticket, available here: https://developercommunity.visualstudio.com/content/problem/164983/creating-uwp-appxbundle-works-in-vs-store-wizard-b.html

[angelortiz1007]

In working to address issue #xxxx I found the following msbuild command sequence helped me build only what I wanted (x64 vs arm64) when launching mach build -r/-d --uwp.

Without the /property:AppxBundle=Always;AppxBundlePlatforms=platform ("x64" | "arm64), the msbuild attempts to build every project (arm/x64/debug/release) within the solution.

msbuild /m:4 /p:project=ServoApp .\support\hololens\servoapp.sln /p:SolutionDir=.\support\hololens /p:Configuration="Debug" /p:Platform="x64" /property:AppxBundle=Always;AppxBundlePlatforms="x64"

[angelortiz1007]

In working to address issue #xxxx I found the following msbuild command sequence helped me build only what I wanted (x64 vs arm64) when launching mach build -r/-d --uwp. Without the /property:AppxBundle=Always;AppxBundlePlatforms=platform ("x64" | "arm64), the msbuild attempts to build every project (arm/x64/debug/release) within the solution. msbuild /m:4 /p:project=ServoApp .\support\hololens\servoapp.sln /p:SolutionDir=.\support\hololens /p:Configuration="Debug" /p:Platform="x64" /property:AppxBundle=Always;AppxBundlePlatforms="x64"

Sorry to reference issue #xxxx. Issue I meant to reference is: https://github.com/servo/servo/issues/23753

[angelortiz1007]

Create App Packages

martinsuchan:

I'm using your StoreBuildTest.sln from https://1drv.ms/u/s!AlURSa6JiyiVo5FJuYVe4v0k7FSjxQ.

So some of the problem lies in the solution macros and the other 2 project's macros (utils and Mainlib). It appears that when you build with the .sln all the macros know where everything is located and you are able to build the utils.lib, followed by mainlib.dll and finally the main app.

There are 2 issues/problems: Problem 1 When you use msbuild those macros ($(solutiondir) etc...) are not known and therefore msbuild will NOT find the .libs.

Problem 2 Seems to be where utils.lib is being created. When you use the .sln file it is put in the StoreBuildTest**Win32**\Release\lib_UWP.

if you build utils.lib using msbuild using: msbuild /m:4 .\utils\Utils.vcxproj /p:Configuration="release" /p:Platform="x86" /p:AppxBundle=Always;AppxBundlePlatforms="x86"

utils.lib is placed in: StoreBuildTest**Utils**\Win32\release\lib_UWP

Resolution: Try the following in your build.ps1 file. I tried them from a CMD prompt and was able to build. The main one you would like to execute is the last one that uses the mainapp.sln. The key to all the build process was to have the /p:SolutionDir=%CD%\StoreBuildTest\StoreBuildTest\

_msbuild /m:4 .\utils\Utils.vcxproj /p:Configuration="release" /p:Platform="x86" /p:AppxBundle=Always;AppxBundlePlatforms="x86" /p:SolutionDir=%CD%\StoreBuildTest\StoreBuildTest\

msbuild /m:4 .\MainLib\MainLib.vcxproj /p:Configuration="release" /p:Platform="x86" /p:AppxBundle=Always;AppxBundlePlatforms="x86" /p:SolutionDir=%CD%\StoreBuildTest\StoreBuildTest_

msbuild /m:4 mainapp.sln /p:Configuration="release" /p:Platform="x86" /p:AppxBundle=Always;AppxBundlePlatforms="x86" /p:SolutionDir=%CD%\StoreBuildTest\StoreBuildTest\

[martinsuchan]

@angelortiz1007 Hi, we've already fixed the issue some time ago using the UseSubFolderForOutputDirDuringMultiPlatformBuild=false flag, see the answer from @mirkomuggli .