Projects sharing outputs will delete each-others output

[dotMorten]

Steps to Reproduce

1. Create a new iOS Class Library ClassLibrary1
2. Create a second iOS Class Library ClassLibrary2
3. Make CL2 reference CL1.
4. Compile and notice everything is fine.
5. Now change the two project's output paths to share the same output folder. ClassLibrary2 will now fail, because on compilation start, it'll delete the output of ClassLibrary1,

Expected Behavior

ClassLibrary2 shouldn't delete the output of ClassLibrary1.

Actual Behavior

ClassLibrary2 deletes the output of ClassLibrary1 when it begins compiling.

Environment

Microsoft Visual Studio Enterprise 2017 
Version 15.7.3
VisualStudio.15.Release/15.7.3+27703.2026
Microsoft .NET Framework
Version 4.7.03056

Installed Version: Enterprise

Visual C++ 2017   00370-00000-16168-AA762
Microsoft Visual C++ 2017

Application Insights Tools for Visual Studio Package   8.12.10405.1
Application Insights Tools for Visual Studio

ArcGIS Runtime SDK for .NET   100.3.0
ArcGIS Runtime SDK for .NET allows developers to build immersive, native mapping applications for Windows, Android, and iOS devices using C#.  It includes five APIs: WPF to create apps for Windows Desktop, UWP to create Universal Windows apps, Xamarin.Android and Xamarin.iOS for Android and iOS apps that need access to native functionality, and Xamarin.Forms to create apps that share UI layouts across Android, iOS, and UWP.

ASP.NET and Web Tools 2017   15.0.40522.0
ASP.NET and Web Tools 2017

ASP.NET Core Razor Language Services   15.7.31476
Provides languages services for ASP.NET Core Razor.

ASP.NET Web Frameworks and Tools 2017   5.2.60419.0
For additional information, visit https://www.asp.net/

Azure App Service Tools v3.0.0   15.0.40424.0
Azure App Service Tools v3.0.0

Azure Functions and Web Jobs Tools   15.0.40617.0
Azure Functions and Web Jobs Tools

C# Tools   2.8.3-beta6-62923-07. Commit Hash: 7aafab561e449da50712e16c9e81742b8e7a2969
C# components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Common Azure Tools   1.10
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

JavaScript Language Service   2.0
JavaScript Language Service

JavaScript Project System   2.0
JavaScript Project System

JavaScript UWP Project System   2.0
JavaScript UWP Project System

Merq   1.1.19-rc (a4ffc1b)
Command Bus, Event Stream and Async Manager for Visual Studio extensions.

Microsoft Azure Tools   2.9
Microsoft Azure Tools for Microsoft Visual Studio 2017 - v2.9.10420.2

Microsoft Continuous Delivery Tools for Visual Studio   0.3
Simplifying the configuration of continuous build integration and continuous build delivery from within the Visual Studio IDE.

Microsoft JVM Debugger   1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines

Microsoft MI-Based Debugger   1.0
Provides support for connecting Visual Studio to MI compatible debuggers

Microsoft Visual C++ Wizards   1.0
Microsoft Visual C++ Wizards

Microsoft Visual Studio Tools for Containers   1.1
Develop, run, validate your ASP.NET Core applications in the target environment. F5 your application directly into a container with debugging, or CTRL + F5 to edit & refresh your app without having to rebuild the container.

Microsoft Visual Studio VC Package   1.0
Microsoft Visual Studio VC Package

Mono Debugging for Visual Studio   4.10.5-pre (ab58725)
Support for debugging Mono processes with Visual Studio.

NuGet Package Manager   4.6.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.

Project System Tools   1.0
Tools for working with C#, VisualBasic, and F# projects.

ProjectServicesPackage Extension   1.0
ProjectServicesPackage Visual Studio Extension Detailed Info

ResourcePackage Extension   1.0
ResourcePackage Visual Studio Extension Detailed Info

Snapshot Debugging Extension   1.0
Snapshot Debugging Visual Studio Extension Detailed Info

SQL Server Data Tools   15.1.61804.210
Microsoft SQL Server Data Tools

Test Adapter for Boost.Test   1.0
Enables Visual Studio's testing tools with unit tests written for Boost.Test.  The use terms and Third Party Notices are available in the extension installation directory.

Test Adapter for Google Test   1.0
Enables Visual Studio's testing tools with unit tests written for Google Test.  The use terms and Third Party Notices are available in the extension installation directory.

TypeScript Tools   15.7.20419.2003
TypeScript Tools for Microsoft Visual Studio

Visual Basic Tools   2.8.3-beta6-62923-07. Commit Hash: 7aafab561e449da50712e16c9e81742b8e7a2969
Visual Basic components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Visual F# Tools 10.1 for F# 4.1   15.7.0.0.  Commit Hash: 2527e6829ecdc8281ee60d83be8cfd0fa720a648.
Microsoft Visual F# Tools 10.1 for F# 4.1

Visual Studio Code Debug Adapter Host Package   1.0
Interop layer for hosting Visual Studio Code debug adapters in Visual Studio

Visual Studio Tools for CMake   1.0
Visual Studio Tools for CMake

Visual Studio Tools for Universal Windows Apps   15.0.27703.2026
The Visual Studio Tools for Universal Windows apps allow you to build a single universal app experience that can reach every device running Windows 10: phone, tablet, PC, and more. It includes the Microsoft Windows 10 Software Development Kit.

VisualStudio.Mac   1.0
Mac Extension for Visual Studio

Windows Machine Learning Generator Extension   1.0
Windows Machine Learning Visual Studio Extension Detailed Info

Xamarin   4.10.10.1 (f1760154c)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin Designer   4.12.1 (f3257e429)
Visual Studio extension to enable Xamarin Designer tools in Visual Studio.

Xamarin.Android SDK   8.3.3.2 (HEAD/dffc59120)
Xamarin.Android Reference Assemblies and MSBuild support.

Xamarin.iOS and Xamarin.Mac SDK   11.12.0.4 (64fece5)
Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support.

Build Logs

ClassLibrary4_Debug_AnyCPU_Build_2018-08-03T16_14_39.3297140-07_00.zip

Example Project (If Possible)

ClassLibrary3.zip

VS bug #659664

[dotMorten]

It looks like it's caused by this part of the build:

[dotMorten]

Confirmed that if I redeclare the target, the problem goes away:

  <Target Name="_CleanMacBuild" />

[dotMorten]

This file seems to be the culprit: c:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Xamarin\iOS\Xamarin.iOS.Windows.After.targets

    <Target Name="_CleanMacBuild" DependsOnTargets="_SayHello">
        <RemoveDir SessionId="$(BuildSessionId)"
               Condition="'$(MtouchTargetsEnabled)' == 'true'"
               Directories="$(OutputPath);$(IntermediateOutputPath)"
               RemoveAppDir="$(RemoveAppDir)"
               ContinueOnError="true" />

I'm not sure why it is triggering even when it's not a clean, but in any case, I don't think a clean should blow away the entire folder, but only the artifacts that single project is responsible for.

[mandel-macaque]

@dotMorten I have tested the provided test case on VS for Mac and I have not been able to reproduce the issue with the following version.

I have added the appropriate tag for the VS for Windows eng to take a look at the issue and investigate further.

Thanks for the bug report.

[mandel-macaque]

Removing tag since I did a second run and confirmed it does happen on Mac too.

[mandel-macaque]

Just to clarify, the compilation does complete on VSfM and the output is as follows:

. ├── ./ClassLibrary3 │   ├── ./ClassLibrary3/Class1.cs │   ├── ./ClassLibrary3/ClassLibrary3.csproj │   ├── ./ClassLibrary3/ClassLibrary3.csproj.user │   ├── ./ClassLibrary3/Properties │   │   └── ./ClassLibrary3/Properties/AssemblyInfo.cs │   ├── ./ClassLibrary3/Resources │   └── ./ClassLibrary3/obj │   └── ./ClassLibrary3/obj/Debug │   ├── ./ClassLibrary3/obj/Debug/ClassLibrary1.dll │   ├── ./ClassLibrary3/obj/Debug/ClassLibrary1.pdb │   ├── ./ClassLibrary3/obj/Debug/ClassLibrary3.csproj.CoreCompileInputs.cache │   └── ./ClassLibrary3/obj/Debug/ClassLibrary3.csproj.FileListAbsolute.txt ├── ./ClassLibrary3.sln ├── ./ClassLibrary4 │   ├── ./ClassLibrary4/Class1.cs │   ├── ./ClassLibrary4/ClassLibrary4.csproj │   ├── ./ClassLibrary4/Properties │   │   └── ./ClassLibrary4/Properties/AssemblyInfo.cs │   ├── ./ClassLibrary4/Resources │   └── ./ClassLibrary4/obj │   └── ./ClassLibrary4/obj/Debug │   ├── ./ClassLibrary4/obj/Debug/ClassLibrary1.dll │   ├── ./ClassLibrary4/obj/Debug/ClassLibrary1.pdb │   ├── ./ClassLibrary4/obj/Debug/ClassLibrary4.csproj.CopyComplete │   ├── ./ClassLibrary4/obj/Debug/ClassLibrary4.csproj.CoreCompileInputs.cache │   ├── ./ClassLibrary4/obj/Debug/ClassLibrary4.csproj.FileListAbsolute.txt │   └── ./ClassLibrary4/obj/Debug/ClassLibrary4.csprojResolveAssemblyReference.cache └── ./SharedOutput ├── ./SharedOutput/ClassLibrary1.dll └── ./SharedOutput/ClassLibrary1.pdb

which means works as expected. This is a Windows only issues as per comment https://github.com/xamarin/xamarin-macios/issues/4574#issuecomment-410402020

[dotMorten]

Any progress on this? This is a MAJOR showstopper for our entire team. Having multiple class libraries in a single solution is NOT an uncommon thing. Workaround no longer works in the latest VS Update.

[dotMorten]

Here's a good explanation how clean is supposed to be done: https://github.com/Microsoft/msbuild/issues/2408#issuecomment-321082997

[kzu]

@dotMorten what IS an uncommon thing is for teams to set a single output folder. I understand this is important to you, we're investigating ways to improve it. Still, it is not, IMO, a MAJOR showstopper since all you need to do to fix it is to use VS defaults for output folders. With Mac build host having to sync files across two machines, there are a number of things to consider including checks on out of date output on both sides and so on that are non-trivial.

[dotMorten]

what IS an uncommon thing is for teams to set a single output folder

Absolutely not uncommon to have all your artifacts go to the same output. This is quite a typical build server setup, and we're not changing the entire build system setup because Xamarin.iOS decides a build should also do a clean, and in addition the clean is incorrectly implemented.

Second having artifacts (and intermediates) go into the source controlled folder (gitignore or not) is a really bad practice. Especially in larger solution setups. I get why it's the default for VS as it's a simple starting point for a single contained project, but it quickly falls apart once you start going bigger, or you're dealing with larger complex build systems with artifacts in many places.

Third, this is a regression. It' had been working fine for years.

[dotMorten]

...could we perhaps start with not cleaning on a regular build? On an actual clean or rebuild I don't think the damage is so bad. But cleaning the same folder over and over again as each project builds is, obviously, not good.

[kzu]

Absolutely not uncommon to have all your artifacts go to the same output.

Well, I guess without some telemetry or data, there's no way to decide that ;-). Ditto for what is considered a "typical build server setup" or what is "really bad practice". No point in arguing either way.

Xamarin.iOS decides a build should also do a clean

As I mentioned, you can think of it in those simple terms if that helps you, but if you want to really understand the complexities around a XI build, I'd suggest reading http://www.cazzulino.com/how-vs-builds-on-mac-with-xamarin.html at least. If we never cleaned anything on the Windows side in between builds, rest assured that there would be a host of other issues with incremental builds on the remote build host. We need to carefully balance speed of builds vs incremental build and clean.

Third, this is a regression. It' had been working fine for years.

I've checked the commits for that particular target and it hasn't changed in years. So I wouldn't consider it a regression up-front.

Could you try the following workaround?

<Target Name="_DisableMtouchTargets" Condition="'$(DisableMacClean)' == 'true'" BeforeTargets="_CleanMacBuild" DependsOnTargets="_SayHello">
  <PropertyGroup>
    <!-- save original value to restore after the original clean -->
    <_MtouchTargetsEnabled>$(MtouchTargetsEnabled)</_MtouchTargetsEnabled>
    <MtouchTargetsEnabled>false</MtouchTargetsEnabled>
  </PropertyGroup> 
</Target>

<Target Name="_RestoreMtouchTargets" Condition="'$(DisableMacClean)' == 'true'" AfterTargets="_CleanMacBuild">
  <PropertyGroup>
    <MtouchTargetsEnabled>$(_MtouchTargetsEnabled)</MtouchTargetsEnabled>
  </PropertyGroup> 
</Target>

You can place that in an import for the XI projects.

Then just set DisableMacClean=true for your CI build.

Let me know how that goes.

Thanks!

[CSkoubo]

This is still an issue for us as well. We have made it work with the mentioned workaround.

`

<_MtouchTargetsEnabled>$(MtouchTargetsEnabled) false

</Target>
<Target Name="_RestoreMtouchTargets" Condition="'true' == 'true'" AfterTargets="_CleanMacBuild">
  <PropertyGroup>
    <MtouchTargetsEnabled>$(_MtouchTargetsEnabled)</MtouchTargetsEnabled>
  </PropertyGroup> 
</Target> `

However as you also state, it can give complications when building. So at times we need to delete the cache on the mac when in bridge mode.

[ToreDemant]

As @CSkoubo mentions above, then we have this issue as well.

I agree that only telemetry can say what is the most common setup, so I an not sure how @kzu concludes that this is an uncommon setup. However this behavior is at least different from other project types, so in that sense it is not following standards and makes us having to handle a Xamarin.iOS project different from Xamarin.Android or C#.

For the clean on every build I fully agree with @dotMorten that this seems like an obvious mistake.

[dotMorten]

It doesn't really matter how common it is (apart from priority). No other project type does this, so what's the argument that this project type needs to be the only one doing it?

[CSkoubo]

Is this resolved? If so which version? @vs-mobiletools-engineering-service2

[dalexsoto]

Sorry, looks like automation closed this by mistake.

[mariaghiondea]

This item has been open for a while and we weren’t able to prioritize it over other higher impact issues we received, based on the votes and comments from others in the community and our understanding of the issue. Based on that, our current and future priorities in this area and others, I’m closing it since the chances of it becoming high enough in priority for a fix in the future are very low. I understand that may be disappointing, but I don’t want to create false hopes.

[kzu]

Late to the party :). @dotMorten

No other project type does this, so what's the argument that this project type needs to be the only one doing it?

No other project type does remote builds like XI does, so, there's that.