[3.0.100-preview-009791] 'dotnet pack' uses intermediate output files for embedded resources

[nguerrera]

From @GeertvanHorrik on November 29, 2018 13:42

I am updating all the build scripts to use the 3.0 SDK, and most of it is working, except the embedding of resources.

C:\Program Files\dotnet\sdk\3.0.100-preview-009791\Sdks\NuGet.Build.Tasks.Pack\buildCrossTargeting\NuGet.Build.Tasks.Pack.targets(199,5): error NU5026: The file 'C:\CI_WS\Ws\118473\Source\Orc_Analytics\src\Orc.Analytics\obj\Release\net46\de\Orc.Analytics.resources.dll' to be packed was not found on disk. [C:\CI_WS\Ws\118473\Source\Orc_Analytics\src\Orc.Analytics\Orc.Analytics.csproj]

Note that these builds were succeeding with the 2.x SDK stuff.

I am adding a custom output directory, here are the properties specified for the build (using Cake):

        var msBuildSettings = new MSBuildSettings {
            Verbosity = Verbosity.Quiet, // Verbosity.Diagnostic
            ToolVersion = MSBuildToolVersion.Default,
            Configuration = ConfigurationName,
            MSBuildPlatform = MSBuildPlatform.x86, // Always require x86, see platform for actual target platform
            PlatformTarget = PlatformTarget.MSIL
        };

        var toolPath = GetVisualStudioPath(msBuildSettings.ToolVersion);
        if (!string.IsNullOrWhiteSpace(toolPath))
        {
            msBuildSettings.ToolPath = toolPath;
        }

        // Note: we need to set OverridableOutputPath because we need to be able to respect
        // AppendTargetFrameworkToOutputPath which isn't possible for global properties (which
        // are properties passed in using the command line)
        var outputDirectory = string.Format("{0}/{1}/", OutputRootDirectory, component);
        Information("Output directory: '{0}'", outputDirectory);
        msBuildSettings.WithProperty("OverridableOutputPath", outputDirectory);
        msBuildSettings.WithProperty("PackageOutputPath", OutputRootDirectory);

This has worked perfectly, but with the update to 3.x (preview), dotnet pack can no longer find the embedded resources since it's not looking in the output directory, but in the intermediate directory (which is not copied over from build agent to build agent since we are only interested in the output directory).

Here is the call to dotnet pack:

            var msBuildSettings = new DotNetCoreMSBuildSettings();

            // Note: we need to set OverridableOutputPath because we need to be able to respect
            // AppendTargetFrameworkToOutputPath which isn't possible for global properties (which
            // are properties passed in using the command line)
            msBuildSettings.WithProperty("OverridableOutputPath", outputDirectory);
            msBuildSettings.WithProperty("PackageOutputPath", OutputRootDirectory);
            msBuildSettings.WithProperty("ConfigurationName", ConfigurationName);
            msBuildSettings.WithProperty("PackageVersion", VersionNuGet);

            var packSettings = new DotNetCorePackSettings
            {
                MSBuildSettings = msBuildSettings,
                OutputDirectory = OutputRootDirectory,
                Configuration = ConfigurationName,
                NoBuild = true,
            };

            DotNetCorePack(projectFileName, packSettings);

Copied from original issue: dotnet/core-sdk#192

[nguerrera]

From @GeertvanHorrik on November 29, 2018 14:22

Doing some additional investigation with diagnostic output of MSBuild while executing Dotnet pack:

15:20:04.224   1:5>Done building target "CreateManifestResourceNames" in project "Orc.Analytics.csproj".: (TargetId:22)
15:20:04.224   1:5>Target "CreateCustomManifestResourceNames: (TargetId:23)" in file "C:\Program Files\dotnet\sdk\3.0.100-preview-009791\Microsoft.Common.CurrentVersion.targets" from project "C:\Source\orc.analytics\src\Orc.Analytics\Orc.Analytics.csproj" (target "PrepareResourceNames" depends on it):
15:20:04.224   1:5>Done building target "CreateCustomManifestResourceNames" in project "Orc.Analytics.csproj".: (TargetId:23)
15:20:04.224   1:5>Target "PrepareResourceNames: (TargetId:24)" in file "C:\Program Files\dotnet\sdk\3.0.100-preview-009791\Microsoft.Common.CurrentVersion.targets" from project "C:\Source\orc.analytics\src\Orc.Analytics\Orc.Analytics.csproj" (target "SatelliteDllsProjectOutputGroup" depends on it):
15:20:04.224   1:5>Done building target "PrepareResourceNames" in project "Orc.Analytics.csproj".: (TargetId:24)
15:20:04.224   1:5>Target "SatelliteDllsProjectOutputGroup: (TargetId:25)" in file "C:\Program Files\dotnet\sdk\3.0.100-preview-009791\Sdks\Microsoft.NET.Sdk.WindowsDesktop\targets\Microsoft.WinFX.targets" from project "C:\Source\orc.analytics\src\Orc.Analytics\Orc.Analytics.csproj" (target "_GetBuildOutputFilesWithTfm" depends on it):
                   Using "CreateItem" task from assembly "Microsoft.Build.Tasks.Core, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a".
                   Task "CreateItem" (TaskId:19)
                     Task Parameter:Include=obj\Release\net46\de\Orc.Analytics.resources.dll (TaskId:19)
                     Task Parameter:AdditionalMetadata=TargetPath=de\Orc.Analytics.resources.dll (TaskId:19)
                     Output Item(s): 
                         SatelliteDllsProjectOutputGroupOutputIntermediate=
                             obj\Release\net46\de\Orc.Analytics.resources.dll
                                     TargetPath=de\Orc.Analytics.resources.dll (TaskId:19)
                   Done executing task "CreateItem". (TaskId:19)
                   Task "CreateItem" (TaskId:20)
                     Task Parameter:Include=obj\Release\net46\es\Orc.Analytics.resources.dll (TaskId:20)
                     Task Parameter:AdditionalMetadata=TargetPath=es\Orc.Analytics.resources.dll (TaskId:20)
                     Output Item(s): 
                         SatelliteDllsProjectOutputGroupOutputIntermediate=
                             obj\Release\net46\es\Orc.Analytics.resources.dll
                                     TargetPath=es\Orc.Analytics.resources.dll (TaskId:20)
                   Done executing task "CreateItem". (TaskId:20)

[nguerrera]

From @GeertvanHorrik on November 29, 2018 14:39

Trying to enforce overriding the intermediate output path:

msBuildSettings.WithProperty("IntermediateOutputPath", System.IO.Path.Combine(outputDirectory, "$(TargetFramework)") + System.IO.Path.DirectorySeparatorChar);

Alsmost seems to work, getting this:

C:\Source\orc.analytics\output\Release\Orc.Analytics\$(TargetFramework)\de\Orc.Analytics.resources.dll

[nguerrera]

From @dasMulli on November 29, 2018 15:5

does it change when you don't set NoBuild=true?

.WithProperty("OverridableOutputPath", outputDirectory);

This looks like a custom property, how do you use it inside the project file?

[nguerrera]

From @GeertvanHorrik on November 29, 2018 15:6

When I set NoBuild = false, it will probably (re)build the project, but I want to prevent it because it's already built (and analyzed using SonarQube, etc).

It's a custom property which I enabled for some workarounds:

  <!-- Explicit additional project properties that require stuff set before -->
  <PropertyGroup>
    <!-- 
      Use MsBuildProjectName, this allows us to use the same assembly name for different
      versions, e.g.:

      Orc.EntityFramework5 => Orc.EntityFramework.dll
      Orc.EntityFramework6 => Orc.EntityFramework.dll

      If we would use AssemblyName, they would be compiled into the same directory
    -->
    <OverridableOutputPath>$(ProjectDir)..\..\output\$(Configuration)\$(MSBuildProjectName)\</OverridableOutputPath>
    <OutputPath>$(OverridableOutputPath)</OutputPath>
    <DocumentationFile>bin\$(Configuration)\$(TargetFramework)\$(AssemblyName).xml</DocumentationFile>
  </PropertyGroup>

[nguerrera]

From @GeertvanHorrik on November 29, 2018 15:23

Got it working!

1) For local builds, I "simulate" the build agents by deleting the dll from the obj folders, but always do a restore to ensure projects.assets.json 2) Explicitly set a custom property on msbuild for the Dotnet pack:

            // Fix for .NET Core 3.0, see https://github.com/dotnet/core-sdk/issues/192, it
            // uses obj/release instead of [outputdirectory]
            msBuildSettings.WithProperty("DotNetPackIntermediateOutputPath", outputDirectory);

3) In a targets file, use this:

  <!-- 
    Fix for .NET Core 3.0, see https://github.com/dotnet/core-sdk/issues/192, it 
    uses obj/release instead of [outputdirectory] 
  -->
  <PropertyGroup Condition=" '$(DotNetPackIntermediateOutputPath)' != '' ">
    <IntermediateOutputPath>$(DotNetPackIntermediateOutputPath)</IntermediateOutputPath>
  </PropertyGroup>

Here is the result:

You can find a working repro here:

https://github.com/wildgums/orc.analytics

Just open the directory in powershell and run:

.\build.ps1 -target buildandpackagelocal

[nguerrera]

From @GeertvanHorrik on November 29, 2018 15:24

Btw, this shows a workaround, I still think it's weird that packaging with nobuild=true somehow requires the /obj/release folders.