Please provide a way to publish from a specific build directory (Azure pipelines)

[gh-andre]

Is your feature request related to a problem? Please describe.

Azure DevOps pipelines provide distinct directories for different build purposes that work out well in C++ projects, but not so much for .Net Core ones.

In a C++ project I can build source in Build.SourcesDirectory, output to Build.BinariesDirectory and package artifacts in Build.ArtifactStagingDirectory. This allows me to maintain incremental builds on my own build agents because the build output directory keeps previous build object files and when new source is checked out, only changed source files are being compiled.

With dotnet builds, I can either build in a different directory or publish to a different directory, but I don't see a way to combine those. That is, a typical dotnet build would look like this:

dotnet restore -p:Platform="Any CPU" myapp.sln
dotnet clean --configuration Release -p:Platform="Any CPU" myapp.sln
dotnet build --no-restore --configuration Release -p:Platform="Any CPU" myapp.sln
dotnet publish --no-self-contained --no-build `
    --configuration Release -p:Platform="Any CPU" `
    --output $(Build.ArtifactStagingDirectory) myapp.sln

This, however, builds right in the source directory, which gets cleaned with every checkout. What I want is to be able to tell dotnet publish to output to Build.ArtifactStagingDirectory, avoid building and instead use build output from Build.BinariesDirectory for publishing.

Just to be clear, yes, there's a way not to clean the source directory, but that's a hack - I want the source directory behave as it is intended - get the fresh source on every check-out, so there are no lingering source files, as some of them may get deleted in development.

Describe the solution you'd like

I would like to be able to specify for dotnet publish build output and publish output directories, like this:

dotnet build --no-restore --configuration Release -p:Platform="Any CPU" `
    --output $(Build.BinariesDirectory) myapp.sln
dotnet publish --no-self-contained --no-build --configuration Release -p:Platform="Any CPU" `
    --build-output $(Build.BinariesDirectory) `
    --output $(Build.ArtifactStagingDirectory) myapp.sln

[baronfel]

Based on some investigation, you should be able to publish from the previous directory with the following explicit MSBuild parameters:

• OutputPath set to Build.BinariesDirectory
• PublishDir set to Build.ArtifactStagingDirectory

The OutputPath property on the publish command should control where the publish targets look for the outputs of the previous build command, and the PublishDir should set the root path for the publishable outputs of the publish command.

Can you try this and report back?

[gh-andre]

@baronfel Will test this, but wanted to clarify. Do you mean OutDir or OutputPath has some significance here?

MSBuild folks describe OutputPath as a legacy option that should be avoided.

https://github.com/dotnet/msbuild/issues/87#issuecomment-101467157

[baronfel]

OutDir should work equivalently, from a quick glance over in the MSBuild targets.

[gh-andre]

@baronfel I ran a quick test locally and it appears to be working. Later on I will confirm this in an Azure pipeline build and if anything pops up, will add a comment here.

A couple of notes.

PublishDir doesn't come up in a list of macros in Visual Studio IDE, but when typed manually, does expand to a path.

In the context of this thread it works out without specifying PublishDir explicitly. Here's what my quick test looked like.

dotnet build --no-restore --configuration Release -p:Platform="Any CPU" `
    --output c:\temp\myapp\build_output myapp.sln
dotnet publish --no-self-contained --no-build --configuration Release -p:Platform="Any CPU" `
    -p:OutDir="c:\temp\myapp\build_output" `
    --output c:\temp\myapp\publish_output myapp.sln

The output contained the expected set of files selected for publishing.

Thanks for the suggestion.

[baronfel]

Glad to hear it! Your final version doesn't surprise me that it works - the publish command's --output option just sets the PublishDir property, so I was pretty sure it would work. The OutputPath recommendation actually came from the same logic for the build command's --output option. @rainersigwald should the SDK move to forwarding OutDir instead of OutputPath for dotnet build --output <some random dir>?

[rainersigwald]

Unfortunately I don't think we can change between OutputPath and OutDir without subtly breaking something. Those properties are Trouble.

For the problem described above, I'd take a different approach: change the repo to have the source/bin/publish separation you desire by configuring folders in Directory.Build.props/targets. You can default the folders (so it works on local machines) but respect the AzDO configuration if it exists. Then your build script becomes

dotnet build --no-restore --configuration Release -p:Platform="Any CPU" myapp.sln
dotnet publish --no-self-contained --no-build --configuration Release -p:Platform="Any CPU" myapp.sln

with no path mappings.

[gh-andre]

@rainersigwald Thanks for the suggestion, but these paths cannot be in the source because one CI pipeline may build/publish in/to different directories. Just like build numbers, these directories are expected to be provided by the build environment.

[baronfel]

@gh-andre if your CI system exposes the paths via environment variables you could use those in the Directory.Build.props/targets instead, since environment variables are usable by MSBuild.

[gh-andre]

@baronfel This sounds like a workaround for keeping these paths in the source, which will result in more complex build setups because these variables will need to be set up for all builds, including local ones, even if they are defaulted. Having paths passed in explicitly is a self-documenting approach that works for any path.

Anyway, having said that, it's good to know that this approach is possible and maybe somebody who comes across of this thread will find it more usable for their purposes.

As far as this issue is concerned, OutDir solves this for me, if you want to close it. I would suggest documenting this on the dotnet publish page. I use OutDir for C++ builds, but it didn't occur to me that it would work out here as well. Others may be in the same boat.

Thanks again for your help.

[voroninp]

I was also surprised to discover that the `dotnet publish' command does not have a parameter to reference a directory from which to retrieve build artifacts.