Unable to publish multiple projects at the same time.

[joelharkes]

When i publish with dotnet cli, the output is:

publish: Published to ../output/project1 Published 1/1 projects successfully

Which indicates i could publish multiple projects at te same time (why else say: 1/1)

so i tried this.

Steps to reproduce

dotnet publish project1 project2 -o ../output

Expected behavior

I expect 2 projects to be published in this output folder (each in there own folder, having the same name as the project folder).

Actual behavior

Unrecognized command or argument project2

Environment data

.NET Command Line Tools (1.0.0-preview2-1-003155)

Product Information:
 Version:            1.0.0-preview2-1-003155
 Commit SHA-1 hash:  d7b0190bd4

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.14393
 OS Platform: Windows
 RID:         win10-x64

[TheRealPiotrP]

@blackdwarf I agree with @joelharkes's assertion that the experience in preview2.1 was confusing. Thoughts? Note, this is a bug on the PJ CLI so it may not apply directly, but what is our intent on multi-project [sln?] publish?

[joelharkes]

@piotrpMSFT I think multi publish should be considered.

Our architecture is made up of 2 kestrel console apps and 3 other console applications. now i have to execute 5 times dotnet publish. 5 times going over library projects and all executing in synchronous order (otherwise i don't know if it will run into locking problems with trying to build the same project). This is a lot of overhead. if this is managed by the CLI, projects could be run asynchronously, speeding up the build performance with a big factor.

[TheRealPiotrP]

@blackdwarf will propose the intention change here.

[blackdwarf]

For publish, I don't think publishing from a solution falls naturally because it is highly dependent on the solution architecture and the overall type of projects that are contained within. For some solutions, like the one @joelharkes mentions, it may make sense. For others that I saw and discussed as I was thinking about this, the expected behavior would be exactly the opposite, that is, publish would be expected not to work on SLN files as that would produce an output that would not be wanted.

There are also many nuances that need to be thought of when thinking on this, mostly w.r.t higher-level tooling that will also consume this target and how does running that target on the solution works for their UX-es and capabilities.

So, for V1 at least, I don't see us supporting this behavior.

[neman]

This is can be easy be done with powershell script. You just have to create for each project build/publish script, and then execute it within loop. This way I build/publish/create nuget/push nuget with one command for more than 30 projects in one solution.

This code could be in some PublishMultipleProjects.ps1 script. You can extend this by adding parms what to include and so on...

Push-Location $PSScriptRoot     
$files = get-childitem -Recurse -Include "Build.ps1"   
foreach ($file in $files)    
{   
    # Execute each Build.ps1 script   
    & $file   
}   
Push-Location $PSScriptRoot

[joelharkes]

@neman That is exactly what you want to avoid for 2 main reasons:

• They ported back to msbuild so you could use an msbuild file for this instead of powershell
• This is either: Slow because its all executed after each other or gets into locking issues because trying to build multiple projects at once.

I made a script that builds the dependency trees for all projects and then executes builds in parallel (Promises nodejs) according to the builds. I got a major decrease in build/publish time (on a multi core environment).

Too bad though our build server agents only use 1 processor per agent but, but locally for developers its still better this way.

[dasMulli]

I've managed to bring down build+test time by using msbuild scripts like this (even in non-parallel builds): https://gist.github.com/dasMulli/69f5303aa79a8cd4060e44891c90fd2d

Essentially, making a single build.proj file that calls publish on others via <MSBuild/> tasks (=> dotnet msbuild build.proj.

[joelharkes]

@dasMulli looks good, but ContinueOnError="ErrorAndContinue" this seems very dangerous/error-prone? are you sure dependencies are always build before projects higher in the tree are build?

[dasMulli]

I've used ErrorAndContinue to make sure it at least tries to test all test projects. The target will still return errors but ensures that all tasks are executed. Dependencies are always built before individual projects and msbuild should already handle concurrency situations inside a single msbuild invocation. For lots of test projects, it's probably even faster to build a solution and then running tests in parallel using VSTestNoBuild=true as additional parameter.

[joelharkes]

We are planning our migration to csproj. I seem to have the same problem here.

I want to publish 5 startup projects at the same time not needing to rebuild library projects more than once. But still there seems no possiblity?

if i dotnet publish solution.sln it puts all the 5 projects into 1 folder.

If i run dotnet publish project1.csproj it publishes 1 project but first builds all the project refernces.

Could there be an option made to either specify separate publish projects in the solution file? Or an option: --no-build like on dotnet pack to publish a project without rebuilding it?

[joelharkes]

@blackdwarf is there anything on the roadmap for this feature? it could greatly speed up our build process.

[blackdwarf]

@joelharkes sorry, but I'm not with Microsoft anymore so I really don't know what the roadmap is like these days. @livarcocc @richlander and @bleroy could help with this though, I would think.

FWIW, when we took a look at publishing based on the SLN file, I remember that there was a big split on whether the behavior could be made sane enough for most use cases to work. Some people were very for it (similar to dotnet restore or dotnet build), while others thought it was exactly the wrong thing to do. Add to it the options for redirecting output (--output) and you have a nice can 'o' worms there.

Your idea of having a --no-build option on dotnet publish does sound useful, so I guess a separate issue could be made for that, but that would also have to go into the dotnet/sdk repo IINM.

[gandarez]

having the same behaviour here, I have two projects to be published and "dot net" puts both on the same page 👎

[livarcocc]

cc @KathleenDollard

@gandarez are you publishing a solution or publishing with -o?

[gandarez]

@livarcocc I'm publishing through vsts using -o

[KathleenDollard]

This issue is still open, meaning we haven't said no. But we also do not have it on the roadmap as far as I know. I believe it needs some thought to get right.

@ganderez - I am a bit confused about your scenario. Can you send more information so we can figure out if this is a CLI issue or a VSTS issue?

[gandarez]

@KathleenDollard The solution contains two web api core projects and I'm using dot net publish to publish them. But the cli put every single project to the same root output directory. The MSBUILD does not have that behaviour, it groups each project in its respective folder.

[dasMulli]

@gandarez when you say "The MSBUILD does not have that behaviour", how have you been using msbuild? the msbuild equivalent of dotnet publish -o foo would be msbuild /t:Publish /p:PublishDir=foo. If you have been using /p:DeployOnBuild=true with a publish profile, that is a different way of publishing specific to asp.net and asp.net core and may behave differently. It should also be noted that if you pass a relative path to the -o parameter, it is always interpreted relative to the individual csproj file, not relative to the path the dotnet publish command is invoked from.

[livarcocc]

@gandarez can you answer @dasMulli questions above so that we can understand your scenario better?

[housten]

One solution that seems simple to me would be to have a variable that references the current project being built/published so one can have dotnet publish -o foo\$(Build.CurrentProject) or msbuild /t:Publish /p:PublishDir=foo\$(Build.CurrentProject) if one wants separate folders for each publish in a solution. Or maybe this already exists? I am not seeing it in the build variables documentation at any rate.

[avodovnik]

Has there been any progress on this issue, or do we know of any other way around this?

[rojepp]

We publish several console apps to the same dir and distribute that. ILLink.Tasks runs as part of publish. Without the ability to publish several projects at once, how can the linker do its job properly?

[joelharkes]

Closing issue, there is probably an alternative by now.

[haf]

@joelharkes I haven't found an alternative yet. Paket's multi-push is broken as well https://github.com/fsprojects/Paket/issues/3537

[mariusGundersen]

@joelharkes why was this closed? There is no alternative solution and nothing in this issue suggests there is.

[Lanayx]

Probably the workaround is to join projects into a solution and run dotnet publish on solution, this makes a simultaneous publish.

[mariusGundersen]

@Lanayx no this is exactly what doesn't work, and what brought me to this issue. Running dotnet publish -o outputPath on the solution will not create a separate folder for each project, it will dump everything into the single folder outputPath.

[Lanayx]

@mariusGundersen I use dotnet publish -c Release -o artifacts and it publishes each project to the artifacts folder that is situated at each project's location

[mariusGundersen]

That will result in the path to the output being ./{project}/artifacts while what I would want is ./artifacts/{project}. Unfortunately that is not possible

[Lanayx]

@mariusGundersen how do you expect artifacts to be situated there? Should they reflect the initial folder structure? If yes, then how is it better than the available option?

[mariusGundersen]

Compare it to how dotnet pack -o outputDir works. It uses the name of the csproj for the name of the nupkg. Publish could work the same way, taking the name of the csproj and using it as the name of the directory that the content is published to.

[joelharkes]

Sorry guys, was not working in dotnet core for a long while so i though clean up my open issues im no longer working on (and might already be outdated).

I'll re-open it again for you guys

[haf]

@mariusGundersen Still; this issue is about publishing multiple projects with a command, not to change the directory structure of publishes of a single .sln file. I think it would be appropriate to file a new issue if you want another type of change, than what this thread is about.

Workaround: I started using https://github.com/eiriktsarpalis/snippets/tree/master/SlnTools to package multiple projects with FAKE.

[tai-yi]

Here is a workaround to publish multi projects in a solution

Directory example

📦src ┣ 📂Web ┃ ┗ 📜Web.csproj ┣ 📂WebApp2 ┃ ┗ 📜WebApp2.csproj ┣ 📜publish.targets ┗ 📜sln.sln

1. Add shared MSbuild target file as below you can change default PublishRootDir or use command line args /p:PublishRootDir=somedir

   <Project>
   <PropertyGroup>
       <PublishRootDir>..\..\bin</PublishRootDir>
   </PropertyGroup>
   <Target Name="PublishAssets" AfterTargets="Publish">
       <ItemGroup>
           <_PublishAssets Include="$(OutDir)\publish\**\*.*" />
       </ItemGroup>
       <Message Importance="High" Text="Publishing assets " />
       <Copy SourceFiles="@(_PublishAssets)" DestinationFolder="$(PublishRootDir)\$(TargetName)\%(RecursiveDir)" SkipUnchangedFiles="true" />
   </Target>
   </Project>

2. Reference it to projects you want to publish <Import Project="$(MSBuildThisFileDirectory)..\publish.targets" />
3. Try run 'dotnet publish' in directory src 😁

[cphillips83]

This is a 3 year old issue that should be trivial to fix, so I'm not sure why this is still lingering around? Several suggested solutions have been presented and even the referencing issues to this one outline how this mistake came to be.

1. What we need is to either have the -o append each project name (just like pack is doing) to create a sub folder for each project.
2. Or allow us to add a variable argument to the output path that is replaced.

It does not make sense to dump everything to a single folder when publishing a solution and I would argue that when it would, this feature would be the minority and you would want a flag to enable it as the amount of issues this can cause has been outlined in other issues.

For now, I have choosen to build tooling around my tooling and removed the -o param and have instead wrote a script to recursively find all the publish folders generated under each project and to move them to a root level folder.

[megakid]

We would love this too, our build times would be greatly reduced if this feature was added 👍

[Ryan2065]

This would be helpful for us.

something like:

dotnet publish -o OutDir -PublishAllSeparately

dotnet publish -o OutDir -ProjectsSeparated Project1,Project2,Project3

The it could publish them in folders individually or just publish the ones specified in the folders (sometimes there's libraries in the solution you wouldn't want published by itself)

[williamdenton]

found this which seems relevant:

https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-publish You can publish a solution (not just a project).

dotnet publish YourSolution.sln -c Release -p:PublishDir=.\publish

If you specify a relative path when publishing a solution, all output for all projects goes into the specified folder relative to the current working directory. To make publish output go to separate folders for each project, specify a relative path by using the msbuild PublishDir property instead of the --output option. For example, dotnet publish -p:PublishDir=.\publish sends publish output for each project to a publish folder under the folder that contains the project file.

this was added in .NET Core 3.0

[AVee]

Since this issue was reported nearly 5 years ago, someone bothered to change the error instead of resolving the actual issue: MSBUILD : error MSB1008: Only one project can be specified.

Based on the comment from @williamdenton we can have both scenarios described here (publish to separate folders and publish to a single folder) for a solution file. So why not for multiple projects from the command line? Sure, I can create a .sln for any combination of projects I'd want to publish in a single go but surely it shouldn't be too difficult to just fix this?

[StingyJack]

The earlier build systems for TFS used to dump all the project outputs into a single folder, it was a pain. I had to write a tool to go re-assemble those loose files back into the different programs and library groups and run that at the end of the workflow. I've authored several hundred definitions in different versions of TFS, AzDo, and other systems and for different organizations and I have yet to find a case where a single out folder was the right choice for building a solution. It really feels like a niche or specific use case that can be covered by a post build script. Can someone who is a proponent of the single folder explain when it is necessary to have this?

[billybraga]

We build a single docker image for our Api and Cli projects (we use the Cli project to run k8s jobs, like run migrations before deploying the new Api version). We want to publish those two projects and their dependencies without having the test projects and their dependencies.

[AVee]

The earlier build systems for TFS used to dump all the project outputs into a single folder, it was a pain.

The proposal here is specifically not to dump all output from a solution into a single folder, but to be able to selectively publish multiple projects to a single folder.

Can someone who is a proponent of the single folder explain when it is necessary to have this?

My usecase is similar to what @billybraga is doing (although there is no docker), I have an API with related CLI applications for tooling. They belong together, share the same libraries and need to be deployed together. This involves things like database migration tools as mentioned above, but in my case also export/export tools intended to be run from shell manually or from cron. In Unix-like operating systems it is pretty common to have several related executables sharing supporting libraries which together make up a single package, I don't see a clean way to do this in dotnet.

(Also, it's kinda annoying that dotnet build will happily publish all projects to a single folder when presented with a .sln, but claim it's impossible when presented with a list of specific projects)

[StingyJack]

They belong together, share the same libraries and need to be deployed together. This involves things like database migration tools as mentioned above, but in my case also export/export tools intended to be run from shell manually or from cron.

@AVee - Thanks, this makes sense. I've needed this before on rare occasion and would use the post build event on each supporting application to copy its files into the primary app's output folder. Thats probably the cleanest way to do it in dotnet as it behaves the same way locally and on a build server, it's easy to troubleshoot, and as long as you use powershell core commands to do the copying it will work on any platform also.

The proposal here is specifically not to dump all output from a solution into a single folder, but to be able to selectively publish multiple projects to a single folder.

I read the ask as wanting to publish two projects with the same build command, each project output goes into a subfolder with the same name as the project, where the subfolders' parent is the publish path. So building a solution with projectA and projectB or naming them individually in the msbuild command along with a publish folder of c:\temp should put the output of each in c::\temp\projectA and c:\temp\projectB respectively.

[mmalloy-cnb]

It finally works!

I hope this helps someone else out; it did work for my pipeline, building multiple project with one command. After trying multiple configurations, this is what worked for me.

I added modifyOutputPath: true to the dotnet publish task and it seems to work now with -o <root directory> argument. I have not tested with dotnet build or other commands.

my YAML task:

- task: DotNetCoreCLI@2
  displayName: 'dotnet publish'
  inputs:
    command: 'publish'
    publishWebProjects: false
    projects: '**/*.csproj'
    arguments: '-p:Version=$(Version) -o $(Build.ArtifactStagingDirectory) --no-restore'
    zipAfterPublish: false
    modifyOutputPath: true    <------ add this

[kbutto]

^ would be nice to have something like this for Github Actions!

[jhudsoncedaron]

We have been publishing with "dotnet publish tmp.sln -c " + configuration + " -r " + rid + " --self-contained true -p:ShouldUnsetParentConfigurationAndPlatform=false"; however Microsoft has been for some time recommending to not do this via a .sln file. There is a Microsoft.Build.NoTargets that seems to be able to solve the problem but it does not do so. The resulting error trying to do that is "C:\Program Files\dotnet\sdk\6.0.202\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.ConflictResolution.targets(112,5): error NETSDK1152: Found multiple publish output files with the same relative path: C:\Users\jhudson\dev\formbuilder\src\Cedaron.FormBuilder.Presentation\appsettings.json"

[mmalloy-cnb]

We have been publishing with "dotnet publish tmp.sln -c " + configuration + " -r " + rid + " --self-contained true -p:ShouldUnsetParentConfigurationAndPlatform=false"; however Microsoft has been for some time recommending to not do this via a .sln file. There is a Microsoft.Build.NoTargets that seems to be able to solve the problem but it does not do so. The resulting error trying to do that is "C:\Program Files\dotnet\sdk\6.0.202\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.ConflictResolution.targets(112,5): error NETSDK1152: Found multiple publish output files with the same relative path: C:\Users\jhudson\dev\formbuilder\src\Cedaron.FormBuilder.Presentation\appsettings.json"

Ran into this myself, seems Microsoft updated the later SDKs (noticed it when moving to .NET6) to flag this error. I added the <ErrorOnDuplicatePublishOutputFiles>false</ErrorOnDuplicatePublishOutputFiles> property to a <PropertyGroup> in the .csproj file that was throwing the error and builds are successful.

[jhudsoncedaron]

@mmalloy-cnb : I don't think that's the correct solution. I need one publish directory for each project.

[mmalloy-cnb]

@jhudsoncedaron please refer to my previous reply above. That is what is working for me; where I have a solution containing multiple projects, which are published with one task into an artifact that contains each project output in their own folder. The key was to make sure to have modifyOutputPath: true as part of the tasks configuration.

[aaronr93]

From joelharkes

I made a script that builds the dependency trees for all projects and then executes builds in parallel (Promises nodejs) according to the builds. I got a major decrease in build/publish time (on a multi core environment).

@joelharkes this script you mentioned 5 years ago could be very useful to us. Do you mind sharing the script please, if you still have it?

We are publishing over a hundred projects that all share a web of 10-20 core dependencies. Publishing in parallel could decimate our deployment time.