dotnet publish is slow

[cmeyertons]

Steps to reproduce

Create a small console app and add some NuGet packages. Run dotnet publish to publish app

Expected behavior

The publish step should perform roughly similar to a robocopy operation on newer files

Actual behavior

Dotnet publish is considerably slower

Environment data

dotnet --info output:

[livarcocc]

dotnet publish will never work on the same perf level as robocopy as it does a lot more than a robocopy would.

We need to evaluate the project, references etc, run a bunch of targets, kickoff multiple processes (dotnet, dotnet msbuild, etc).

We have done substantial to improve the performance of what we consider inner loop dev operations, like dotnet run, dotnet build but we don't consider dotnet publish part of the inner loop cycle.

As such, we don't have any plans in the near future to look at dotnet publish performance. Also, it would be interesting to know which dotnet version you are using to know if you are already benefiting from the inner loop improvements I mentioned above or not. dotnet publish is kind of a superset of dotnet build and any dotnet build perf gains would also benefit dotnet publish.

Unless you have some scenarios where dotnet publish performance is impacting you badly, we will likely close this issue.

[cmeyertons]

Hmm good to understand the prioritization.

Yeah I understand the entire process being longer because it has to include the build. That makes sense.

I guess I was hoping the publish portion (post-build) would be on par with a copy / web deploy activity, but it is considerably slower.

I guess that means going back to dotnet build + some optimized copy mechanism (robocopy) is the preferred approach for those not only developing locally, bummer.

[turowicz]

A lot of people now use Docker, where the publish is only meant to package the app.

The following takes really long time:

RUN dotnet publish src/${project}/${project}.csproj -c Release -o /app/out/ --no-restore --no-build

[turowicz]

@livarcocc please note the restore and build are disabled.

[cmeyertons]

I would love a reopen as well. If the publish is a part of the dev inner loop, wouldn’t you want that step to be optimized as well?

And to second @turowicz, it has nothing to do with the build and restore, only with the publish portion

[dasMulli]

A lot of people now use Docker, where the publish is only meant to package the app.

Is this spinning up a new container for each build? Not sure if I/O perf is on par when running docker-for-windows with linux containers vs. native I/O / containers on linux. Also, a lot of the inner loop optimizations relies on processes hanging around between builds (msbuild, c# compiler, razor compiler) and just being reused (dotnet build-server shutdown would be needed to stop them). If you're always building a container, it wouldn't benefit from any of these changes.

[turowicz]

@dasMuli I use Unix both as my development computer, and on the CI server.

I'm running docker build -f <MyDockerFile> which then calls the RUN dotnet publish (...).

[xyfy]

I am using a linux container to publish a netcore project.

It is very slow when run dotnet publish,it took may ten minutes and above

my docker file is like below

FROM yungongchang/dotnet3.1-base AS base
WORKDIR /app
EXPOSE 80

FROM yungongchang/dotnet3.1-sdk-withnodejs AS build

WORKDIR /src
COPY . .
WORKDIR "/src/applications/Yun.AuthServer.Host"

RUN dotnet restore "Yun.AuthServer.Host.csproj" -nowarn:msb3202,nu1503

RUN cnpm install
RUN gulp

RUN dotnet build "Yun.AuthServer.Host.csproj" --no-restore -c Release -o /app

FROM build AS publish
RUN dotnet publish "Yun.AuthServer.Host.csproj" --no-restore -c Release -o /app

FROM base AS final
WORKDIR /app

COPY --from=publish /app .
ENV PYTHONUNBUFFERED definitely
ENTRYPOINT ["dotnet", "Yun.AuthServer.Host.dll"]

It will be stop in Yun.AuthServer.Host -> XXXX/Yun.AuthServer.Host.Views.dll. Get stuck and wait a dozen minutes or more to finish it

when I exec the command on windows,it is very quickly.

it is slowly in both dev PC and Server

[dasMulli]

@xyfy seint that the app seems to need nodejs, is this an app containg a client side application? If so, publish may also trigger tasks like npm install, angular / react / * builds etc. which could take a while...

[xyfy]

but I'm  very fast on windows  publish

---Original--- From: "Martin Andreas Ullrich"<notifications@github.com> Date: Mon, Jun 15, 2020 19:33 PM To: "dotnet/sdk"<sdk@noreply.github.com>; Cc: "Mention"<mention@noreply.github.com>;"xyfy"<benguahao@foxmail.com>; Subject: Re: [dotnet/sdk] dotnet publish is slow (#10182)

@xyfy seint that the app seems to need nodejs, is this an app containg a client side application? If so, publish may also trigger tasks like npm install, angular / react / * builds etc. which could take a while...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[GeraudFabien]

I believe there is a bug since a publish on my machine take less than a minute a publish on docker with docker compose take a lot of time (IDK the only time i wait enough time to end i was afk for 40 minute ...) and most of the time just timeout.

Some on so seem's to raise the same problem. https://stackoverflow.com/questions/57669467/dotnet-restore-incredibly-slow-inside-docker-compose-build

[InfinitiesLoop]

I am also experiencing this. I use docker to build, then publish a console application. The publish call takes ~10 seconds on my dev windows machine, but it takes >30 MINUTES on a micro VM. During that time the machine is practically idle. It gets stuck in between two projects, as if one particular project is the culprit (which is not the same one as publish was called on). I totally expect terrible performance on a micro VM, but this is not just performance, it's something gone terribly wrong....

[GeraudFabien]

I forget to explain why i put mine as resolved. It's a issue with docker (for my case) not related at all with dotnet.

I don't know why it use 9Go even if all the container didn't even use 2Go (It was using 97+% of ram) (We have the problem only on docker WSL2). That why it was slow on my machine? Knowing that i launch one instance of VS and close my web browser before running and it work fine (I try to be around 87% and i identify on windows that it start getting slow at 90-93% of ram usage).

[eduardo-salazar]

@xyfy where you able to solve this issue. I am facing the same issues.

[xyfy]

it is Perhaps  for *.view.dll.something  complied  for razor  pages发自我的手机-------- 原始邮件 --------发件人： Eduardo Salazar @.>日期： 2022年3月18日周五 02:11收件人： dotnet/sdk @.>抄送： xyfy @.>, Mention @.>主 题： Re: [dotnet/sdk] dotnet publish is slow (#10182) @xyfy where you able to solve this issue. I am facing the same issues.

—Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.Message ID: @.***>

[RobWhitmore]

I am experiencing significantly longer Pipeline build times in Azure DevOps since migrating an app from .NET 6 to 7. The logs show details of the dotnet publish step. The most significant consumer is the "Optimizing assemblies for size" step.

Here is a 36m 13s example from the log: 2023-03-05T22:32:59.7705951Z Optimizing assemblies for size. This process might take a while. 2023-03-05T23:09:12.4221508Z Optimizing assemblies for size may change the behavior of the app. Be sure to test after publishing. See: https://aka.ms/dotnet-illink

A summary of Azure DevOps Pipeline builds highlighting the Optimize step consumption (dates are in local TZ): Pipeline 72 build.ps1 Date: 04-11-2022 Ver 2.210.0 SDK Ver 6.0.402 No Optimize step Total Duration: 5m 38s Pipeline 83 build.ps1 Date: 09-02-2023 Ver 2.212.0 SDK Ver 7.0.102 Optimize 9m 35s Total Duration: 17m 31s Pipeline 86 build.ps1 Date: 06-03-2023 Ver 2.212.0 SDK Ver 7.0.200 Optimize 36m 12s Total Duration: 44m 41s Pipeline 87 build.ps1 Date: 08-03-2023 Ver 2.212.0 SDK Ver 7.0.200 Optimize 27m 4s Total Duration: 35m 40s Pipeline 88 build.ps1 Date: 09-03-2023 Ver 2.212.0 SDK Ver 7.0.200 Optimize 17m 42s Total Duration: 23m 50s Pipeline 90 build.ps1 Date: 09-03-2023 Ver 2.212.0 SDK Ver 7.0.201 Optimize 19m 54s Total Duration: 27m 18s Pipeline 91 build.ps1 Date: 15-03-2023 Ver 2.212.0 SDK Ver 7.0.201 Optimize 11m 59s Total Duration: 19m 47s Pipeline 92 build.ps1 Date: 20-03-2023 Ver 2.212.0 SDK Ver 7.0.201 Optimize 20m 26s Total Duration: 26m 40s

I ran the build.ps1 script locally for the same source as Pipeline 92, but there is no breakdown for the Optimize step locally: Local run of 92 build.ps1 Date: 20-03-2023 dotnet publish 11m 39s Total Duration: 12m 18s

[RobWhitmore]

Here is the .NET 7 parameter default setting, breaking change, which extends build times, while fully minimising the app size. The trade-off between build time and app size is influenced by the TrimMode parameter: <TrimMode>partial</TrimMode> vs <TrimMode>full</TrimMode> full is the default = no params. Using partial made a 26% improvment to my build duration and no difference to the built app zip size.

• No Parameters: Publish 13m 58s cbapp.zip is 90,850,808 bytes Total Duration: 14m 40s
• TrimModePartial: Publish 11m 1s cbapp.zip is 90,850,808 bytes Total Duration: 11m 40s

Setting <PublishTrimmed>false</PublishTrimmed> significantly reduced the duration and added 17% to the app zip file size:

• PubTrimFalse Publish 0m 18s cbapp.zip is 106,493,843 bytes Total Duration: 1m 7s

For Production and overnight builds, Trimming is the way to go, but for rapid builds during the day no trimming appears useful https://learn.microsoft.com/en-us/dotnet/core/compatibility/deployment/7.0/trim-all-assemblies Individual assemblies can be set to opt-in for trimming via parameters: https://learn.microsoft.com/en-us/dotnet/core/deploying/trimming/trimming-options?pivots=dotnet-7-0

[RobWhitmore]

Here is my modified Document.Build.props file modified to react to a buildTrim = true setting of the DevOps Pipeline variable buildTrim which I added the Pipeline.