search

NuGet / Home

Repo for NuGet Client issues
Other
1.5k stars 253 forks source link

dotnet restore hangs indefinitely #5583

Closed davkean closed 7 years ago

davkean commented 7 years ago

From @richlander on July 3, 2017 20:57

See:

/cc @rrelyea

Copied from original issue: dotnet/core#724

davkean commented 7 years ago

From @MathiasKowoll on July 3, 2017 22:45

I looked that while having many libs in the folder wwwroot\lib dotnet restore takes a while to restore the files, If I remove that folder it restores immediately.

This happens with al dotnet commands. CLI and Visual Studio 2017 Preview 3

davkean commented 7 years ago

From @dsyme on July 4, 2017 22:22

I put a full process dump of a hung dotnet.exe here: https://1drv.ms/u/s!Aq5gXOPBI5HdiroRLPy1hDY3yK80kQ

C:\GitHub\misc>dotnet --version
2.0.0-preview2-006497
davkean commented 7 years ago

From @dsyme on July 4, 2017 23:5

I eventually realised that I had a very large number of files in the subdirectories under the c:\misc in which I was running "dotnet". The c:\misc directory itself contained just the results of a dotnet new. The subdirectories included Git repos and much more - 20,000 files and 1.6GB

Now, it's my belief that the presence of these files shouldn't have caused problems for a simple dotnet restore but it seems they did.

I tried to procmon.exe the activity of dotnet.exe but the trace was large. The screenshot below showed that it is indeed accessing all the subdirectories, In this case, it seemed like the thing just never terminated, I don't know what it was doing though o all those blocked threads

capture

Anyway, I think the bug here is that running dotnet.exe in a folder containing 1GB of misc files in subdirectories either takes a very long time or simply doesn't terminate

Now I know that I can get on with the rest of my work :)

davkean commented 7 years ago

From @Petermarcu on July 7, 2017 7:43

@rrelyea should this get moved to the nuget repo to get tracked?

davkean commented 7 years ago

From @Lanayx on July 7, 2017 9:1

I got the same issue. For me the problem was that new dotnet restore (from dotnet-sdk-2.0.0-preview2-006497-win-x64) ignores disabledPackageSources configuration in NuGet.Config, and I had some of them unreachable. Removing these sources helped.

davkean commented 7 years ago

From @richlander on July 7, 2017 13:53

@Petermarcu I was going to make @ https://github.com/NuGet/NuGet.Client but issues are not enabled.

davkean commented 7 years ago

From @Petermarcu on July 7, 2017 15:24

@richlander you make it on the nuget/home repo

davkean commented 7 years ago

From @forki on July 7, 2017 18:0

Same issue here: https://github.com/fable-compiler/Fable/issues/1042

davkean commented 7 years ago

From @rrelyea on July 7, 2017 18:22

Working on creating NuGet/home issues ... will link when created. Update: Actually, I lied...let's drill into these 2 issues a bit first...

davkean commented 7 years ago

From @rrelyea on July 7, 2017 18:24

@dsyme - in dotnet cli v1 previews, we used to walk all the sub directories. I had thought we changed that when we switched to make restore recursive (across project references) by defeault. do you have the same problem if you do a "dotnet restore --no-dependencies" in that same directory?

davkean commented 7 years ago

From @rrelyea on July 7, 2017 18:25

@MathiasKowoll - can you please give me repro steps for your problem (full details), and how you avoided it.

davkean commented 7 years ago

From @rrelyea on July 7, 2017 18:34

@forki - I am unable to repro. after you have done dotnet new, a restore should have already been completed. when you do a build, it will also do a restore, if it deems that one is necessary. Can you please file an issue in NuGet/Home about the issue you are seeing, with repro steps (including setup steps if necessary?

davkean commented 7 years ago

From @MathiasKowoll on July 7, 2017 18:49

@rrelyea create mvc app with individual authentication.

replace bower.sjon with the following.

{ "name": "ItsomaxCms", "private": true, "dependencies": { "jquery": "^3.2.1", "jquery-ui": "1.12.1", "bootstrap": "^3.3.7", "font-awesome": "4.3.0", "jquery-validation": "1.16.0", "jquery-validation-unobtrusive": "3.2.6", "datatables.net": "^1.10.15", "datatables.net-bs": "^2.1.1", "datatables.net-buttons": "^1.3.1", "datatables.net-buttons-bs": "^1.3.1", "datatables.net-fixedheader": "^3.1.2", "datatables.net-fixedheader-bs": "^3.1.2", "datatables.net-keytable": "^2.2.1", "datatables.net-responsive": "^2.1.1", "datatables.net-responsive-bs": "^2.1.1", "datatables.net-scroller": "^1.4.2", "datatables.net-scroller-bs": "^1.4.2", "pdfmake": "^0.1.29", "echarts": "^3.6.1", "smartwizard": "latest", "fastclick": "latest", "nprogress": "latest", "ionicons": "latest", "slim-scroll": "latest", "icheck": "latest", "switchery": "latest", "bootstrap-table": "latest", "tableExport.jquery.plugin": "latest", "x-editable": "latest" }, "resolutions": { "jspdf": "1.1.239 || 1.3.2" } }

make bower install

run dotnet restore.

davkean commented 7 years ago

From @forki on July 7, 2017 18:51

@rrelyea thanks for looking into this. I notified the original author of the issue. We have multiple people reporting this on the fable gitter chat. I assume it's related to node_modules and dotnet restore iterating long time over it

davkean commented 7 years ago

From @forki on July 7, 2017 19:24

Seems that things worked with preview1 see https://github.com/fable-compiler/Fable/issues/1042

forki commented 7 years ago

I don't think it's nuget issue since we also see slow behavior in dotnet build.

mikkeljohnsen commented 7 years ago

If I install "kendo-ui-core" through bower "dotnet build" will hang indefinitely.

If I delete "wwwroot/lib/kendo-ui-core/" it will build again.

Please fix this ASAP, so we can get on with development.

mishra14 commented 7 years ago

Tagging @emgarten for restore visibility.

@forki @mikkeljohnsen : Are you seeing an indefinite hang in dotnet restore as well?

mikkeljohnsen commented 7 years ago

@mishra14 dotnet restore is not indefinitely but takes about 5 minutes. and dotnet build is also not indefinitely from the command line it takes about 5 minutes as well. (did not wait that long before)

forki commented 7 years ago

The same is happening on azure now. This issue spreads.https://github.com/dotnet/core/issues/724#issuecomment-314487750

emgarten commented 7 years ago

This appears to be an issue with msbuild file globbing in the project on dotnet CLI. MSBuild in desktop does not have this issue. I also don't see any changes to the SDK around the props including these.

Using the bower.json file above here are the numbers I am getting:

Scenario Time
dotnet msbuild /t:clean 56s
dotnet build --no-restore 60s
dotnet restore 167s
dotnet restore /p:EnableDefaultItems=false 2s
(desktop) msbuild /t:restore 3s

Restore appears to make the existing problem worse, there may be a target that is changing properties and causing things to be evaluated additional times. I'll take a look at what can be improved there. The main issue appears to be the time globbing takes, the project should not take this long to evaluate just to return the project and package references that restore needs.

Workaround

dotnet restore /p:EnableDefaultItems=false

EnableDefaultItems=false will disable globbing for project directory files during restore.

mikkeljohnsen commented 7 years ago

EnableDefaultItems=false. Will also disable all items in VS (Mac). So not really useful.

And VS (Mac 7.1 build 1281) Still hangs on "Run" (Build)

emgarten commented 7 years ago

@mikkeljohnsen EnableDefaultItems=false should only be used with restore where the items are not needed. For build you will of course still need them, there you can run with --no-restore to speed things up.

forki commented 7 years ago

@emgarten is there a fix to see on the horizon?

emgarten commented 7 years ago

@forki not yet, it is still being investigated. I'll keep you posted.

mikkeljohnsen commented 7 years ago

@emgarten "dotnet build --no-restore" still takes more than 1 minute for a simple project with kendo-ui-core installed. And from VS (Mac) it is not able to compile at all.

Hope there will be a fix very soon. Because only solution is to downgrade to preview1

dsplaisted commented 7 years ago

I think I've figured out what is causing this, and you should be able to workaround it for now by setting the SetLinkMetadataAutomatically property to false in your projects.

emgarten commented 7 years ago

@dsplaisted is there an issue tracking the work needed for this? Is it going to be on the SDK side?

mikkeljohnsen commented 7 years ago

@dsplaisted Thanks that works.

emgarten commented 7 years ago

This issue was moved to dotnet/sdk#1434