search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.16k stars 4.71k forks source link

build.sh on OS X has problems with package restore #15533

Closed adityamandaleeka closed 4 years ago

adityamandaleeka commented 9 years ago

When running build.sh on a Mac, I'm seeing a hang during package restore. I've built corefx successfully on this Mac before, but it's been at least a few weeks since I last tried.

./build.sh clean
Setting up directories for build
WARNING: Mono 4.0.1.44 or later is required to build corefx. Unable to assess if current version is supported.
Restoring NuGet.exe...
Restoring MSBuild...
Attempting to resolve dependency 'Microsoft.Tpl.Dataflow (≥ 4.5.24)'.
Installing 'Microsoft.Tpl.Dataflow 4.5.24'.
Successfully installed 'Microsoft.Tpl.Dataflow 4.5.24'.
Installing 'Microsoft.Build.Mono.Debug 14.1.0.0-prerelease'.
Successfully installed 'Microsoft.Build.Mono.Debug 14.1.0.0-prerelease'.
  Restoring build tools...
  Restoring all packages...

It was stuck on that part for a long time, and eventually, I got some timeout errors like the following (this is a small sampling of them):

Error: DownloadPackageAsync: https://www.myget.org/F/dotnet-core/api/v2/package/runtime.win7.System.IO.Pipes/4.0.0-beta-23328
    Error getting response stream (InitRead): ReceiveFailure
  Error: DownloadPackageAsync: https://www.myget.org/F/dotnet-core/api/v2/package/runtime.win7.System.Net.Http/4.0.1-beta-23328
EXEC : error : SendFailure (Error writing headers) [/Users/adityam/code/corefx/build.proj]
  Error: DownloadPackageAsync: https://www.myget.org/F/dotnet-core/api/v2/package/runtime.win7.System.IO.FileSystem.Watcher/4.0.0-beta-23328
    HTTP request timed out. Exiting.
  Error: DownloadPackageAsync: https://www.myget.org/F/dotnet-core/api/v2/package/runtime.win7.System.IO.MemoryMappedFiles/4.0.0-beta-23328
    HTTP request timed out. Exiting.
  Error: DownloadPackageAsync: https://www.myget.org/F/dotnet-core/api/v2/package/runtime.win7.System.Net.Requests/4.0.11-beta-23328
    HTTP request timed out. Exiting.
  Error: DownloadPackageAsync: https://www.myget.org/F/dotnet-core/api/v2/package/runtime.win7.System.Net.Primitives/4.0.11-beta-23328
    HTTP request timed out. Exiting.

I then tried manually running the same dnu restore command that the script is trying to run. The first time I tried this, it seemed to download all the packages successfully, but then I ran into problems like the following when running build.sh:

 Restoring all packages...
/corefx/packages/Microsoft.Build.Mono.Debug.14.1.0.0-prerelease/lib/Microsoft.CSharp.CurrentVersion.targets(383,9): error MSB6006: "csc2.exe" exited with code 1. [/corefx/src/Microsoft.CSharp/ref/Microsoft.CSharp.csproj]
/corefx/packages/Microsoft.Build.Mono.Debug.14.1.0.0-prerelease/lib/Microsoft.CSharp.CurrentVersion.targets(383,9): error MSB6006: "csc2.exe" exited with code 1. [/corefx/src/Microsoft.VisualBasic/ref/Microsoft.VisualBasic.csproj]

I've also tried deleting everything and starting from scratch and this time, I got some request timeouts when running the dnu command manually (and it still hangs on package restore when running build.sh).

Is this a problem with myget or could there be something else going wrong?

kapilash commented 9 years ago

AFAICT, the issue here seems to be with mono especially when there are a lot of dependencies to download.
There's a work around using corefx dnu.

For the current build-tool version and assuming that $HOME/src/corefx is the location of corefx sources, the following fetches all the dependencies

 dnu restore  --packages "$HOME/src/corefx/packages" --source https://www.myget.org/F/dotnet-core/ --source https://www.myget.org/F/dotnet-coreclr/ --source https://www.myget.org/F/dotnet-corefxtestdata/ --source https://www.myget.org/F/dotnet-buildtools/ --source https://www.nuget.org/api/v2/ "$HOME/src/corefx/src" "$HOME/src/corefx/packages/Microsoft.DotNet.BuildTools.1.0.25-prerelease-00107/lib/"

Run that and proceed to run build.sh and it will go through. After the build, Project.lock.jsons have to git checked out, tho.

ellismg commented 9 years ago

The major change recently is I think we are now using the ToolRuntime for running GenFacades. This requires us to restore the tool runtime, which means we have to pull down a lot of packages and dnu on top of mono is not great at that (as @kapilash says).

We should look in adopting a CoreCLR based DNU for our package restore needs. On Linux this can lead to some bootstrapping problems (if you are on a distro we don't have packages for), but it should be straight-forward for OSX.

This should be part of our long term goal to stop needing mono for our builds.

akoeplinger commented 9 years ago

You can usually workaround that problem by setting a MONO_THREADS_PER_CPU=50 env var.

joshfree commented 8 years ago

Related https://github.com/dotnet/corefx/issues/5021

mconnew commented 8 years ago

I have just submitted a PR aspnet/dnx#3298 to fix this issue to the dnx repo. Once that's done, a newer package version will need to be used. I've tested this with WCF as we have nearly 300 packages needing to be downloaded and we were hitting this pretty much every clean build.

joperezr commented 8 years ago

@adityamandaleeka can you still repro this? given that we now use dotnet cli to do the package restore, this shouldn't happen any more. Can you please verify if this repros for you with latest code in corefx? If there is no repro, then we can close this issue.

citizenmatt commented 8 years ago

I can still repro this. Pull latest today, run .build.sh and it hangs at Restoring all packages…. I've tried the dnu restore command above, and it runs through ok, but still hangs. Any ideas how to progress?

ellismg commented 8 years ago

@citizenmatt The package restore step actually takes a long while now :-(. You might try increasing the verbosity to the console and see if 'dotnet restore' is running.

It is expected that the restore will take ~20 mins.

citizenmatt commented 8 years ago

Wow. Didn't realise that. I did the dnu restore step, which seemed to complete ok (and much quicker than 20 mins!), then ran .\build.sh /p:RestoreOnBuild=false, which seems to be working - it's running through a load of tests right now.

ellismg commented 8 years ago

You may run into errors. Restore takes so long because we have to invoke dotnet restore (which wraps DNX today) hundreds of times to work around bugs in how DNX selects what project.json's to restore.

If you run into breaks, let the restore during build do it's thing once.

dotnet/corefx#5507 tracks making restore faster.

Note when building cross platform we can't use the -m option with MSBuild so the build will take a long time (for me it was taking ~2 hours on my VM, and that was with me skipping running the tests).

citizenmatt commented 8 years ago

For anyone else who might be hitting this, I got a clean build with sh ./build.sh, but restoring packages still took a long time.

ILyoan commented 8 years ago

@citizenmatt FWIW, I was caught in similar situation and in my case ulimit -n 2048 solved the problem.

joperezr commented 8 years ago

Ok, since it looks like the restore is no longer failing I will go ahead and close this issue. I completely agree that package restore takes too long, so thanks a lot @ellismg for building the wrapper that will make package restore way more efficient :+1: