dotnet-install with -SharedRuntime is failing

rynowak commented 7 years ago

Steps to reproduce

cd scripts\obtain .\dotnet-install.ps1 -SharedRuntime -InstallDir .dotnet -Channel master -Architecture x64

Expected behavior

Downloads the shared runtime

Actual behavior

Failed to download https://dotnetcli.blob.core.windows.net/dotnet/master/dnvm/latest.sharedfx.win.x64.version.
StatusCode: 404, ReasonPhrase: 'The specified blob does not exist.', Version: 1.1, Content:
System.Net.Http.StreamContent, Headers:
{
  x-ms-request-id: 6e0d6dd8-0001-0022-07ee-c408b3000000
  x-ms-version: 2009-09-19
  Date: Thu, 04 May 2017 15:53:03 GMT
  Server: Windows-Azure-Blob/1.0
  Server: Microsoft-HTTPAPI/2.0
  Content-Length: 215
  Content-Type: application/xml
}
At D:\dotnet\cli\scripts\obtain\dotnet-install.ps1:163 char:13
+             throw $ErrorMsg
+             ~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Failed to downl...lication/xml
}:String) [], RuntimeException
    + FullyQualifiedErrorId : Failed to download https://dotnetcli.blob.core.windows.net/dotnet/master/dnvm/latest.sha
   redfx.win.x64.version.  StatusCode: 404, ReasonPhrase: 'The specified blob does not exist.', Version: 1.1, Content
  : System.Net.Http.StreamContent, Headers:
{
  x-ms-request-id: 6e0d6dd8-0001-0022-07ee-c408b3000000
  x-ms-version: 2009-09-19
  Date: Thu, 04 May 2017 15:53:03 GMT
  Server: Windows-Azure-Blob/1.0
  Server: Microsoft-HTTPAPI/2.0
  Content-Length: 215
  Content-Type: application/xml
}

Environment data

N/A

livarcocc commented 7 years ago

Use release/1.1.0. We have a separate issue already tracking making this better.

rynowak commented 7 years ago

@livarcocc What does that mean? use release/1.1.0 where? Is that the channel?

michellemcdaniel commented 7 years ago

For a lot of our testing, we really need to have access to dotnet cli at head of master (or whatever the most recent master branch shared framework drop we have is). Many different dev branches of products (jitbench, optimization work, etc) relies on being able to run the dotnet-install script that we have to download the most recent master version of the shared framework. Using release/1.1.0 doesn't cut it for most of our dev situations. Can someone please look into getting the .version files back? Is there another way to know what the most recent version of the cli is that we can download from blob storage?

ryanbrandenburg commented 7 years ago

I ran into this issue on aspnet/Razor and now aspnet/Mvc as well (though I haven't yet figured out the exact steps which got me there.). It seems to have just broken aspnet/Korebuild in general.

livarcocc commented 7 years ago

@adiaaida The CLI uses the master channel, however, the runtime does not. The runtime uses release/1.1.0 channel for its latest (2.0) releases. So, if you are installing the runtime, use channel release/1.1.0. If you want to install the latest CLI, use master. Though, note that the preview1 CLI is shipped out of the release/2.0.0 channel.

If you want the runtime to start using master as their channel, please, file an issue under dotnet/core-setup.

rynowak commented 7 years ago

It looks like the real issue that I'm having is that they changed the path in blob storage from: https://dotnetcli.blob.core.windows.net/dotnet/master/dnvm/latest.sharedfx.win.x64.version TO https://dotnetcli.blob.core.windows.net/dotnet/master/dnvm/latest.sharedfx.win-x64.version

https://github.com/dotnet/core-setup/pull/2282

Notice the win.x64 is now win-x64.

I'm really only going on about this here because this script is now outdated. If there's a more appropriate place to discuss this please redirect me.

/cc @eerhardt

eerhardt commented 7 years ago

This is fixed in the latest. See https://github.com/dotnet/core-setup/pull/2282.

official master builds are blocked right now. I'm working on fixing them.

rynowak commented 7 years ago

Oh I see, you are unbreaking it. Thanks, this has been causing me some consternation.

eerhardt commented 7 years ago

Yes, sorry I was too short. In core-setup/master, we accidentally changed the URL format. But that PR I referenced above is putting the URL format back to the way it originally was.

However, the latest build doesn't have this fix yet because of a different break that is blocking getting new builds out.

JosephTremoulet commented 7 years ago

Did the format of the .version file change also, and will it be changed back in the next official build? If I open https://dotnetcli.blob.core.windows.net/dotnet/master/dnvm/latest.sharedfx.win-x64.version, it contains just the version string, but dotnet-install.ps1 expects the hash as well. The script also doesn't expect content-type "text/plain; charset=utf-8". So is dotnet-install.ps1 still broken?

eerhardt commented 7 years ago

Did the format of the .version file change also, and will it be changed back in the next official build?

Yes, and no.

Yes, it appears there are still 3 issues that we need to fix.

the old version files didn't have "-" in their URLs.
we are not putting the CommitHash in the version files
The content-type changed.

The first 2 need to be fixed in core-setup. I'm not sure on the fix for the third yet. We can try to fix it in core-setup, or we can change the dotnet-install script to accept the new content-type.

rynowak commented 7 years ago

thanks @eerhardt - so to clarify, the dotnet-install scripts should keep working as is? When should we see a build that's fixed available?

eerhardt commented 7 years ago

That's the hope/intent is to keep the install script as-is.

One place that will need to be changed in the install script on Linux is that we are no longer producing distro-specific tar.gz files. Instead, we have a single Linux-x64.tar.gz that will work on all supported Linux distros.

rynowak commented 7 years ago

is there an issue tracking this?

eerhardt commented 7 years ago

I've opened https://github.com/dotnet/cli/issues/6538 to track the linux distro issue.

I've verified that on the latest 'master' build of the shared framework now works again on windows:

PS F:\cli\scripts\obtain> .\dotnet-install.ps1 -SharedRuntime -Channel master
dotnet-install: Downloading https://dotnetcli.azureedge.net/dotnet/master/Binaries/2.0.0-preview2-25306-06/dotnet-win-x6
4.2.0.0-preview2-25306-06.zip
dotnet-install: Extracting zip from https://dotnetcli.azureedge.net/dotnet/master/Binaries/2.0.0-preview2-25306-06/dotne
t-win-x64.2.0.0-preview2-25306-06.zip
dotnet-install: Adding to current process PATH: "C:\Users\eerhardt\AppData\Local\Microsoft\dotnet\". Note: This change w
ill not be visible if PowerShell was run as a child process.
dotnet-install: Installation finished

dotnet / sdk