Downgrade of host packages

[ericstj]

Steps to reproduce

Restore following project.json

{
  "dependencies": {
    "Microsoft.NETCore.App": "1.1.0-preview1-001069-00"
  },
  "frameworks": {
    "netcoreapp1.0": { }
  },
  "runtimes": {
    "win7-x64": { }
  }
}

Expected behavior

No warnings

Actual behavior

Warnings:

warn : Detected package downgrade: Microsoft.NETCore.DotNetHostResolver from 1.2.0-beta-000984-00 to 1.1.0-preview1-001069-00
warn :  testrestore (>= 1.0.0) -> Microsoft.NETCore.App (>= 1.1.0-preview1-001069-00) -> Microsoft.NETCore.DotNetHostPolicy (>= 1.1.0-preview1-001069-00) -> runtime.win7-x64.Microsoft.NETCore.DotNetHostPolicy (>= 1.1.0-preview1-001069-00) -> Microsoft.NETCore.DotNetHostResolver (>= 1.2.0-beta-000984-00)
warn :  testrestore (>= 1.0.0) -> Microsoft.NETCore.App (>= 1.1.0-preview1-001069-00) -> Microsoft.NETCore.DotNetHostPolicy (>= 1.1.0-preview1-001069-00) -> Microsoft.NETCore.DotNetHostResolver (>= 1.1.0-preview1-001069-00)
warn : Detected package downgrade: Microsoft.NETCore.DotNetHost from 1.2.0-beta-000984-00 to 1.1.0-preview1-001069-00
warn :  testrestore (>= 1.0.0) -> Microsoft.NETCore.App (>= 1.1.0-preview1-001069-00) -> Microsoft.NETCore.DotNetHostPolicy (>= 1.1.0-preview1-001069-00) -> Microsoft.NETCore.DotNetHostResolver (>= 1.1.0-preview1-001069-00) -> runtime.win7-x64.Microsoft.NETCore.DotNetHostResolver (>= 1.1.0-preview1-001069-00) -> Microsoft.NETCore.DotNetHost (>= 1.2.0-beta-000984-00)
warn :  testrestore (>= 1.0.0) -> Microsoft.NETCore.App (>= 1.1.0-preview1-001069-00) -> Microsoft.NETCore.DotNetHostPolicy (>= 1.1.0-preview1-001069-00) -> Microsoft.NETCore.DotNetHostResolver (>= 1.1.0-preview1-001069-00) -> Microsoft.NETCore.DotNetHost (>= 1.1.0-preview1-001069-00)

Copied from https://github.com/dotnet/corefx/issues/12114. /cc @pakrym

[gkhanna79]

@ericstj Can you please elaborate on what this warning means? 1.1.0 build does not have dependency on 1.2.0 DotNetHostResolver, so how is that coming into the mix?

[ericstj]

You'll need to investigate to determine what is happening. Perhaps your runtime.json has the incorrect value or there is a non-exact match and nuget is picking the "closest" which happens to be 1.2.0-*

[ericstj]

If you look at the lock file you can see what's happening.

NuGet's only ref to runtime.win7-x64.Microsoft.NETCore.DotNetHostPolicy is 1.1.0-preview1-001069-00, but NuGet downloaded 1.2.0-beta-000984-00.

If I check the feed for runtime.win7-x64.Microsoft.NETCore.DotNetHostPolicy I see that it is missing a number of builds when compared to Microsoft.NETCore.DotNetHostPolicy.

So it would seem that you're publishing incomplete builds. The versions repo claims that the build was consistent but for some reason the windows package wasn't published. https://github.com/dotnet/versions/commit/06175dc8977a503429497e9cca4065107b0ec5a4

/cc @dagood

[gkhanna79]

@dagood do you know why publishing will skip runtime* packages? I can confirm that the build is producing all of them.

[dagood]

Looking into this, I don't have a known reason for it.

[dagood]

I believe this was because the build infra wasn't compatible with the arm64 build. For now I've disabled the arm64 build (cc @wtgodbe) and made a dummy commit which triggered https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_build/index?buildId=351282.

The blob storage for 1.1.0-preview1-001071-00 for example has these Windows badges:

sharedfx_Windows_x64__version_badge.svg
sharedfx_Windows_x64_Release_version_badge.svg
sharedfx_Windows_x86_Release_version_badge.svg 

The logic in https://github.com/dotnet/core-setup/blob/aa07f3c166977c9adea8e2ce50c43577fae9d74f/build_projects/dotnet-host-build/PublishTargets.cs#L176 uses StartsWith, and that sharedfx_Windows_x64__version_badge.svg badge messes with the build's ability to tell if the build is done yet. I'm not 100% sure why we see the exact results we do, this is a big problem regardless.

To fix this, we need to update https://github.com/dotnet/core-setup/blob/release/1.1.0/build_projects/Microsoft.DotNet.Cli.Build.Framework/CurrentArchitecture.cs to detect arm64, and I'm guessing add arm64 to https://github.com/dotnet/core-setup/blob/release/1.1.0/build_projects/Microsoft.DotNet.Cli.Build.Framework/Enumerations/BuildArchitecture.cs.

[dagood]

Once arm64 is supported we can revert https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_git/DotNet-BuildPipeline/pullrequest/38537?path=%2Fsrc%2FPipeBuild%2FPipeBuild%2Fpipelines.core-setup.json&fullScreen=false&_a=overview to turn arm64 builds back on.

[gkhanna79]

I believe we need to update https://github.com/dotnet/core-setup/blob/aa07f3c166977c9adea8e2ce50c43577fae9d74f/build_projects/dotnet-host-build/PublishTargets.cs#L178 Dictionary as well to account for WinArm64 build.

Likewise, the list at https://github.com/dotnet/core-setup/blob/aa07f3c166977c9adea8e2ce50c43577fae9d74f/build_projects/dotnet-host-build/PublishTargets.cs#L114.

@ianhays PTAL at this - assuming the disabling of arm64 build comes through, we need this fixed for 1.1.0 RTM. Also, I presume this fix will need to be in master as well.

[dagood]

The build just succeeded that had arm64 disabled, and https://github.com/dotnet/versions/blob/10d7042c096cee2e458884cb7d049d9f178f75ab/build-info/dotnet/core-setup/release/1.1.0/Latest_Packages.txt looks good to me. Once MyGet warms up (max ~10 minutes after packages were published), we should be able to have the restore succeed.

[gkhanna79]

Agreed - this looks much better!

@dagood Can you please confirm that Azure blob has all the right artifacts?

[dagood]

Yes, here's the full list of badges for 1.0.0-preview1-001075-00:

sharedfx_CentOS_x64_Release_version_badge.svg
sharedfx_Debian_x64_Release_version_badge.svg
sharedfx_Fedora_23_x64_Release_version_badge.svg
sharedfx_openSUSE_13_2_x64_Release_version_badge.svg
sharedfx_openSUSE_42_1_x64_Release_version_badge.svg
sharedfx_OSX_x64_Release_version_badge.svg
sharedfx_RHEL_x64_Release_version_badge.svg
sharedfx_Ubuntu_16_04_x64_Release_version_badge.svg
sharedfx_Ubuntu_16_10_x64_Release_version_badge.svg
sharedfx_Ubuntu_x64_Release_version_badge.svg
sharedfx_Windows_x64_Release_version_badge.svg
sharedfx_Windows_x86_Release_version_badge.svg

[ianhays]

So it looks like the version badge creation is using the current architecture instead of the target architecture, so during the finalize step it's thinking we're done because we have the win10-arm64 badge labeled as an x64 badge.

We should be able to fix this by using TargetArch instead of CurrentArch when the badge name is determined. We'll also need to add the arm64 badge to the expected badges lists linked above.

[ianhays]

I'll put up a PR in a few but it's not really locally testable so we'll just have to re-enable the arm64 build and fire off a run to see what happens.

[gkhanna79]

@ianhays Please test the fix on master first and then bring it over to 1.1.0.

[gkhanna79]

@ianhays I am reopening this until both the following are true:

1. You have validated that the blob storage for master is looking as expected. You czn work with @dagood to confirm the same.
2. Post 10/10, we need to port this change to 1.1.0 branch as well. Until then, Arm64 builds are disabled in 1.1.0 branch and once ported, work with @dagood to enable them again.

[ianhays]

for release/1.1.0, this was resolved in https://github.com/dotnet/core-setup/pull/394

The pipebuild for these changes passed without issue: https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_build/index?buildId=395486&_a=summary

The x86 build triggered by the pipebuild successfully uploaded the packages as 1139-00.

And finally, blob storage for 1139-00 looks as expected.

Closing.