Replicate tarball's CI (build and smoke-test) for VMR-lite

[premun]

Context

Presently, we source-build and run a set of E2E smoke tests over the tarball in installer's PR and for installer's main. The runs are run for a matrix of Linux distributions and various build parameters (using previous SDK, poisoning, comparing with MSFT SDK, (non-)bootstrapping...

To replace the tarball, we need to have the same set of builds ready for the VMR-lite.

Goal

• [x] PR builds of dotnet/installer build/test the VMR-lite
• [x] Official source-build of the VMR happens for each of its commits. This build potentially produces artifacts consumed by future builds.

[premun]

I made good progress on the pipelines. I'd like to validate where I'm heading before the next step.

I decided to develop this within a single build (installer-official-ci) so that I can iterate fast but it should be easy to split out again. I have compared errors of a build (not the latest but from a previous) and it matched the latest main' tarball build exactly.

Example build: https://dev.azure.com/dnceng/internal/_build/results?buildId=2053162&view=results

Questions / requests:

1. I imagine we don't want to run it within the installer's official build as it would slow down the rolling build? It should be a separate pipeline?
2. Can you please check the matrix definition, if this shape would work for you? It is an improvement in the sense that all legs are together now (even the arm ones) but it is a regression in the sense that some parameters are static and the values are repeated the same for each of the copies. Unfortunately, I was not able to use strategy/matrix as variables from these cannot be used as parameter inputs for templates.. I was also able to bundle/merge some associated parameters. https://dev.azure.com/dnceng/internal/_git/dotnet-installer?path=%2Feng%2Fpipelines%2Ftemplates%2Fstages%2Fvmr-source-build.yml&version=GBprvysoky%2Fvmr-source-build&_a=contents

Next steps

The only way I can think of linking a VMR SHA corresponding with an installer build (to get the right MSFT SDK from) is that during installer's build I will:

• Synchronize the VMR directly from installer's pipeline
• For PRs I would just validate the VMR sync instead of doing it in a separate pipeline, like we do now. For main build, I'd push too
• The VMR step would publish an artifact - a text file with the newly created SHA that is pushed during the main build
• The VMR Source-Build stage will be split into a new pipeline dotnet-dotnet-official-ci that will trigger when installer's is finished. This way it will have both MSFT SDK + corresponding commit in the VMR ready to work with

The above means we will end up with:

• installer-official-ci - current shape + newly VMR sync/push
• dotnet-dotnet-official-ci - new pipeline which will depend on installer (same as the tarball does)
  This pipeline will download the MSFT SDK + the SHA file and work with those
• We will remove following pipelines:
  • VMR validation
  • VMR synchronization
  • Source-Build official

❔ I have to figure one thing - how to preserve the VMR Build in a PR so that the matrix is defined in one place but it should be possible to re-use the new pipeline's YAML.

cc @dotnet/source-build-internal

[premun]

Just a side-note - this is roughly what the suggested process looks like for the main build:

flowchart TD
    Commit[dotnet/installer main commit]
    CI_Build[Build job]
    CI_Synchronize[Synchronize VMR job]
    CI_Push[Push to dotnet/dotnet]
    CI_SHA[Publish the SHA artifact]
    CI_SDK[Publish MSFT SDK]

    Commit-->installer-official-ci
    installer-official-ci--triggers-->dotnet-dotnet-official-ci

    subgraph installer-official-ci
        CI_Synchronize-->CI_Push
        CI_Push-->CI_SHA
        CI_Build-->CI_SDK
    end

    CI_SHA-..->CI_Checkout1
    CI_SHA-..->CI_Checkout2
    CI_SHA-..->CI_Checkout3
    CI_SHA-..->CI_Checkout4
    CI_SDK-..->CI_Download_SDK1

    CI_Stage1[CentOS job]
    CI_Checkout1[Checkout dotnet/dotnet]
    CI_SourceBuild1[Source-build CentOS]
    CI_Test1[Smoke-test CentOS]
    CI_Download_SDK1[Download MSFT SDK]
    CI_Compare_SDK1[Compare MSFT SDK]
    CI_Publish1[Publish CentOS SDK]

    CI_Stage2[Fedora job]
    CI_Checkout2[Checkout dotnet/dotnet]
    CI_SourceBuild2[Source-build Fedora]
    CI_Test2[Smoke-test Fedora]
    CI_Publish2[Publish Fedora SDK]

    CI_Stage3[Fedora job\nusing previous SB SDK]
    CI_Checkout3[Checkout dotnet/dotnet]
    CI_SourceBuild3[Source-build Fedora\nusing previous SB SDK]
    CI_Test3[Smoke-test Fedora]
    CI_Publish3[Publish Fedora SDK]

    CI_Stage4[Debian job]
    CI_Checkout4[Checkout dotnet/dotnet]
    CI_SourceBuild4[Source-build Debian]
    CI_Test4[Smoke-test Debian]
    CI_Publish4[Publish Debian SDK]

    subgraph dotnet-dotnet-official-ci
        CI_Stage1-->CI_Checkout1
        CI_Checkout1-->CI_SourceBuild1
        CI_SourceBuild1-->CI_Test1
        CI_Test1-->CI_Download_SDK1
        CI_Download_SDK1-->CI_Compare_SDK1
        CI_Compare_SDK1-->CI_Publish1

        CI_Stage2-->CI_Checkout2
        CI_Checkout2-->CI_SourceBuild2
        CI_SourceBuild2-->CI_Test2
        CI_Test2-->CI_Publish2

        CI_Stage3-->CI_Checkout3
        CI_Checkout3-->CI_SourceBuild3
        CI_SourceBuild3-->CI_Test3
        CI_Test3-->CI_Publish3

        CI_Publish2-->CI_Stage3
        CI_Publish2-..->CI_SourceBuild3

        CI_Stage4-->CI_Checkout4
        CI_Checkout4-->CI_SourceBuild4
        CI_SourceBuild4-->CI_Test4
        CI_Test4-->CI_Publish4
    end

    classDef Stage fill:#14FF2B,stroke:#333,stroke-width:4px;
    class CI_Build,CI_Synchronize,CI_Stage1,CI_Stage2,CI_Stage3,CI_Stage4 Stage;

The PR build then could go like this:

flowchart TD
    Commit[dotnet/installer PR commit]
    CI_Build[Build job]
    CI_SDK[Publish MSFT SDK]
    CI_Stage[CentOS job]

    Commit-->installer

    subgraph installer
        CI_Build-->CI_SDK

        CI_Stage-->CI_Checkout
        CI_Checkout-->CI_Synchronize
        CI_Synchronize-->CI_SourceBuild1
        CI_SourceBuild1-->CI_Test1
        CI_Test1-->CI_Download_SDK1
        CI_Download_SDK1-->CI_Compare_SDK1
        CI_Compare_SDK1-->CI_Publish1
    end

    CI_SDK-.pulls artifact.->CI_Download_SDK1

    CI_Checkout[Checkout dotnet/dotnet]
    CI_Synchronize[Synchronize PR into VMR locally]
    CI_SourceBuild1[Source-build CentOS]
    CI_Test1[Smoke-test CentOS]
    CI_Download_SDK1[Download MSFT SDK]
    CI_Compare_SDK1[Compare MSFT SDK]
    CI_Publish1[Publish CentOS SDK]

    classDef Stage fill:#14FF2B,stroke:#333,stroke-width:4px;
    class CI_Build,CI_Stage Stage;