Architect how to import charts release process on ecm-distro-tools

[nicholasSUSE]

Scope

The charts repository release process was transferred to the release team with me as a team member.

The charts repository release process is partially automated through the charts-build-scripts repo.

Much of the release process is still manual.

Goal

I must provide rancher release team members with a better understanding of the automation in place at charts-build-scripts repo and future ones. I must gather a basic understanding of how automations work and are used from this repository. We must reach a consensus on how the in-place and future automations of charts-build-scripts repo can be implemented on ecm-distro-tools.

Tasks

• [x] Set a dev environment for ecm-distro-tools and learn how it operates and what it does.
• [x] Install, uninstall, and execute what is possible, taking care not to execute any actual releases on ecm-distro-tools.
• [x] Provide a summary description of how charts-build-scripts work.
• [x] Differentiate the features for generating and releasing charts in the above description.
• [x] Provide an architecture proposal for implementing charts-build-scripts release capabilities on ecm-distro-tools.
• [ ] Discuss with the team the pros and cons.

[nicholasSUSE]

Summary of charts-build-scripts

These are all the available commands/actions that charts-build-scripts enables to be used with charts:

flowchart LR

    root["`**charts-build-scripts**`"]

    root --> dev("chart development")
    root --> old_release("old release automation")
    root --> new_release("new advanced release automation")
    root --> gha("CI/CD GH Actions")
    root --> dont_know("I dont know what this is")

    dev --> list("list")  -.-> txt1[/Print a list of all packages tracked in the current repository/]
    dev --> prepare("prepare") -.-> txt2[/Pull in the chart specified from upstream to the charts directory and apply any patch files/]
    dev --> patch("patch") -.-> txt3[/Apply a patch between the upstream chart and the current state of the chart in the charts directory/]
    dev --> charts("charts") -.-> txt4[/Create a local chart archive of your finalized chart for testing/]
    dev --> index("index") -.-> txt5[/Create or update the existing Helm index.yaml at the repository root/]
    dev -->  clean("clean") -.-> txt6[/Clean up your current repository to get it ready for a PR/]
    dev --> icon("icon") -.-> txt7[/ Download the chart icon locally and use it/]

    old_release --> regsync("regsync") -.-> txt8[/Create a regsync config file containing all images used for the particular Rancher version/]
    old_release --> zip("zip") -.-> txt9[/Take the contents of an asset under assets/ and unzip the chart/]
    old_release --> unzip("unzip") -.-> txt10[/Take the contents of an asset under assets/ and unzip the chart/]
    old_release --> validate("validate") -.-> txt11[/Run validation to ensure that contents of assets and charts won't overwrite released charts/]

    new_release --> enforce-lifecycle("enforce-lifecycle") -.-> txt12[/Remove all assets versions that don't belong on this branch according to the lifecycle rules/]
    new_release --> lifecycle-status("lifecycle-status") -.-> txt13[/Print the status of the current assets and charts based on the branch version and chart version according to the lifecycle rules/]
    new_release --> auto-forward-port("auto-forward-port") -.-> txt14[/Execute the forward-port script to forward port a chart or all to the production branch/]

    gha --> standardize("standardize") -.-> txt15[/Standardize a Helm repository to the expected assets, charts, and index.yaml structure of these scripts/]
    gha --> check-rc("check-rc") -.-> txt16[/Checks if there are any images with RC tags or charts with RC versions in the charts repository/]
    gha --> check-images("check-images") -.-> txt17[/Checks all container images used in the charts repository/]

    dont_know --> template("template") -.-> txt18[/Updates the current directory by applying the configuration.yaml on upstream Go templates to pull in the most up-to-date docs, scripts, etc/]
    dont_know --> clean-cache("clean-cache") -.-> txt19[/Experimental: Clean cache/]

I separated the commands into 4 categories:

• chart development: executed by the chart owners to develop new charts
• old release automation: used in manual validations and some GH Actions
• new advanced release automation: the ones implemented by Nicholas to automate the release process
• CI/CD GH Actions: commands used in automated validations at GH Actions

It is not possible to show in the flowchart but all the automations (old and new) use the commands featured in chart development.

---

Summary of charts release process

flowchart TB
    subgraph 3rd phase - clean up
    direction TB
    forwardPort("forward-ports charts that were released to the next dev-v2.(x+1)")
    forwardPort --> updateBranchBack("update rancher branch references back to dev branch")
    updateBranchBack --> startAgain("start again the next 2.X+1 line")
    end

    subgraph 2nd phase - release work
    direction TB
    releaseMerge("Start merging charts from dev-v2.X to release-v2.X")
    releaseMerge --> monitorMirror("monitor git mirror")
    monitorMirror --> updateSupportMatrix("update charts versions on the product support matrix")
    updateSupportMatrix --> updateBranchRef("update rancher branch references to release branch")
    end

    subgraph 1st phase - development work
    direction TB
    new_release_cycle("new release cycle announced")
    new_release_cycle --> chart("chart development work")
    chart --> devMerge("merge on dev-v2.X branch")
    devMerge --> qaSignOff("QA tests and sign off the chart")
    end

[nicholasSUSE]

Architecture review of ecm-distro-tools:

The best option to include the charts release process is the: cmd/release package.

A quick summary of the cmd/release package architecture:

flowchart LR

    root["`**release**
    central command to perform
    RKE2 K3s and Rancher Releases`"]

    root -- manage the release cli config file --> config("config")
    root -- various utilities to generate release artifacts --> generate("generate")
    root -- list resources --> list("list")
    root -- push things from your local to a remote --> push("push")
    root -- tag releases --> tag("tag")
    root -- update files and other utilities --> update("update")

    config --> edit("edit")
    config --> gen("gen")
    config --> view("view")

    generate --> k3s("k3s")
    generate --> rancher("rancher")
    generate --> rke2("rke2")

    list --> rancher

    push --> k3s

    tag --> k3s
    tag --> rancher
    tag --> rke2
    tag --> system-agent-installer-k3s("system-agent-installer-k3s")

    update --> k3s

[nicholasSUSE]

Architecture proposal

To implement charts-build-scripts into ecm-distro-tools in order to develop advanced charts release processes:

I propose to add the object of charts in the already existing commands that have similar approaches.

At first glance the following sub commands would not be relevant because there is nothing similar in the charts release process:

• tag
• generate

Also let's get rid of config for simplicity for now.

The current ecm-distro-tools architecture simplified:

flowchart LR

    root["`**release**
    central command to perform
    RKE2 K3s and Rancher Releases`"]

    root -- list resources --> list("list")
    root -- push things from your local to a remote --> push("push")
    root -- update files and other utilities --> update("update")

    list --> rancher("rancher")
    update --> k3s
    push --> k3s("k3s")

Proposed ecm-distro-tools architecture:

flowchart LR

    root["`**release**
    central command to perform
    RKE2 K3s Rancher and Charts Releases`"]

    root -- list resources --> list("list")
    root -- push things from your local to a remote --> push("push")
    root -- update files and other utilities --> update("update")

    list --> rancher("rancher")
    list --> charts("charts")
    update --> k3s("k3s")
    update --> charts
    push --> k3s
    push --> charts

Proposed ecm-distro-tools charts automation workflow:

The charts-build-scripts repository is supposed to be used by the charts repository.

What I mean is:

the normal procedure is to 1st clone charts and then from within execute, the charts-build-scripts at it's bin/ folder.

Before any command execution:

• update your config file to a valid charts repository
• create a charts configuration and follow the same pattern for the k3s release configuration.

If I understood it correctly, there must be a k3s cloned repository configured before the execution of:

• release push k3s

This configurations is verified and loaded at:

<workspace>/cmd/release/cmd/push.go:

        k3sRelease, found := rootConfig.K3s.Versions[version]
        if !found {
            return errors.New("verify your config file, version not found: " + version)
        }

And the path for the repository is used at:

<workspace>/release/k3s/k3s.go:

    fmt.Println("opening kubernetes repo")
    repo, err := git.PlainOpen(filepath.Join(r.Workspace, "kubernetes"))
    if err != nil {
        return err
    }

flowchart LR
    list["`__list__
    the current assets that need to be released/forward-ported`"]
    list --> update["`__update__
    current production branch (release-v2.X) with charts versions to be **released/forward-ported**`"]
    update --> push["`__push__
    and organize the branches with the respective assets versions`"]

Proposed ecm-distro-tools files to work on:

To be modified:

cmd/release/cmd/

• list.go
• push.go
• root.go
• update.go

cmd/release/config/

• config.go
• config_test.go
• test_data/config.json

To be created: release/charts/

• charts.go

Proposed charts data-structure:

The data structure for the k3s release is defined at: <workspace>/cmd/release/config/config.go:

type K3sRelease struct {
    OldK8sVersion                 string `json:"old_k8s_version"`
    NewK8sVersion                 string `json:"new_k8s_version"`
    OldK8sClient                  string `json:"old_k8s_client"`
    NewK8sClient                  string `json:"new_k8s_client"`
    OldSuffix                     string `json:"old_suffix"`
    NewSuffix                     string `json:"new_suffix"`
    ReleaseBranch                 string `json:"release_branch"`
    Workspace                     string `json:"workspace"`
    NewGoVersion                  string `json:"-"`
    K3sRepoOwner                  string `json:"k3s_repo_owner"`
    SystemAgentInstallerRepoOwner string `json:"system_agent_installer_repo_owner"`
    K8sRancherURL                 string `json:"k8s_rancher_url"`
    K3sUpstreamURL                string `json:"k3s_upstream_url"`
    DryRun                        bool   `json:"dry_run"`
}

At the same file we will have at least something like:

type ChartsRelease struct {
        DevBranch         string `json:"dev_branch"`        // dev-v2.9
    ReleaseBranch     string `json:"release_branch"`    // release-v2.9
    Workspace         string `json:"workspace"`         // <path/to/charts/repo>
    ChartsRepoOwner   string `json:"charts_repo_owner"` // git user name
    DryRun            bool   `json:"dry_run"`           // just log and don't execute
}