Use GH actions for build/release - rancher/rancher

[tashima42]

Migrate http://github.com/rancher/rancher to GH Actions

• Migration Guide: https://docs.google.com/document/d/16SQF6H7HNvKvZ4FDuqA5jRJsfWieTpaTMPmMZPRr7aQ/edit
• Reference: https://github.com/rancher/.github-private/blob/main/profile/README.md

Tasks:

• [x] Map components and scripts using Drone @rafaelbreno
• [x] Map Drone steps and triggers @tashima42
• [x] Map secrets @rafaelbreno
• [x] Create issue in https://github.com/rancherlabs/eio/ to migrate secrets @rafaelbreno
• [ ] Migrate release/v2.7
  • [x] PR
  • [x] Push on master or release branches
  • [x] Tag
  • [ ] Create patch files to be applied to debug releases
  • [x] Promote
  • [ ] Integration Tests
  • [ ] Provisioning tests
• [ ] Migrate release/v2.8
  • [x] PR
  • [x] Push on master or release branches
  • [x] Tag
  • [ ] Create patch files to be applied to debug releases
  • [x] Promote
  • [ ] Integration Tests
  • [ ] Provisioning tests
• [x] Migrate release/v2.9
  • [x] PR
  • [x] Push on master or release branches
  • [x] Tag
  • [ ] Create patch files to be applied to debug releases
  • [ ] Promote
  • [x] Integration Tests
  • [x] Provisioning tests
• [ ] Investigate how to remove Dapper from our build stack

[rafaelbreno]

[rancher/rancher] Drone to GHA migration

Scripts mentioning Drone:

• scripts/create-components-file.sh
• scripts/generate-linux-digest.sh
• scripts/only-ui-bump.sh
• scripts/promote-docker-image.sh
• scripts/run
• scripts/version
• scripts/chart/copy
• scripts/chart/promote-to-stable.sh
• scripts/chart/promote-to-stable.sh
• scripts/check-release-images-exist.sh
• scripts/windows/package.ps1
• scripts/windows/version.ps1
• Dockerfile.dapper
• Dockerfile-windows.dapper

Secrets from_secret

• FOSSA_API_KEY
• docker_password
• docker_username
• github_token
• google_auth_key
• stage_registry_password
• stage_registry_username
• prime_artifacts_uploader-access_key
• prime_artifacts_uploader-secret_key
• optimus_charts_access_key
• optimus_charts_secret_key

[tashima42]

GitHub Workflows

⚠️ Steps are not in order yet ⚠️

Pull Request

• [ ] provisioning-tests-k3s -> provisioning-tests-pr
• [ ] provisioning-tests-rke2 -> provisioning-tests-pr
• [ ] provisioning-operations-tests-k3s -> provisioning-operations-tests-pr
• [ ] provisioning-operations-test-setA-rke2 -> provisioning-operations-tests-pr
• [ ] provisioning-operations-test-setB-rke2 -> provisioning-operations-tests-pr
• [x] default-linux-amd64 -> build-pr
• [x] default-linux-arm64 -> build-pr
• [x] default-windows-1809 -> build-pr
• [x] default-windows-ltsc2022 -> build-pr
• [ ] integration tests

Push on Master or Release Branches

• [ ] provisioning-tests-k3s -> provisioning-tests-push
• [ ] provisioning-tests-rke2 -> provisioning-tests-push
• [ ] provisioning-operations-tests-k3s -> provisioning-operations-tests-push
• [ ] provisioning-operations-test-setA-rke2 -> provisioning-operations-tests-push
• [ ] provisioning-operations-test-setB-rke2 -> provisioning-operations-tests-push
• [x] default-linux-amd64 -> build-push-tag
• [ ] default-linux-amd64 -> fossa-amd64 (this step always fails, will implement it later)
• [x] default-linux-amd64 -> stage-binaries
• [x] default-linux-amd64 -> docker-publish-head
• [x] default-linux-amd64 -> docker-publish-head-installer
• [x] default-linux-amd64 -> docker-publish-head-agent
• [x] default-linux-arm64 -> build-push-tag
• [x] default-linux-arm64 -> stage-binaries
• [x] default-linux-arm64 -> docker-publish-head
• [x] default-linux-arm64 -> docker-publish-head-installer
• [x] default-linux-arm64 -> docker-publish-head-agent
• [x] default-windows-1809 -> build-push-tag
• [x] default-windows-1809 -> stage-binaries
• [x] default-windows-1809 -> docker-publish-head-agent
• [x] default-windows-ltsc2022 -> build-push-tag
• [x] default-windows-ltsc2022 -> stage-binaries
• [x] default-windows-ltsc2022 -> docker-publish-head-agent
• [x] manifest -> push-installer-manifest
• [x] manifest -> push-installer-manifest-head
• [x] manifest -> push-agent-manifest
• [x] manifest -> push-agent-manifest-head
• [x] manifest -> push-manifest-head
• [x] manifest -> push-manifest
• [x] manifest -> build-chart
• [ ] integration tests

Tag Creation

• [x] default-linux-amd64 -> build-push-tag
• [ ] default-linux-amd64 -> fossa-amd64 (this step always fails, maybe we should remove it)
• [x] default-linux-amd64 -> stage-binaries
• [x] default-linux-amd64 -> docker-publish
• [x] default-linux-amd64 -> docker-publish-agent
• [x] default-linux-amd64 -> github_binary_prerelease (Only pre-releases)
• [x] default-linux-amd64 -> github_binary_release (Only GA releases)
• [ ] default-linux-amd64 -> check-release-images-exists (won't do, the generate images digests step will fail if any image is missing from the registry)
• [x] default-linux-arm64 -> build-push-tag
• [x] default-linux-arm64 -> stage-binaries
• [x] default-linux-arm64 -> docker-publish
• [x] default-linux-arm64 -> docker-publish-installer
• [x] default-linux-arm64 -> docker-publish-agent
• [x] default-windows-1809 -> build-push-tag
• [x] default-windows-1809 -> stage-binaries
• [x] default-windows-1809 -> docker-publish-agent
• [x] default-windows-ltsc2022 -> build-push-tag
• [x] default-windows-ltsc2022 -> stage-binaries
• [x] default-windows-ltsc2022 -> docker-publish-agent
• [x] manifest -> push-installer-manifest
• [x] manifest -> push-agent-manifest
• [x] manifest -> push-manifest
• [x] manifest -> build-chart
• [ ] manifest -> chart-publish
• [x] docker-image-digests-linux -> docker-image-digests
• [x] docker-image-digests-linux -> github_binary_release_digests
• [x] docker-image-digests-windows-1809 -> docker-image-digests
• [x] docker-image-digests-windows-ltsc2022 -> docker-image-digests
• [ ] integration tests

Promote There should be two versions, one for stable and the other for latest.

• publish -> chart-promote
• publish -> chart-publish
• publish -> docker-image-promote

Drone Pipelines

Expand

## provisioning-tests-k3s | Label | Value | |-------------|-------------------------------| | Pipeline | provisioning-tests-k3s | | Name | provisioning-tests-pr | | Platform | linux-amd64 | | Triggers | Pull Requests | | Instance | All | | Label | Value | |-------------|------------------------------------| | Pipeline | provisioning-tests-k3s | | Name | provisioning-tests-push | | Platform | linux-amd64 | | Triggers | Push on master or release branches | | Instance | drone-publish.rancher.io | ## provisioning-tests-rke2 | Label | Value | |-------------|--------------------------------| | Pipeline | provisioning-tests-rke2 | | Name | provisioning-tests-pr | | Platform | linux-amd64 | | Triggers | Pull Requests | | Instance | All | | Label | Value | |-------------|-------------------------------------| | Pipeline | provisioning-tests-rke2 | | Name | provisioning-tests-push | | Platform | linux-amd64 | | Triggers | Push on master or release branches | | Instance | drone-publish.rancher.io | ## provisioning-operations-tests-k3s | Label | Value | |-------------|-----------------------------------| | Pipeline | provisioning-operations-tests-k3s | | Name | provisioning-operations-tests-pr | | Platform | linux-amd64 | | Triggers | Pull Requests | | Instance | All | | Label | Value | |----------|-------------------------------------| | Pipeline | provisioning-operations-tests-k3s | | Name | provisioning-operations-tests-push | | Platform | linux-amd64 | | Triggers | Push on master or release branches | | Instance | drone-publish.rancher.io | ## provisioning-operations-test-setA-rke2 | Label | Value | |----------|----------------------------------------| | Pipeline | provisioning-operations-test-setA-rke2 | | Name | provisioning-operations-tests-pr | | Platform | linux-amd64 | | Triggers | Pull Requests | | Instance | All | | Label | Value | |----------|----------------------------------------| | Pipeline | provisioning-operations-test-setA-rke2 | | Name | provisioning-operations-tests-push | | Platform | linux-amd64 | | Triggers | Push on master or release branches | | Instance | drone-publish.rancher.io | ## provisioning-operations-test-setB-rke2 | Label | Value | |----------|----------------------------------------| | Pipeline | provisioning-operations-test-setB-rke2 | | Name | provisioning-operations-tests-pr | | Platform | linux-amd64 | | Triggers | Pull Requests | | Instance | All | | Label | Value | |----------|----------------------------------------| | Pipeline | provisioning-operations-test-setB-rke2 | | Name | provisioning-operations-tests-push | | Platform | linux-amd64 | | Triggers | Push on master or release branches | | Instance | drone-publish.rancher.io | ## default-linux-amd64 | Label | Value | |----------|---------------------| | Pipeline | default-linux-amd64 | | Name | build-pr | | Platform | linux-amd64 | | Triggers | Pull Requests | | Instance | All | | Label | Value | |----------|----------------------------------------------------| | Pipeline | default-linux-amd64 | | Name | build-push-tag | | Platform | linux-amd64 | | Triggers | Push on master or release branches or tag creation | | Instance | drone-publish.rancher.io | | Label | Value | |----------|----------------------------------------------------| | Pipeline | default-linux-amd64 | | Name | fossa-amd64 | | Platform | linux-amd64 | | Triggers | Push on master or release branches or tag creation | | Instance | drone-publish.rancher.io | | Label | Value | |----------|----------------------------------------------------| | Pipeline | default-linux-amd64 | | Name | stage-binaries | | Platform | linux-amd64 | | Triggers | Push on master or release branches or tag creation | | Instance | drone-publish.rancher.io | | Label | Value | |----------|------------------------------------| | Pipeline | default-linux-amd64 | | Name | docker-publish-head | | Platform | linux-amd64 | | Triggers | Push on master or release branches | | Instance | All | | Label | Value | |----------|------------------------------------| | Pipeline | default-linux-amd64 | | Name | docker-publish-head-installer | | Platform | linux-amd64 | | Triggers | Push on master or release branches | | Instance | All | | Label | Value | |----------|------------------------------------| | Pipeline | default-linux-amd64 | | Name | docker-publish-head-agent | | Platform | linux-amd64 | | Triggers | Push on master or release branches | | Instance | All | | Label | Value | |----------|---------------------| | Pipeline | default-linux-amd64 | | Name | docker-publish | | Platform | linux-amd64 | | Triggers | Tag | | Instance | All | | Label | Value | |----------|----------------------| | Pipeline | default-linux-amd64 | | Name | docker-publish-agent | | Platform | linux-amd64 | | Triggers | Tag | | Instance | All | | Label | Value | |----------|--------------------------| | Pipeline | default-linux-amd64 | | Name | github_binary_prerelease | | Platform | linux-amd64 | | Triggers | Pre-release tag creation | | Instance | All | | Label | Value | |----------|-----------------------| | Pipeline | default-linux-amd64 | | Name | github_binary_release | | Platform | linux-amd64 | | Triggers | GA tag creation | | Instance | All | | Label | Value | |----------|-----------------------------| | Pipeline | default-linux-amd64 | | Name | check-release-images-exists | | Platform | linux-amd64 | | Triggers | Tag | | Instance | drone-publish.rancher.io | ## default-linux-arm64 | Label | Value | |----------|---------------------| | Pipeline | default-linux-arm64 | | Name | build-pr | | Platform | linux-arm64 | | Triggers | Pull requests | | Instance | All | | Label | Value | |----------|----------------------------------------------------| | Pipeline | default-linux-arm64 | | Name | build-push-tag | | Platform | linux-arm64 | | Triggers | Push on master or release branches or tag creation | | Instance | drone-publish.rancher.io | | Label | Value | |----------|----------------------------------------------------| | Pipeline | default-linux-arm64 | | Name | stage-binaries | | Platform | linux-arm64 | | Triggers | Push on master or release branches or tag creation | | Instance | drone-publish.rancher.io | | Label | Value | |----------|------------------------------------| | Pipeline | default-linux-arm64 | | Name | docker-publish-head | | Platform | linux-arm64 | | Triggers | Push on master or release branches | | Instance | All | | Label | Value | |----------|------------------------------------| | Pipeline | default-linux-arm64 | | Name | docker-publish-head-installer | | Platform | linux-arm64 | | Triggers | Push on master or release branches | | Instance | All | | Label | Value | |----------|------------------------------------| | Pipeline | default-linux-arm64 | | Name | docker-publish-head-agent | | Platform | linux-arm64 | | Triggers | Push on master or release branches | | Instance | All | | Label | Value | |----------|---------------------| | Pipeline | default-linux-arm64 | | Name | docker-publish | | Platform | linux-arm64 | | Triggers | Tag Creation | | Instance | All | | Label | Value | |----------|--------------------------| | Pipeline | default-linux-arm64 | | Name | docker-publish-installer | | Platform | linux-arm64 | | Triggers | Tag Creation | | Instance | All | | Label | Value | |----------|----------------------| | Pipeline | default-linux-arm64 | | Name | docker-publish-agent | | Platform | linux-arm64 | | Triggers | Tag Creation | | Instance | All | ## default-windows-1809 | Label | Value | |----------|----------------------| | Pipeline | default-windows-1809 | | Name | build-pr | | Platform | windows-amd64 v1809 | | Triggers | Pull Request | | Instance | All | | Label | Value | |----------|----------------------------------------------------| | Pipeline | default-windows-1809 | | Name | build-push-tag | | Platform | windows-amd64 v1809 | | Triggers | Push on master or release branches or tag creation | | Instance | drone-publish.rancher.io | | Label | Value | |----------|----------------------------------------------------| | Pipeline | default-windows-1809 | | Name | stage-binaries | | Platform | windows-amd64 v1809 | | Triggers | Push on master or release branches or tag creation | | Instance | drone-publish.rancher.io | | Label | Value | |----------|------------------------------------| | Pipeline | default-windows-1809 | | Name | docker-publish-head-agent | | Platform | windows-amd64 v1809 | | Triggers | Push on master or release branches | | Instance | drone-publish.rancher.io | | Label | Value | |----------|----------------------| | Pipeline | default-windows-1809 | | Name | docker-publish-agent | | Platform | windows-amd64 v1809 | | Triggers | Tag Creation | | Instance | All | ## default-windows-ltsc2022 | Label | Value | |----------|----------------------| | Pipeline | default-windows-ltsc2022 | | Name | build-pr | | Platform | windows-amd64 ltsc2022 | | Triggers | Pull Request | | Instance | All | | Label | Value | |----------|----------------------------------------------------| | Pipeline | default-windows-ltsc2022 | | Name | build-push-tag | | Platform | windows-amd64 ltsc2022 | | Triggers | Push on master or release branches or tag creation | | Instance | drone-publish.rancher.io | | Label | Value | |----------|----------------------------------------------------| | Pipeline | default-windows-ltsc2022 | | Name | stage-binaries | | Platform | windows-amd64 ltsc2022 | | Triggers | Push on master or release branches or tag creation | | Instance | drone-publish.rancher.io | | Label | Value | |----------|------------------------------------| | Pipeline | default-windows-ltsc2022 | | Name | docker-publish-head-agent | | Platform | windows-amd64 ltsc2022 | | Triggers | Push on master or release branches | | Instance | drone-publish.rancher.io | | Label | Value | |----------|----------------------| | Pipeline | default-windows-ltsc2022 | | Name | docker-publish-agent | | Platform | windows-amd64 ltsc2022 | | Triggers | Tag Creation | | Instance | All | ## manifest depends_on: - default-linux-amd64 - default-linux-arm64 - default-windows-1809 - default-windows-ltsc2022 | Label | Value | |----------|----------------------------------------------------| | Pipeline | manifest | | Name | push-installer-manifest | | Platform | linux-amd64 | | Triggers | Push on master or release branches or tag creation | | Instance | drone-publish.rancher.io | | Label | Value | |----------|------------------------------------| | Pipeline | manifest | | Name | push-installer-manifest-head | | Platform | linux-amd64 | | Triggers | Push on master or release branches | | Instance | drone-publish.rancher.io | | Label | Value | |----------|----------------------------------------------------| | Pipeline | manifest | | Name | push-agent-manifest | | Platform | linux-amd64 | | Triggers | Push on master or release branches or tag creation | | Instance | drone-publish.rancher.io | | Label | Value | |----------|------------------------------------| | Pipeline | manifest | | Name | push-agent-manifest-head | | Platform | linux-amd64 | | Triggers | Push on master or release branches | | Instance | drone-publish.rancher.io | | Label | Value | |----------|------------------------------------| | Pipeline | manifest | | Name | push-manifest-head | | Platform | linux-amd64 | | Triggers | Push on master or release branches | | Instance | drone-publish.rancher.io | | Label | Value | |----------|----------------------------------------------------| | Pipeline | manifest | | Name | push-manifest | | Platform | linux-amd64 | | Triggers | Push on master or release branches or tag creation | | Instance | drone-publish.rancher.io | | Label | Value | |----------|----------------------------------------------------| | Pipeline | manifest | | Name | build-chart | | Platform | linux-amd64 | | Triggers | Push on master or release branches or tag creation | | Instance | drone-publish.rancher.io | | Label | Value | |----------|---------------| | Pipeline | manifest | | Name | chart-publish | | Platform | linux-amd64 | | Triggers | Tag | | Instance | All | ## docker-image-digests depends_on: - default-linux-amd64 - default-linux-arm64 | Label | Value | |----------|----------------------------| | Pipeline | docker-image-digests-linux | | Name | docker-image-digests | | Platform | linux-amd64 | | Triggers | Tag | | Instance | drone-publish.rancher.io | | Label | Value | |----------|-------------------------------| | Pipeline | docker-image-digests-linux | | Name | github_binary_release_digests | | Platform | linux-amd64 | | Triggers | Tag | | Instance | drone-publish.rancher.io | ## docker-image-digests-windows-1809 depends_on: - default-linux-amd64 - default-windows-1809 | Label | Value | |----------|-----------------------------------| | Pipeline | docker-image-digests-windows-1809 | | Name | docker-image-digests | | Platform | windows-amd64 v1809 | | Triggers | Tag | | Instance | drone-publish.rancher.io | ## docker-image-digests-windows-ltsc2022 depends_on: - default-linux-amd64 - default-windows-ltsc2022 | Label | Value | |----------|-----------------------------------| | Pipeline | docker-image-digests-windows-ltsc2022 | | Name | docker-image-digests | | Platform | windows-amd64 ltsc2022 | | Triggers | Tag | | Instance | drone-publish.rancher.io | ## publish | Label | Value | |----------|---------------------------------------| | Pipeline | publish | | Name | chart-promote | | Platform | linux-amd64 | | Triggers | Promote (Workflow Dispatch) | | Instance | All | | Label | Value | |----------|---------------------------------------| | Pipeline | publish | | Name | chart-publish | | Platform | linux-amd64 | | Triggers | Promote (Workflow Dispatch) | | Instance | All | ## promote-docker-image | Label | Value | |----------|-----------------------------| | Pipeline | publish | | Name | docker-image-promote | | Platform | linux-amd64 | | Triggers | Promote (Workflow Dispatch) | | Instance | All |

[rafaelbreno]

created issue for secrets & docker images access: #2273