[drone] `DRONE_` and `CI_` variable references must convert to a Harness expressions

jimsheldon commented 1 year ago

Drone provides many variables that can be referenced in pipelines, see https://docs.drone.io/pipeline/environment/reference/ and https://github.com/drone/runner-go/blob/master/environ/environ.go

Harness CI does not support environment variables outside of shell command execution, so we must convert them to expressions.

Here is an example Drone pipeline:

name: default
type: docker
kind: pipeline

steps:
  - name: example
    image: busybox
    commands:
    - echo this is build number $DRONE_BUILD_NUMBER

  - name: publish
    image: plugins/docker
    settings:
      repo: gcr.io/example/repo
      tag: 1.0.${DRONE_BUILD_NUMBER}

This should convert to this v1 yaml:

options: {}
stages:
- name: default
  spec:
    clone: {}
    runtime:
      spec: {}
      type: machine
    steps:
    - name: example
      spec:
        image: busybox
        run: echo this is build number <+pipeline.sequenceId>
    - name: publish
      spec:
        image: plugins/docker
        with:
          repo: gcr.io/example/repo
          tag: 1.0.<+pipeline.sequenceId>
      type: plugin
  type: ci

Relevant Harness documentation:

We should convert DRONE_ and CI_ variables surrounded by {} and without.

Where possible, we should reference codebase.* values, since they are always available to pipelines with a codebase defined. trigger.* values are not available for manual pipeline executions.

We need to make an effort to support Drone's substitution logic https://docs.drone.io/pipeline/environment/substitution/#string-operations

Drone variables can be escaped, see https://docs.drone.io/pipeline/environment/substitution/#escaping. This conversion should detect these escaped variables as well.

This table shows all DRONE_ variables, the associated deprecated CI_ variable (if applicable) and the equivalent Harness expression (if it is known).

`DRONE_` variable	`CI_` variable	Harness expression
`DRONE_BRANCH`	`n/a`	`<+codebase.branch>`
`DRONE_BUILD_ACTION`	`n/a`	`?`
`DRONE_BUILD_CREATED`	`CI_BUILD_CREATED`	`?`
`DRONE_BUILD_EVENT`	`CI_BUILD_EVENT`	`?`
`DRONE_BUILD_FINISHED`	`CI_BUILD_FINISHED`	`?`
`DRONE_BUILD_LINK`	`CI_BUILD_LINK`	`?`
`DRONE_BUILD_NUMBER`	`CI_BUILD_NUMBER`	`<+pipeline.sequenceId>`
`DRONE_BUILD_PARENT`	`n/a`	`?`
`DRONE_BUILD_STARTED`	`CI_BUILD_STARTED`	`?`
`DRONE_BUILD_STATUS`	`CI_BUILD_STATUS`	`?`
`DRONE_BUILD_TRIGGER`	`n/a`	`?`
`DRONE_CALVER`	`n/a`	`?`
`DRONE_COMMIT`	`n/a`	`?`
`DRONE_COMMIT_AFTER`	`n/a`	`?`
`DRONE_COMMIT_AUTHOR`	`CI_COMMIT_AUTHOR`	`<+codebase.gitUserId>`
`DRONE_COMMIT_AUTHOR_AVATAR`	`CI_COMMIT_AUTHOR_AVATAR`	`?`
`DRONE_COMMIT_AUTHOR_EMAIL`	`CI_COMMIT_AUTHOR_EMAIL`	`?`
`DRONE_COMMIT_AUTHOR_NAME`	`CI_COMMIT_AUTHOR_NAME`	`?`
`DRONE_COMMIT_BEFORE`	`n/a`	`?`
`DRONE_COMMIT_BRANCH`	`CI_COMMIT_BRANCH`	`<+codebase.branch>`
`DRONE_COMMIT_LINK`	`n/a`	`?`
`DRONE_COMMIT_MESSAGE`	`CI_COMMIT_MESSAGE`	`?`
`DRONE_COMMIT_REF`	`CI_COMMIT_REF`	`?`
`DRONE_COMMIT_SHA`	`CI_COMMIT_SHA`	`<+codebase.commitSha>`
`DRONE_DEPLOY_TO`	`n/a`	`?`
`DRONE_FAILED_STAGES`	`n/a`	`?`
`DRONE_FAILED_STEPS`	`n/a`	`?`
`DRONE_GIT_HTTP_URL`	`n/a`	`?`
`DRONE_GIT_SSH_URL`	`n/a`	`?`
`DRONE_PULL_REQUEST`	`n/a`	`<+codebase.prNumber>`
`DRONE_PULL_REQUEST_TITLE`	`n/a`	`<+codebase.prTitle>`
`DRONE_REMOTE_URL`	`CI_REMOTE_URL`	`<+codebase.repoUrl>`
`DRONE_REPO`	`CI_REPO`	`?`
`DRONE_REPO_BRANCH`	`n/a`	`?`
`DRONE_REPO_LINK`	`CI_REPO_LINK`	`?`
`DRONE_REPO_NAME`	`CI_REPO_NAME`	`<+<+codebase.repoUrl>.substring(<+codebase.repoUrl>.lastIndexOf('/') + 1)>"`
`DRONE_REPO_NAMESPACE`	`n/a`	`?`
`DRONE_REPO_OWNER`	`n/a`	`?`
`DRONE_REPO_PRIVATE`	`CI_REPO_PRIVATE`	`?`
`DRONE_REPO_SCM`	`n/a`	`?`
`DRONE_REPO_VISIBILITY`	`n/a`	`?`
`DRONE_SEMVER`	`n/a`	`?`
`DRONE_SEMVER_BUILD`	`n/a`	`?`
`DRONE_SEMVER_ERROR`	`n/a`	`?`
`DRONE_SEMVER_MAJOR`	`n/a`	`?`
`DRONE_SEMVER_MINOR`	`n/a`	`?`
`DRONE_SEMVER_PATCH`	`n/a`	`?`
`DRONE_SEMVER_PRERELEASE`	`n/a`	`?`
`DRONE_SEMVER_SHORT`	`n/a`	`?`
`DRONE_SOURCE_BRANCH`	`n/a`	`?`
`DRONE_STAGE_ARCH`	`n/a`	`?`
`DRONE_STAGE_DEPENDS_ON`	`n/a`	`?`
`DRONE_STAGE_FINISHED`	`n/a`	`?`
`DRONE_STAGE_KIND`	`n/a`	`?`
`DRONE_STAGE_MACHINE`	`n/a`	`?`
`DRONE_STAGE_NAME`	`n/a`	`?`
`DRONE_STAGE_NUMBER`	`n/a`	`?`
`DRONE_STAGE_OS`	`n/a`	`?`
`DRONE_STAGE_STARTED`	`n/a`	`?`
`DRONE_STAGE_STATUS`	`n/a`	`?`
`DRONE_STAGE_TYPE`	`n/a`	`?`
`DRONE_STAGE_VARIANT`	`n/a`	`?`
`DRONE_STEP_NAME`	`n/a`	`?`
`DRONE_STEP_NUMBER`	`n/a`	`?`
`DRONE_SYSTEM_HOST`	`n/a`	`?`
`DRONE_SYSTEM_HOSTNAME`	`n/a`	`?`
`DRONE_SYSTEM_PROTO`	`n/a`	`?`
`DRONE_SYSTEM_VERSION`	`n/a`	`?`
`DRONE_TAG`	`n/a`	`?`
`DRONE_TARGET_BRANCH`	`n/a`	`?`
`DRONE_WORKSPACE`	`n/a`	`?`

jimsheldon commented 1 year ago

Bash-like substitutions:

Tested successfully

Print the first 8 characters of the commit sha

Drone	Harness
`${DRONE_COMMIT_SHA:0:8}`	`<+codebase.commitSha.substring(0,8)>`

Strip v prefix from v1.0.0

Drone	Harness
`${DRONE_TAG##v}`	`<+<+trigger.payload.ref.split('/')[2]>.substring(<+trigger.payload.ref.split('/')[2]>.indexOf('v') + 1)>`

Replace / with -

Drone	Harness
`${DRONE_BRANCH/\//-}`	`<+codebase.branch.replace('/', '-')>`

jimsheldon commented 1 year ago

Initial support added in #118.

We are making progress on this, I'll keep this open since we will likely add support for more variables, this issue can be the source of truth for all substitutions.

drone / go-convert

[drone] `DRONE_` and `CI_` variable references must convert to a Harness expressions #117

Tested successfully