getpopper / popper

Container-native task automation engine.
https://getpopper.io
MIT License
302 stars 61 forks source link

translator: support `env` options in Popper-to-Task translator #992

Closed shumbo closed 3 years ago

shumbo commented 3 years ago

Overview

This PR adds support for env options in Popper-to-Task translator.

It adds the following two functionalities to the translator.

User-defined environment variables

Popper provides two ways to define environment variables.

The first is to use the options.env attribute. The environment variables defined in this attribute would be available from all steps in the workflow.

The second is to use the steps[i].env attribute. The variables defined in this attribute are only available in the step.

When the Popper parser parses a workflow, it automatically reads variables defined in options.env and merges them into steps[i].env.

Therefore, the Popper-to-Task translator reads that merged dictionary (consists of options.env and steps[i].env) and put it in task[i].env. Task will read the attribute and the variables will be accessible from the commands.

Pre-defined environment variables

Popper provides five pre-defined variables (GIT_COMMIT, GIT_BRANCH, GIT_SHA_SHORT, GIT_REMOTE_ORIGIN_URL, GIT_TAG) that users can use in their workflows. Unlike Drone, Task doesn't have any pre-defined variables, hence we need to run commands and define variables.

I used Task's dynamic variables. As shown in Task's documentation, Task can run commands and replace placeholders (e.g. {{.VAR_NAME}}) with their results.

Because the env attribute can contain such placeholders, we can 1) define dynamic variables and 2) use these variables to define environment variables.

The below is an example output of the translator.

env:
  GIT_BRANCH: '{{.GIT_BRANCH}}'
  GIT_COMMIT: '{{.GIT_COMMIT}}'
  GIT_REMOTE_ORIGIN_URL: '{{.GIT_REMOTE_ORIGIN_URL}}'
  GIT_SHA_SHORT: '{{.GIT_SHA_SHORT}}'
  GIT_TAG: '{{.GIT_TAG}}'
vars:
  GIT_BRANCH:
    sh: git branch --show-current 2>/dev/null || echo ""
  GIT_COMMIT:
    sh: git rev-parse HEAD || echo ""
  GIT_REMOTE_ORIGIN_URL:
    sh: git config --get remote.origin.url || echo ""
  GIT_SHA_SHORT:
    sh: git rev-parse --short HEAD 2>/dev/null || echo ""
  GIT_TAG:
    sh: git tag -l --contains HEAD 2>/dev/null | head -n 1

vars will define dynamic variables and we use these variables in env to expose them from environment variables.

Exposing Environment Variables to Docker Containers

Task will make environment variables available to the commands it runs. They won't be available from Docker containers unless we tell Docker which environment variables can be read from inside containers.

The translator will generate options (e.g. --env {VAR1} --env {VAR2} ...) and use the flags to run docker run command.