phoban01 commented 1 year ago

Context

The following is a proposal for supporting the encryption of Timoni values (based on prior art, see: https://github.com/phoban01/cue-sops).

The proposal was first mooted in #71.

Proposal

Timoni enables the encryption of sensitive values via built-in SOPS support.

Given the following config definition:

#Config: {
  api: {
    url:  "https//api.github.com/user"
    token: string
  }
}

A values file providing an API key is marked as sensitive using the @secret() annotation:

# api-values.cue
values: {
    api: token: "gh_personalaccesstoken" @secret()
}

The plaintext value can then be encrypted in-place:

timoni encrypt -f api-values.cue

This will produce the following result:

# api-values.cue
values: {
    api: token: "ENC[AES256_GCM,data:0SeH+BIX6SwJBsgwLmDOJHU7,iv:Fx1bpRKrz4wKztuEXMfa0KuRqLcOu9ZLT8OYdH+i58c=,tag:IoDhNZpGnGhqmDllgUVdUg==,type:str]" @secret()
}

// DO NOT EDIT: auto-generated by timoni
sops: {
    kms:      null
    gcp_kms:  null
    azure_kv: null
    hc_vault: null
    age: [{
        recipient: "age1ethasxep4zkax64yfx35rn2t4yeul4254w764l9gtasvn2rwpv7s733dq7"
        enc: """
            -----BEGIN AGE ENCRYPTED FILE-----
            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBQMk43bjFuUytFNTlIclNW
            Y1RnNWEwc2FGOUd4VW5NODNwdEVKOXlJd2k4ClNubDN0Qktuck5IVnN6ZjZBOTEz
            OFhNRUc3aUs1Y09DQTF6OTlWRU9ZQ00KLS0tIGdGTUlZWUJyVkZKZXdvMzZhV294
            c0E5bHVkSHc0MkhFUnhiODFlbzV5SE0KlNEhfwHl/VDZzfkpGb2/s7KbTFRA4U/K
            u5OM5P2YTvpSkmVbdVLLcX7eFHVyLZOukarFXEZ65rq9baMO0lJ3Vg==
            -----END AGE ENCRYPTED FILE-----

            """
    }]
    lastmodified:    "2023-04-01T12:00:00Z"
    mac:             "ENC[AES256_GCM,data:heUT68PAirogTfcV+4pR8RNjx+d3cEE+Zn5e97xNy2wJvwZ4ecxnxItDj60E71aTK80UxCxkWkfjg2ZGKscPCMKoAXBkli6y/ab0e0+9uulvqjbd51m7mzGo/DMt65Ab7C6hq6S/VuI9JvvR7OVdgpvrliQzlCx2VENYNG6/r/0=,iv:gPvKgisLoTuOEIMNQgwY3zhPUEDkjJrRTyGWEEMr1ww=,tag:P8OlN/XDfWZqo6ZIchwbzw==,type:str]"
    pgp:             null
    encrypted_regex: "token|SECRET"
    version:         "3.7.3"
}

Timoni will decrypt values before applying an instance to the cluster:

timoni -n default apply gh-app \
  oci://ghcr.io/phoban01/modules/gh-app \
  --values api-values.cue

Timoni can also decrypt a file in-place:

timoni decrypt -f api-values.cue

Encrypt multiple fields

To encrypt all fields in a file (optionally matching field names via regex), it is possible to use a global annotation:

# api-values.cue
@secret(include="[regex-pattern]", exclude="[regex-pattern]")

values: {
    api: token: "gh_personalaccesstoken"
}

SOPS configuration

It is possible to specify the encryption service used by SOPs per field:

# api-values.cue
values: {
    age: "supersecret" @secret(age="age1yt3tfqlfrwdwx0z0ynwplcr6qxcxfaqycuprpmy89nr83ltx74tqdpszlw")
    pgp: "supersecret" @secret(pgp_fp="85D77543B3D624B63CEA9E6DBC17301B491B3F21")
    aws: "supersecret" @secret(aws_kms="arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e")
    aws: "supersecret" @secret(aws_encryption_context="Environment:production,Role:web-server")
    gcp: "supersecret" @secret(gcp_kms="projects/my-project/locations/global/keyRings/sops/cryptoKeys/sops-key")
    azure: "supersecret" @secret(azure_kv="https://sops.vault.azure.net/keys/sops-key/some-string")
    hashicorp_vault: "supersecret" @secret(hc_vault_transit="https://vault-server:8200/v1/sops/keys/firstkey")
}

It is also possible to define encryption service providers globally using the @sops() annotation. Providers can subsequently be referenced using labels:

# api-values.cue
@sops(label="aws-master", aws_kms="arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e")
@sops(label="hashi-vault", hc_vault_transit="https://vault-server:8200/v1/sops/keys/firstkey")

values: {
    aws: "supersecret" @secret(label="aws-master")
    hashicorp_vault: "supersecret" @secret(label"aws-master")
}

If a single @sops() annotation is present then labels can be omitted and the specified service will be used to encrypt all values:

# api-values.cue
@sops(aws_kms="arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e")

values: {
    aws: "supersecret" @secret()
    hashicorp_vault: "supersecret" @secret()
}

Bundles

Values provided in a Bundle may also be encrypted. Encryption services can be defined as part of the bundle spec and then referenced by name in the @secret annotation.

#Bundle: {
    apiVersion: string
    encryption: [{
      name: string
      type: age | pgp_fp | aws_kms | aws_encryption_context | gcp_kms | azure_kv | hc_vault_transit
      value: string
    }]
    instances: [string]: {
        module: {
            url:     string
            digest?: string
            version: *"latest" | string
        }
        namespace: string
        values: {
          api_key: "123457890abcdefg" @secret(name=string)
        }
    }
}

To encrypt a bundle use the bundle encrypt subcommand:

timoni bundle encrypt -f bundle.cue

Sensitive values will be automatically decrypted when the bundle is applied:

timoni bundle apply -f bundle.cue

Timoni bundle diffs will decrypt both previous and current sensitive values and display the diff in cleartext:

timoni bundle apply --dry-run --diff -f bundle.cue

stefanprodan commented 1 year ago

@phoban01 thanks for the proposal, this looks great to me.

Can you please include Bundles, as stated in the docs, Bundles are preferred over using Values and imperative commands.

phoban01 commented 1 year ago

@stefanprodan Updated proposal to take bundles into account.

primeroz commented 1 year ago

I just started looking into timoni and was wondering what the status of sops integration is.

Is this proposal proceeding right now ?

Thank you

stefanprodan commented 1 year ago

Is this proposal proceeding right now ?

This is a not top priority right now, or at least I personally have no plans to work on this in the near future. Decryption within the Timoni CLI has little value, it would make more sense to implement this for the timoni-controller, when that will be a thing.

Currently there are several ways of injecting secrets at apply time in Timoni Bundles using runtime attributes @timoni(runtime:string:SECRET-NAME) (docs here: https://timoni.sh/bundle-runtime/).

Injecting secrets in CI

When using a CI runner to deploy apps with Timoni, you can pass secrets from the runner secret store to Timoni's Bundles.

Example of a bundle that injects the GIT_TOKEN secret:

bundle: {
    apiVersion: "v1alpha1"
    name:       "flux-aio"
    instances: {
        "cluster-addons": {
            module: url: "oci://ghcr.io/stefanprodan/modules/flux-git-sync"
            namespace: "flux-system"
            values: git: {
                token: string @timoni(runtime:string:GIT_TOKEN)
                url:   "https://github.com/my-org/my-private-repo"
                ref:   "refs/head/main"
                path:  "./test/cluster-addons"
            }
        }
    }
}

In a GitHub workflow, you can map secrets from GitHub secrets to env vars, that Timoni will use at apply-time:

export GIT_TOKEN=${{ secrets.GITHUB_TOKEN }}
timoni bundle apply -f flux-aio.cue --runtime-from-env

Injecting secrets from Kubernetes Secrets

The same GIT_TOKEN from the above example, can be injected from a Kubernetes Secret, assuming you're using some external-secret controller that syncs secrets from a Vault in etcd.

Example of a Timoni bundle runtime that fetches the GIT_TOKEN from the cluster:

runtime: {
    apiVersion: "v1alpha1"
    name:       "production"
    values: [
        {
            query: "k8s:v1:Secret:infra:git-auth"
            for: {
                "GIT_TOKEN": "obj.data.token"
                "GIT_CA":   "obj.data.\"ca.crt\""
            }
        },
    ]
}

At apply-time you pass the runtime definition and Timoni will read the secret from the cluster and use it when applying the bundles:

timoni bundle apply -f flux-aio.cue --runtime runtime.cue

Injecting secrets with SOPS

When using SOPS, you can decrypt the secrets and pipe those values to env vars then use --runtime-from-env.

Another option is to extract the secret values of a Timoni Bundle to an YAML file, that you encrypt/decrypt with SOPS.

Example of Bundle composition

Main bundle file bundle.main.cue:

bundle: {
    apiVersion: "v1alpha1"
    name:       "flux-aio"
    instances: {
        "cluster-addons": {
            module: url: "oci://ghcr.io/stefanprodan/modules/flux-git-sync"
            namespace: "flux-system"
            values: git: {
                // The token is omitted here!
                url:   "https://github.com/my-org/my-private-repo"
                ref:   "refs/head/main"
                path:  "./test/cluster-addons"
            }
        }
    }
}

Bundle partial in YAML format bundle.secret.yaml:

bundle:
  instances:
    cluster-addons:
      values:
        git:
          token: my-token

Assuming the bundle.secret.yaml file is kept encrypted with SOPS, at apply-time you can run the SOPS decryption, and pass the plain YAML to Timoni's apply command like so:

sops -d bundle.secret.yaml > bundle.secret.plain.yaml

timoni bundle apply -f bundle.main.cue -f bundle.secret.plain.yaml

rm bundle.secret.plain.yaml