Support for docker-compose secret syntax

[bradrydzewski]

docker supports secrets which are stored as key value pairs and are mounted as volumes in the container. So for example slack_token secret would be available at /run/secrets/slack_token

this can now be defined in your docker-compose file: https://docs.docker.com/compose/compose-file/#short-syntax-1

since the project attempts to conform to the docker-compose specification and implement its concepts, we should evaluate how this fits into drone and can be used with our secret implementation.

example syntax 1

pipeline:
  deploy:
    image: plugins/slack
    channel: foo
    secrets: [ slack_token ]

example syntax 2

pipeline:
  deploy:
    image: plugins/slack
    channel: foo
    secrets: 
      - source: slack_token
        target: slack  # mounts to /run/secrets/slack

Plugins could (for example) have the ability to source parameters from files, in addition to sourcing from flags and environment variables.

It will also be interesting to see if Docker ends up supporting pluggable secret backends. Will they support vault and others, for example, and what will this syntax look like in docker-compose.

How would this be used? And why?

We tried to move away from secret interpolation for improved security, limiting which containers had access to which secrets. This ended up being terrible for UX and support. There were also some limitations as described in https://github.com/drone/drone/issues/1888

If we consider the below example:

drone secret add octocat/hello-world --image=plugins/slack slack_token=....
---
pipeline:
  deploy:
    image: plugins/slack
    channel: foo
    secrets: [ slack_token ]

And then consider how the extended docker-compose syntax could be used to solve the issues faced in https://github.com/drone/drone/issues/1888

drone secret add octocat/hello-world --image=plugins/slack slack_alt_token=....
---
pipeline:
  deploy:
    image: plugins/slack
    channel: foo
    secrets:
      - source: slack_alt_token
        target: slack_token

This might allow us to revisit our attempts to improve secret security while removing limitations. This still leaves the UX issues which perhaps could be resolved by improving our matching logic. For example:

• Match the image names (strip tags before comparing)
• Maybe make the --image flag optional for private repositories
• Promote using the UI for secret management
• Provide better documentation

Or alternately we could remove --image all together and try to implement some sort of gating based on yaml changes. The --image flag was really meant to protect against malicious pull requests attempting to expose secrets. Perhaps this could be removed all together with simple yet effective gating policy.

[bradrydzewski]

note that this is related to #1901

[bradrydzewski]

In preparation I authored https://github.com/urfave/cli/pull/614 to provide the ability to read flag values from file. Hopefully we can get it merged, if the maintainers are interested in extending the functionality.

[bradrydzewski]

OK, for now the below example is working in master. Secrets are sourced from environment variable instead of file. Note that I'm still planning to source secrets from file, but this won't really impact the end user either way.

drone secret add octocat/hello-world --image=plugins/slack slack_token=....
---
pipeline:
  deploy:
    image: plugins/slack:1.0.0
    channel: foo
    secrets: [ slack_token ]

Note that for improved usability we exclude tags from our matching logic. This means plugins/slack matches plugins/slack:1.0.0.

[atomi]

So for an SSH_KEY env variable would this do?

drone secret add octocat/hello-world --image=plugins/slack SSH_KEY @/home/atomi/.ssh/id_rsa
---
pipeline:
  deploy:
    image: plugins/slack:1.0.0
    channel: foo
    secrets: [ SSH_KEY ]

[bradrydzewski]

yep :)