Add in explicit references to creds in ocelot.yml

[tjtelan]

Right now, we operate on an underdocumented secret (creds) injection functionality.

This is generally going to be ok if we improve the documentation, and do better to set user expectations.

However, if you need several secrets made available (like, if you need docker, maven, npm, and kubernetes creds), how is a user meant to achieve that? You just need to know how they move from registered creds to being injected into the buildspace. This is a lot of implicit behavior.

My biggest gripe is that the process doesn't document well enough for an ocelot.yml to be copied from a project in one (account, repo) to serve as the base of another (account, repo).

I think secrets can take more of an implementation style from kubernetes secrets. https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets

Particularly, by specifying the secret name, and at least specifying where/how the secret is mounted. Any feedback about how to approach this is welcome, as I haven't quite thought it through.

[shankj3]

Seems like it would make the ocelot.yml very large

[tjtelan]

Yup, but the ocelot.yml would fully describe secrets being used in a codified way.

In my opinion, this certainly offers a bit more hints to the user about what their build requires as inputs and supports the pipeline's overall representation in code. It would be in-line with how Shippable/TravisCI/CircleCI implement using secrets, with the expectation that secrets be provisioned prior to builds succeeding. (And gives some foundation for how we might add user accounts in the future so secret exposure may be limited to those who need it)

I'm mixed about keeping the implicit behavior at the moment (since it mostly does "just work"), but additionally including explicit secret references would only make this easier for a new user to understand.

[shankj3]

Idk, ocelot was made because build files get large and unwieldy easily, what exactly do you see this looking like

[tjtelan]

Developers who are new to Ocelot, or developers inheriting new projects don't have a story for how to manage or audit credentials for their build jobs. It's all buried in the wiki-documented, but kind of magical with the auto-injection/login/mount behavior. It's an all-or-nothing feature.

The ocelot.yml could (and probably should) help reinforce the words ocelot uses (creds as opposed to secrets -- even though I'm using them interchangibly), and help build confidence in making changes. Additionally, it will build a bridge to functionality that seems to be completely cli-driven only.

---

Perhaps when we add creds, there can be an additional option to turn off the implicit behavior?

Example: Add ssh keys to the level11consulting account that isn't auto-injected $ ocelot creds ssh add --identifier level11consulting-deploy --acctname level11consulting --sshfile-loc /home/leveler/.ssh/id_rsa --no-auto

This could reduce the level of unnecessary/accidental secrets injection (remembering that this not inherently more secure, just more declarative), and reduce the potential of unrelated integrations interfering with builds.

---

Lastly, what do I see this looking like? I don't think it has to end up adding a lot of bulk.

I think there are 2 useful representations :

Reference secrets at the top:

image: docker
buildTool: docker
creds:
  - repo/dockerrepo1
  - k8s/cluster1
stages:
  - name: Example stage
    script:
      - docker pull <privaterepo via dockerrepo1>/imagename
      - kubectl --kubeconfig=<path via cluster1> --context=cluster1 get ns

Reference secrets per stage:

image: docker
buildTool: docker
stages:
  - name: first stage
    creds:
      - repo/dockerrepo1
    script:
      - docker pull <privaterepo via dockerrepo1>/imagename
  - name: second stage
    creds:
      - k8s/cluster1
    script:
      - kubectl --kubeconfig=<path via cluster1> --context=cluster1 get ns

My first thought that these representations would be functionally equivilent. The only difference being that the 2nd example runs the commands over 2 stages. Creds load during the same non-user stages stage/preflight (as opposed to the secrets-per-stage example having just-in-time behavior where secrets are loaded prior per-stage since there's unnecessary exposure to having secrets changing during builds).

Parsing the config 2nd config would identify all the creds, and load them up during setup/preflight (I forget which one).

Referencing non-existent creds would fail validation with a non-existent creds error.

[shankj3]

there is something to be said for the simplicity of hey I know all builds under my vcs account will have my credentials without me adding in explicit injection for every buildfile

[shankj3]

It's much harder to undo the complexity, especially the perception of complexity for tools like this so forgive my abundance of caution; I just think there should be a lot of scrutiny for adding to the build file

[tjtelan]

The simplicity is a double-edged sword. It is rooted in a single-user/single-org mindset.

If we ever allow 3rd parties (like contractors) access to an established Ocelot instance, there is no structure for guarding/auditing the creds to only what they should be allowed to have access to (security through obscurity, or otherwise). We are making the assumption that we trust our users to never make mistakes, that they are allowed to have access to secrets if they have access to the network, and they never forget what secrets belong to what jobs.

If we ever needed to revoke a secret that was being used in multiple places, then we might actually have to find the gaps by letting jobs fail. Auditing is not a straightforward task, since the auditor needs to be pretty familiar with how Ocelot does secrets (which is not unreasonable. I just thought it should be said).

I hear you on the complexity argument, but honestly, much of the competition has has an explicit secret mounting user workflow. It starts on the backend with adding (or encrypting strings) secrets, and ends with a reference that is laid out in their build yml/job.

http://docs.shippable.com/platform/security/credential-mgmt/ https://docs.travis-ci.com/user/encryption-keys/ https://support.cloudbees.com/hc/en-us/articles/203802500-Injecting-Secrets-into-Jenkins-Build-Jobs

CircleCI does not. They have a very minimal secrets system that uses env vars and/or Hashicorp Vault. https://support.circleci.com/hc/en-us/articles/360006717953-Storing-secret-files-certs-etc-

---

Lastly, I strongly believe the ocelot.yml should help educate developers to have the Infrastructure as Code mindset. Implicit secrets (without an explicit reference, or an opt-out to automatic injection) leaves a gap in documenting a build's external dependencies.

Here's an analogy, since you know Kubernetes:

What if all secrets in a namespace were automatically mounted into every pod in the namespace? This is essentially the situation we have.

The Kubernetes Controller has both an implicit (like service accounts) and explicit behavior https://kubernetes.io/docs/concepts/configuration/secret/

[shankj3]

The contractor and security reference is not relevant because they still would have access to all secrets , they'd just have to mount them themselves.

[shankj3]

To use Kubernetes as an argument is also a double edged sword, as it is so complex and confusing that most developers make their PE people do all the work. However, it is your prerogative, considering this is now under Level 11's umbrella.

[tjtelan]

I used Kubernetes simply bc I know you know it well. K8s Namespaces (vs accounts), pods (vs build jobs), and secrets are a concise comparison w/o bringing in all of K8s moving parts. We namespace on accounts, and all secrets are visible to the job in that namespace (disregarding k8s RBAC arguments bc Ocelot doesn't support auth at the moment).

And any security talk is relevant since we lack basic security measures. It's a huge bummer to say that secrets in your build system are going to expose more sensitive info than a contractor has clone/commit access to because they have unauthenticated, root-like access. I think we shouldn't disallow it right now (as we have no structure for replacement), but it should be harder and more obvious when trying to take advantage of this security hole.

Adding in security features now will be easier than adding it in when there's a larger userbase. I believe it helps us talk about credentials in an objective way.

• Which jobs are using cred X?
• When was the dependency on cred X added?
• When was cred X deleted?
• Job failed to start bc cred X has been deleted and it is required (as opposed to a runtime error after a 10-15 minute build)

By allowing to mount them explicitly, then there is documented intention for using the secrets. What even is the purpose of backing our secrets in Vault if we're going to be shy about securing them at the user level? This is opt-in behavior anyway since there probably should be workflows with implicit secrets.

[shankj3]

I agree with locking down credentials, I think there should be more security and I talked about that considerably the last week before I left. This just wasn't considered in the scope of the original issue. For this type of auditing are you talking about actually tracking cred usage in a table? This seems to be evolving more into a complete credential overhaul, which I am all for if it means more security. But that does mean a lot more than the trivial section of adding paths into an ocelot file. You don't get any of those bullet points from adding that creds block.