New feature : sops support for encryption

[learsix]

Hi,

I'm using sops in several projects and I've searched how to integrate it with git.

For the moment I'm only using gpg encryption with sops so it made sense to try to adapt git-secret. I've been doing some work in my own fork (no test or doc yet).

Would you be interested to merge this feature ?

[joshrabinowitz]

I think it would be interesting to see the PR you've worked up. I hadn't heard of SOPS until just now.

I can't speak to the likelihood of the PR being merged/accepted (especially as git-secret is relatively easy to describe now, being built on gpg), but it sounds like an interesting project.

Thanks for making us aware! I look forward to seeing your PR (though again I cannot guarantee acceptance of your PR)

[joshrabinowitz]

@learsix I've been thinking about this more, and how it would be nice to add SOPS support to git-secret.

Could it be implemented by adding a -Sswitch to git-secret? What would a first integration of git-secret and SOPS https://github.com/mozilla/sops look like from a user's perspective?

[learsix]

@joshrabinowitz I'm almost through a first implementation. I still have tests to add before submitting a PR (I try to stick to the contributing rules ;) I'm not sure I'll have time to push it by the end of the week but it will definitely be done in the next days.

Below a few indications about the way I've implemented it. If you have feedback, I'd be happy to update the code before pushing the PR.

Impacts for user / git-secrets commands

From the user's perspective it's integrated as described below. All other commands are not impacted. Everything defaults to git-secret as it works now.

choosing btw sops and pgp

sops or legacy pgp "mode" is chosen at the repo level. The user can define it with the init command and it defaults to pgp.

Example: git secret init -m sops

Notes:

• based on your previous comment, maybe you'd like to replace -m sops with a -s
• for now, mode can be changed by re-applying git secret init with a different -m option. I've not changed messages so the user won't guess it. There will still be work to do here.

using sops group feature

To support groups (ie. Shamir Secret Sharing), the user can choose a group when an email address is included. If no group is provided it defaults to a default group.

Example: git tell -g group1 foo.bar@example.org

To display groups I've extended the whoknows command with the long display option to add -group <groupname> at the end of each line. It's only displayed when sops mode is enabled in the repo.

Using keyservice during decryption

User can pass keyservice urls with the -k option for the reveal command. Multiple -k options are supported.

The user can also set a global git config param (key: git-secret.keyservice).

Note: by the way, we cannot pass the gpg passphrase to sops, so the -p option cannot be used. Instead the user will have to rely on a local gpg agent (or remote keyservice).

Code impacts

sops vs pgp mode

I've added functions in _git_secret_tools.sh to manage (set / read) the mode. Right now it's saved in the local git config with the git-secret.mode key.

The init command has been modified to generate a .gitsecret/sops dir in the same way keys and path subdirs are generated (see below - adding new users).

decryption

Decryption was already managed with a function in _git_secret_tools.sh. I've created _decrypt_gpg and _decrypt_sops. The _decrypt function calls one of them depending on the mode of the repo.

encryption

Encryption was directly handled in the hide command. I've extracted all of it and created functions _encrypt, _encrypt_gpg, _encrypt_sops in _git_secret_tools.sh. It works like _decrypt functions.

One ugly hack: encryption relies on the keyring contained in the .gitsecret/keys directory. There is no way to pass that argument to sops directly. We can only pass a filepath that's supposed to be a wrapper for gpg. I've changed main.sh : if it detects one gpg option when git-secret is passed, a gpg wrapper function is called that will add the --homedir param. It's not very nice, but it's the only way to keep git-secret in a single file. I've also considered using softlinks in a busybox way, but that would require to change the install process.

Encryption with sops is configured with .gitsecrets/sops/sops.yml file passed to sops. In this file we can specify groups and gpg key fingerprint to use. That file is generated when adding new users.

adding new user

Sops specifics are handled after the git-secret tell process.

• first a .gitsecret/sops/groups file is updated to store groups and associated user ids. A few functions are introduced to manage this file.
• a few functions have been added to extract fingerprints from git-secret keyring (since sops does not work with user ids)
• those functions are used to generate a .gitsecret/sops/sops.yml based on groups / ids stored in the first file and key fingerprints extracted with the functions above.

The drawback here is that if someone modified the keyring directly without using the git-secret command, then the keyring and the sops config file will be desynchronized.

Notes:

• the kill-person command has been modified symetrically to remove their keys and ids from sops configuration.
• the whoknows -l command uses content of the .gitsecret/sops/groups.
• The .gitsecret/sops/groups structure provisions the possibility to add google/aws kms ids. The rest of the code should also support these with a few modifications.

[simbo1905]

Encrypting the values of yaml rather than the whole file would be very useful as many templated infrastructure projects use yaml (e.g., kubernetes, ansible, aws cloud formation templates, ...).

I was thinking that it would be a cool feature to find or write some code that encrypts yaml keys and add that as a feature to git-secret. SOPS does that but it also seems to do a lot of other things. What is it that SOPS is missing that git-secret could add to it?

[learsix]

SOPS is missing integration with git, which means most of the things that git-secret does really well with gpg:

• keeping track of the public keys when you use the gpg backend
• not having to search for key fingerprint (sops does not support uid with gpg)
• inserting unencrypted files in .gitignore to avoid commiting them by mistake
• having an extended git command line (ie. not having to learn sops)

On the other hand sops has 2 other features that can enhance git-secret:

• the possibility to rely on GC & AWS KMS (ie. not only gpg)
• the keyservice function that allows to bring a remote proxy to access keys (ie. like ssh agent forward, that can be also done with gpg but with several tricks) - that could interesting in a CI/CD context.

[simbo1905]

That’s is a really helpful comparison of features. This is a really interesting proposal.