Way to "dry-run" a decrypt

[james-callahan]

For file validation purposes, we'd love a way to "dry-run" a decrypt, so that given e.g. a sops-encrypted YAML file; everything that would be decrypted, would instead be replaced with *********

Basic example

example.enc.yaml: ```yaml example_key: ENC[AES256_GCM,data:GuFvI4uWtnutMgBE1w==,iv:rYPH8/XFF1oBEeo+hzHkjr8yyn1SDObITONbQLqrMhE=,tag:cuvAbQQBAYKCEcMlJk3W6w==,type:str] sops: kms: [] gcp_kms: [] azure_kv: [] hc_vault: [] age: [] lastmodified: "2022-03-03T03:39:40Z" mac: ENC[AES256_GCM,data:1v6SMIebvS7VB9XFzWWP4Ti1r2JQea/lGGSRTKSfOUaF9TXH7eRRxx9ociH/F/iq72DOZV940tZZXxb42dSkLBssVRWOA+vcEOhGK9KdUIci7pgt2wSL/n3cxUc+f0Vr1bfHW/aMsW7oJyYDNQRIzEKMXUVem+Sxai7+w0gw8nE=,iv:ET0n680qIH8TJ+8Lt/t0Atm9pro9VsKa+eUJTqEMsgQ=,tag:aN6w/BjuZpgnzSTpX/2OUA==,type:str] pgp: - created_at: "2022-03-03T03:39:27Z" enc: | -----BEGIN PGP MESSAGE----- hQIMA3lxnYU9D1ZiAQ//dWNV3iTDe5YSJRz7+mzMBWt7kJ4PWWizNvV+iEtCnHaq TZdfFmS6LPwkXDKu7Z8B/b8ev+TIxdM0JBbpfPRxj09N/Sh9pmYjl1tLQpuKx9KZ vtEPDbzTlIXyuhBBBbWLZYsB0ImOGIIbrxIdeB80+kGGQU94isBdVWEqHoEzPDk5 /jJF6AxgOMCf5sZ+/sYNNMl3Mf0l+NwI6TfV7Pqd6lbQuxs4umIsYMCj3O3eatYh 6g5wDPvnZNSE0m3DQKPMaI21fRkJRyexbjkYFx6WYUwbAjZ8Tpxi7orkOdiPtiMn IAdxIIT0T7kVGzk4IMB5tTLXcs09FbBWo0x6q5raHeVuYG19s0bjGYx0wfi4N+gH bCsqnYj/klWxR/+KJ2fIqX6kx1vEphhMdwCXFP/nkX9OCF/l9x8EkWo4gUhpUPT5 mFbfLJulQNKzQxMq8Zds75HaUtMxT1WS2nboJS05iDfd/M+JdUMsz8Nc9NPbTV82 tZjyp2KwvcG7jij4UcqUxfu+tb0h4qM8o9m4WbWFap5gPC583btyrZZfdebbDjVp cVoCT8yTz349NxueQXiR/EYYN4TbT84Se6SiogzlFyk/1eEA2YFcm66/dgmsvDZz 6Ka7PSgsWL4U+ifJBHJGzToP7NOm4ESRj24WslBeK6PQumYNy1AM1Kv9eUFiBc/S XgHYRlewZRrLz5Lsl1RuHL9VBRZ716KGWNQDHGlQGccQBMUTvp2AlbvnSlGByNvN U5vB1Cy3g3bT5knBSQOjVwylbstyHAe618fttY6sJVLLQ00V8EH9k14247/4m0A= =nmOn -----END PGP MESSAGE----- fp: C67898BEDF14E103FDFDB067A37403A776EBF4E5 unencrypted_suffix: _unencrypted version: 3.7.1 ``` `sops -d --dry-run="******" example.enc.yaml` output: ```yaml example_key: "******" ```

When an `encrypted_regex` or other field is present, that would be respected.

```yaml foo: ENC[AES256_GCM,data:rZN1,iv:57K06DrR1eOGFUTlux/91WiPc95c3enruqRhm6uh5ok=,tag:xZNyP+8tyRaOYGEsWKaufQ==,type:str] bar: bar sops: kms: [] gcp_kms: [] azure_kv: [] hc_vault: [] age: [] lastmodified: "2022-03-03T03:41:16Z" mac: ENC[AES256_GCM,data:mm1hLjZl23TgXHhUv5hwRYxR6cO3pRamS0ktggoJ5k3z2wVbxh24tNcU2X4cChz8K7NFq4mAwZR77GOpoYv5pHt4gsjc09T2NUQkFPKSxKrxjGhpRcBVKXArPS8V61mAY7qOZsVcasEmFgemO+6uNVSh9wk1vKy0VIMbZ9RjijE=,iv:jYH+D/lLnG3J02mjr/fTFIPyW9DCaMNW9CiZG4NTI7Y=,tag:R63f1hXAV3bTSXSLRZ5/oQ==,type:str] pgp: - created_at: "2022-03-03T03:41:03Z" enc: | -----BEGIN PGP MESSAGE----- hQIMA3lxnYU9D1ZiAQ/+OY4J8cBgDEmlfzt37TOqFcz3p5o5ZrQupPTCkAJn90ZM S+ZXcfBJK3ZnxokB7dSV5aY4UVpoV3a83SEUeO5p3zShlMR/SvOwpYa+IdHaaL0A A/POGnYK0aI8DMl7J0KX2MofiLHuQlSxmDatQBH6wvTO078VdkIrhBJZLdNweCiV xFus3IvtAsgx9lQ3wzm7eXYcGhXwhvGBoEEm7kI9B8l9lrF+WKsqUTmyoU6SZh9Y 4qXxwdkAclFx5bSbVHpI4t6L5/e4tzB7lRlxXH4hh/sXnpZRWc/c0tUYoS/Hecss 3vGcmrQNmhnzpFuFztJV8pim6TLnSi6IPX3puVYDfoOCWyV9cxUpD0Ry/658cmFv ND0urR0LgCRQovI6l7BVDywruGq58HH1tPWwN7PtUsiJpSDysaMqp1vZj3BhENgI sMr6+dJhL+mY25EQrKGyXO0W36Yb3Jm2CjI+TjawwswQ0nzYpR4RqtBXwh7lRBaw MWQvyNLRdN7dWjxfo+RkP8w1bhIzLR7G2imNRa8IkurEqvtR6iqfgMDlKR8JQVES fwLgtuz94WGx86XICWjg3CVzJAavLStK8AdRD815iwYC9Wuui1ylDQ7IB67vjDKO udFYb8o97hdXrot7ewwZH1g64G/RJivVpU5AYU0C0cjohZmFnvnKtFEQGJ8PAYPS XAEbb6vVQw0X7xQSuww7INQaRCyF6LLCnro2C9+zsbFoc4iisomu9QL+qYGJP01N VnFNHGsL405lvQrDkHgFnx2fOXHdmZWZtmL/NG+QaB3+6FpneHaHmsiQyjiT =Bu76 -----END PGP MESSAGE----- fp: C67898BEDF14E103FDFDB067A37403A776EBF4E5 encrypted_regex: foo version: 3.7.1 ``` `sops -d --dry-run="******"` output: ```yaml foo: "******" bar: bar ```

[ajvb]

@james-callahan can you describe your use-case for this?

[james-callahan]

@james-callahan can you describe your use-case for this?

I'd like people to be able to validate their sops encrypted files; without granting access to the actual values.

One destination I'd like to get to, is running ksops in this newly proposed dry-run mode; and then validate the kubernetes manifests that come out.

[ajvb]

I'm not a fan of this. First off, if someone has the ability to decrypt a value then they have access to it. If this is purely about decrypting a value remotely then redacting the display of it, that should be implemented within the system displaying the value.

[james-callahan]

if someone has the ability to decrypt a value then they have access to it. If this is purely about decrypting a value remotely then redacting the display of it, that should be implemented within the system displaying the value.

No, the intention is a way to get a "sample" (in terms of fields present) of a decrypt when you don't have the key/ability to perform a decryption.

[james-callahan]

Have you considered this further at all?

[fedaykin-b]

Hi, a bit too late on this topic.

Recently I had to write a pipeline with SOPS, and I thought a dry-run could help me to check what files were encrypted. For context, the folder with YAML had both encrypted files and without encryption. At first I thought something like:

for file in path/to/manifests/*; do
    sops -d --dry-run $file && sops -d -i $file
done

but I just realized that as I would test all the files, actually I could just rewrite this as the operation itself:

for file in path/to/manifests/*; do
    sops -d -i $file || true
done

Usually I use dry-run in other tools to validate that an operation could have success, but I think SOPS doesn't have to work this way, because it is a All-or-Nothing operation, there is no possible way to validate a symmetric cipher without really doing the decryption itself.

[james-callahan]

I think SOPS doesn't have to work this way, because it is a All-or-Nothing operation, there is no possible way to validate a symmetric cipher without really doing the decryption itself.

A dry-run of sops wouldn't do the decryption; but instead

• validate that the file is syntactically correct (e.g. not invalid yaml/env file)
• validate the contents of the sops field matches the sops schema (e.g. has a mac sub-field)
• remove the sops field that sops adds on output and fix other escaping/renames it does

[felixfontein]

@james-callahan I don't think what you are describing should be called dry run. That's a kind of lint operation for the sops file format, not a dry run for decryption (which would be a decryption without printing the result).

I'm also not sure what your last bullet point should actually do, this sounds like a reformatting tool, and not like a lint or dry run operation. Or are you simply describing a YAML/JSON/env/... parser?

[fedaykin-b]

Something like a sops --validate <file> (not a dry-run) would be useful for use cases like this one about pre-commit hook validation: https://github.com/onedr0p/sops-pre-commit/issues/24.

As SOPS doesn't have such functionality, the project above relies on custom logic that is restricted to the underlying structure of K8s Secrets. It would be useful for script integration if SOPS could offer a command to signal that a file looks like an encrypted file, or that a file doesn't have the field described in the suffix/regex command flags.

[james-callahan]

I'm also not sure what your last bullet point should actually do, this sounds like a reformatting tool, and not like a lint or dry run operation. Or are you simply describing a YAML/JSON/env/... parser?

When sops encrypts a file, it adds a sops element to it to keep the metadata for itself.

When sops decrypts a file, it doesn't show the sops element in the output (implemented at e.g. https://github.com/getsops/sops/blob/c6ce6dfcae44032f6a9b5e9166f44764534f301d/stores/yaml/store.go#L285). When running a dry-run, sops should equally not show the sops element in the output.