If `.sops.yaml` contains a `path_regex` sops_encrypt doesn't work

[onedr0p]

Hi 👋🏼

I am trying to use this module but it doesn't work when I have a .sops.yaml configuration file present. If I remove the .sops.yaml config file my task works.

env

✖ ansible --version
ansible [core 2.15.1]
  config file = None
  configured module search path = ['/Users/devin/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/homebrew/Cellar/ansible/8.1.0/libexec/lib/python3.11/site-packages/ansible
  ansible collection location = /Users/devin/.ansible/collections:/usr/share/ansible/collections
  executable location = /opt/homebrew/bin/ansible
  python version = 3.11.4 (main, Jun 20 2023, 17:23:00) [Clang 14.0.3 (clang-1403.0.22.14.1)] (/opt/homebrew/Cellar/ansible/8.1.0/libexec/bin/python3.11)
  jinja version = 3.1.2
  libyaml = True

❯ ansible-galaxy collection list | grep sops
community.sops                1.6.4

sops configuration file

# /tmp/test/.sops.yaml
---
creation_rules:
  - path_regex: kubernetes/.*\.sops\.ya?ml
    encrypted_regex: "^(data|stringData)$"
    key_groups:
      - age:
          - "{{ bootstrap_age_public_key }}"
  - path_regex: ansible/.*\.sops\.ya?ml
    key_groups:
      - age:
          - "{{ bootstrap_age_public_key }}"

example task

    - name: Template out encrypted files
      community.sops.sops_encrypt:
        path: "/tmp/test/kubernetes/test.sops.yaml"
        encrypted_regex: ^(data|stringData)$
        age: ["{{ age_public_key }}"]
        content_yaml: "{{ lookup('file', 'templates/test.sops.yaml') | from_yaml }}"
        mode: "0755"
        force: true

ansible logs

TASK [Template out encrypted files] ******************************************************************************************
failed: [localhost] (item={'src': 'kubernetes/cert-manager-secret.sops.yaml.j2', 'dest': './kubernetes/test.sops.yaml', 'encrypt': True}) => {"ansible_loop_var": "item", "changed": false, "msg": "error with file to stdout: ErrorGeneric exited with code 1: error loading config: no matching creation rules found\n"}

It would be nice if we could unset config_path from always being present as I have confirmed that without this present I can encrypt. However maybe there is something else going on?

[onedr0p]

Actually this looks might be related https://github.com/ansible-collections/community.sops/issues/104

[onedr0p]

For now I added a tasks to rename the sops configuration file, use sops_encrypt and then restore the sops configuration file. I wonder if there is a more elegant way to solve this though?

[felixfontein]

It is exactly #104: /dev/stdin is not covered by any of your creation_rules, so sops itself balks. This is essentially https://github.com/getsops/sops/issues/594.

I hope that with the new maintainers of sops we now have a chance to get this fixed (i.e. something implemented that helps to work around this).

[felixfontein]

LOL, I just realized I managed to overlook that you are one of the new maintainers :D Sorry for that.

[felixfontein]

You should be able to reproduce this when changing directory to /tmp/test/kubernetes and running echo 'foo: bar' | sops --encrypted-regex '^(data|stringData)$' --age '...' --encrypt /dev/stdin.

While using --output as suggested in https://github.com/getsops/sops/issues/594 helps when doing this on the CLI (by adding --output test.sops.yaml), it won't help sops_encrypt since it needs to make sure the file is written with the correct permissions etc. in an atomic fashion. For that, we cannot use --output (and even if we could, it would be the wrong path to use). So basically we need another way to tell sops "assume this filename when looking up the creation rules", or (alternatively) a way to tell it "use what we provide on the command line, and not what is in .sops.yaml". AFAIK even the latter is not possible at the moment (and even if it would, we would need a new feature to allow using it).

[onedr0p]

LOL, I just realized I managed to overlook that you are one of the new maintainers :D Sorry for that.

No problem :D glad to see you are also helping out there as well!

I'll rename this issue and hopefully we can get a fix for it once upstream adds support for the issue you linked.

[clearlybaffled]

I'm not sure if this is exactly a good idea, but I just made a new creation_rule with path_regex: /dev/stdin and it works without complaining...

Thanks for creating this ticket btw, i just spent a couple of hours trying figure out why it worked just fine on the command line but not in ansible. It was a couple rounds of google before i landed here..

[onedr0p]

@clearlybaffled that works, not sure why I didn't think of that. Thanks for the tip!

[felixfontein]

I've started with https://github.com/getsops/sops/pull/1332, which allows to fix this.

[felixfontein]

190 implements a SOPS version detection and will fix this for SOPS 3.9.0 and newer using the new --filename-override encryption option.