add `vault_read` module and lookup plugin

[briantist]

SUMMARY

Adds a vault_read module and lookup plugin.

This is the first module in the collection, and the second lookup.

These are meant to do generic reads from Vault.

ISSUE TYPE

• New Module Pull Request

COMPONENT NAME

vault_read

ADDITIONAL INFORMATION

The hashi_vault lookup also does generic reads, but it also tries to interpret the returned data as possibly being a kv store type secret.

These plugins replace the special casing and other things that got added onto hashi_vault over the years, and will be the basic way to do generic reads.

Other plugins will deal with more specific purposes, like reading kv store secrets.

[codecov[bot]]

Codecov Report

Merging #156 (a0de091) into main (54b007a) will increase coverage by 1.43%. The diff coverage is 96.10%.

@@            Coverage Diff             @@
##             main     #156      +/-   ##
==========================================
+ Coverage   89.99%   91.43%   +1.43%     
==========================================
  Files          37       40       +3     
  Lines        1469     1587     +118     
  Branches      113      124      +11     
==========================================
+ Hits         1322     1451     +129     
+ Misses        136      123      -13     
- Partials       11       13       +2     

Flag	Coverage Δ
env_docker-default	91.43% <96.10%> (+1.43%)	:arrow_up:
integration	75.24% <89.61%> (+1.38%)	:arrow_up:
py2.6	36.13% <55.26%> (+1.17%)	:arrow_up:
py2.7	83.23% <96.10%> (+1.68%)	:arrow_up:
py3.10	90.54% <94.80%> (+1.43%)	:arrow_up:
py3.5	83.49% <96.10%> (+1.66%)	:arrow_up:
py3.6	83.49% <96.10%> (+1.66%)	:arrow_up:
py3.7	83.49% <96.10%> (+1.66%)	:arrow_up:
py3.8	90.61% <96.10%> (+1.50%)	:arrow_up:
py3.9	90.61% <96.10%> (+1.50%)	:arrow_up:
sanity	37.48% <48.05%> (+2.04%)	:arrow_up:
target_ansible-doc	100.00% <ø> (ø)
target_auth_approle	89.47% <ø> (ø)
target_auth_cert	56.20% <ø> (-0.31%)	:arrow_down:
target_auth_jwt	91.30% <ø> (ø)
target_auth_none	100.00% <ø> (ø)
target_auth_token	71.42% <ø> (ø)
target_connection_options	74.76% <ø> (ø)
target_controller	66.81% <ø> (-0.23%)	:arrow_down:
target_import	37.48% <48.05%> (+2.04%)	:arrow_up:
target_lookup_hashi_vault	79.72% <ø> (ø)
target_lookup_vault_read	87.17% <87.17%> (?)
target_module_utils	90.25% <ø> (+1.49%)	:arrow_up:
target_module_vault_read	92.10% <92.10%> (?)
units	88.65% <ø> (+1.29%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
plugins/lookup/hashi_vault.py	82.43% <ø> (ø)
plugins/module_utils/_authenticator.py	100.00% <ø> (ø)
plugins/lookup/vault_read.py	92.30% <92.30%> (ø)
plugins/modules/vault_read.py	100.00% <100.00%> (ø)
...le_utils/authentication/test_auth_aws_iam_login.py	100.00% <0.00%> (ø)
plugins/module_utils/_auth_method_aws_iam_login.py	51.85% <0.00%> (+30.28%)	:arrow_up:

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 54b007a...a0de091. Read the comment docs.

[github-actions[bot]]

Docs Build 📝

Thank you for contribution!✨

This PR has been merged and the docs are now incorporated into main: https://community-hashi-vault-main.surge.sh

[briantist]

Went over the outstanding items with Felix in IRC, so I think we're good to go here!

[devon-mar]

Are modules able to read the auth config from ansible.cfg or env? I tried using this module without any auth config in the task, but it fails with No Vault Token specified or discovered.

---
- hosts: all
  gather_facts: false
  tasks:
  - community.hashi_vault.vault_read:
      path: "auth/token/lookup-self"
    delegate_to: localhost

[briantist]

Are modules able to read the auth config from ansible.cfg or env? I tried using this module without any auth config in the task, but it fails with No Vault Token specified or discovered.

Nope they aren't. That's also why we in the tests we use module_defaults. I'll also be adding a module defaults "action-group" in 1.5.0 to help with that, but it'll only work in new-ish versions of ansible core (I forget the requirements, will have to check on that):

• https://docs.ansible.com/ansible/latest/user_guide/playbooks_module_defaults.html#module-defaults-groups
• https://docs.ansible.com/ansible/devel/dev_guide/developing_collections_structure.html#meta-directory

I have been thinking about various options for providing the ability to use the ansible.cfg values for module defaults; it would be possible to do so if all of our modules used a custom action plugin for example, but I'm concerned that it's quite un-idiomatic in Ansible, so I'm shying away from that.

Another possibility I was considering was a lookup plugin whose purpose is only to return a set of module defaults (and it would be be able to infer those same values from controller-side env and INI and even ansible vars). It's still a little weird, and just an idea I'm playing with, but it would combine nicely with the action group, allowing something like this:

module_defaults:
  community.hashi_vault.vault_modules: "{{ lookup('community.hashi_vault.vault_ansible_settings', '') }}"

But this is something I'm going to consider very carefully and try to get community input on to avoid implementing what might become an anti-pattern.

For now, setting defaults will have to be done purposefully.

One exception re: env vars, is that env vars that are "late binding" will work in modules, if they are set on the target (which can also be done via the environment: keyword). This may need to be revisited later; deciding which env vars make sense on target vs. controller; may be a good discussion to move into #10 .

[devon-mar]

So there isn't any elegant way (now or future) to apply auth settings to all community.hashi_vault modules except using module_defaults across all my playbooks that use Vault?

I've also tried using ANSIBLE_HASHI_VAULT_AUTH_METHOD as well but it doesn't seem to take any effect.

[briantist]

So there isn't any elegant way (now or future) to apply auth settings to all community.hashi_vault modules except using module_defaults across all my playbooks that use Vault?

That's correct for now, for the future, we'll see! But the current state comes from Ansible itself and is somewhat of a necessary thing, considering that modules and controller-side plugins operate in different environments. So we can try to build some things to make that more seamless, but we need to be cautious about assumptions and look to provide options that will scale and fit in well with the Ansible ecosystem.

I've also tried using ANSIBLE_HASHI_VAULT_AUTH_METHOD as well but it doesn't seem to take any effect.

Right, those vars defined in env: in plugins won't be used for modules; we could make that work, but I would like to ensure we do that deliberately as it may or may not make sense in various contexts, and if we did, it would be reading the env var on the target not from the controller.

For now, you could explicitly opt-in to reading it from the controller when calling the module or setting module_defaults, by looking up the env var:

- community.hashi_vault.vault_read:
    auth_method: "{{ lookup('env', 'ANSIBLE_HASHI_VAULT_AUTH_METHOD') }}"

---

I definitely want to work toward better solutions like the lookup plugin I mentioned in the other comment, so I'm happy to hear your experience and scenarios in using this collection with Vault. I will open a dedicated issue for this so we can have discussion in one place.

[briantist]

@devon-mar

• 166