add k8s auth to lookup hashi_vault - operator

[chris93111]

SUMMARY

Add authentification k8s in lookup, for ansible operator (operator-sdk) use service account jwt token in login

ISSUE TYPE

• Feature Pull Request

COMPONENT NAME

lookup hashi_vault

- name: vault test jwt k8s
  hosts: localhost
  connection: localhost
  tasks:
    - name: try fetch secret with jwt SA
      debug:
        msg: "{{ lookup('community.hashi_vault.hashi_vault', 'secret=ansible/data/tower/credential/user1 auth_method=k8s mount_point=kubernetes-cluster1 role_id=awx url=https://vault:443')}}"

[github-actions[bot]]

Docs Build 📝

Thank you for contribution!✨

The docs for this PR have been published here: https://ansible-collections.github.io/community.hashi_vault/pr/220

You can compare to the docs for the main branch here: https://ansible-collections.github.io/community.hashi_vault/branch/main

The docsite for this PR is also available for download as an artifact from this run: https://github.com/ansible-collections/community.hashi_vault/actions/runs/2809896577

File changes:

• M collections/community/hashi_vault/hashi_vault_lookup.html
• M collections/community/hashi_vault/vault_kv1_get_lookup.html
• M collections/community/hashi_vault/vault_kv1_get_module.html
• M collections/community/hashi_vault/vault_kv2_get_lookup.html
• M collections/community/hashi_vault/vault_kv2_get_module.html
• M collections/community/hashi_vault/vault_login_lookup.html
• M collections/community/hashi_vault/vault_login_module.html
• M collections/community/hashi_vault/vault_pki_generate_certificate_module.html
• M collections/community/hashi_vault/vault_read_lookup.html
• M collections/community/hashi_vault/vault_read_module.html
• M collections/community/hashi_vault/vault_token_create_lookup.html
• M collections/community/hashi_vault/vault_token_create_module.html
• M collections/community/hashi_vault/vault_write_lookup.html
• M collections/community/hashi_vault/vault_write_module.html

Click to see the diff comparison.

**NOTE:** only file modifications are shown here. New and deleted files are excluded. See the file list and check the published docs to see those files. ```diff diff --git a/home/runner/work/community.hashi_vault/community.hashi_vault/docsbuild/base/collections/community/hashi_vault/hashi_vault_lookup.html b/home/runner/work/community.hashi_vault/community.hashi_vault/docsbuild/head/collections/community/hashi_vault/hashi_vault_lookup.html index fd75dce..3790c9d 100644 --- a/home/runner/work/community.hashi_vault/community.hashi_vault/docsbuild/base/collections/community/hashi_vault/hashi_vault_lookup.html +++ b/home/runner/work/community.hashi_vault/community.hashi_vault/docsbuild/head/collections/community/hashi_vault/hashi_vault_lookup.html @@ -185,6 +185,7 @@ To check whether it is installed, run aws_iam

jwt

cert

+

kubernetes

none

Configuration:

@@ -355,6 +356,39 @@ To check whether it is installed, run

+

kubernetes_token

+

string

+

added in 2.5.0 of community.hashi_vault

+

+

The Kubernetes Token (JWT) to use for Kubernetes authentication to Vault.

+

Configuration:

+

+

• Environment variable: ANSIBLE_HASHI_VAULT_KUBERNETES_TOKEN

+

• Variable: ansible_hashi_vault_kubernetes_token

+

+

+ +

+

kubernetes_token_path

+

string

+

added in 2.5.0 of community.hashi_vault

+

+

If no kubernetes_token is specified, will try to read the token from this path.

+

Default: “/var/run/secrets/kubernetes.io/serviceaccount/token”

+

Configuration:

+

+

• INI entry:

  +

  [hashi_vault_collection]
  +kubernetes_token_path = /var/run/secrets/kubernetes.io/serviceaccount/token
  +

  +
  +
+

• Environment variable: ANSIBLE_HASHI_VAULT_KUBERNETES_TOKEN_PATH

+

• Variable: ansible_hashi_vault_kubernetes_token_path

+

+

+ +

mount_point

string

diff --git a/home/runner/work/community.hashi_vault/community.hashi_vault/docsbuild/base/collections/community/hashi_vault/vault_kv1_get_lookup.html b/home/runner/work/community.hashi_vault/community.hashi_vault/docsbuild/head/collections/community/hashi_vault/vault_kv1_get_lookup.html index 34d5aa9..6def43f 100644 --- a/home/runner/work/community.hashi_vault/community.hashi_vault/docsbuild/base/collections/community/hashi_vault/vault_kv1_get_lookup.html +++ b/home/runner/work/community.hashi_vault/community.hashi_vault/docsbuild/head/collections/community/hashi_vault/vault_kv1_get_lookup.html @@ -207,6 +207,7 @@ To check whether it is installed, run aws_iam

jwt

cert

+

kubernetes

none

Configuration:

@@ -389,6 +390,39 @@ To check whether it is installed, run

+

kubernetes_token

+

string

+

added in 2.5.0 of community.hashi_vault

+

+

The Kubernetes Token (JWT) to use for Kubernetes authentication to Vault.

+

Configuration:

+

+

• Environment variable: ANSIBLE_HASHI_VAULT_KUBERNETES_TOKEN

+

• Variable: ansible_hashi_vault_kubernetes_token

+

+

+ +

+

kubernetes_token_path

+

string

+

added in 2.5.0 of community.hashi_vault

+

+

If no kubernetes_token is specified, will try to read the token from this path.

+

Default: “/var/run/secrets/kubernetes.io/serviceaccount/token”

+

Configuration:

+

+

• INI entry:

  +

  [hashi_vault_collection]
  +kubernetes_token_path = /var/run/secrets/kubernetes.io/serviceaccount/token
  +

  +
  +
+

• Environment variable: ANSIBLE_HASHI_VAULT_KUBERNETES_TOKEN_PATH

+

• Variable: ansible_hashi_vault_kubernetes_token_path

+

+

+ +

mount_point

string

diff --git a/home/runner/work/community.hashi_vault/community.hashi_vault/docsbuild/base/collections/community/hashi_vault/vault_kv1_get_module.html b/home/runner/work/community.hashi_vault/community.hashi_vault/docsbuild/head/collections/community/hashi_vault/vault_kv1_get_module.html index 796a7b1..fe14ed1 100644 --- a/home/runner/work/community.hashi_vault/community.hashi_vault/docsbuild/base/collections/community/hashi_vault/vault_kv1_get_module.html +++ b/home/runner/work/community.hashi_vault/community.hashi_vault/docsbuild/head/collections/community/hashi_vault/vault_kv1_get_module.html @@ -186,6 +186,7 @@ To check whether it is installed, run aws_iam

jwt

cert

+

kubernetes

none

[chris93111]

@briantist can you please relaunch the CI and tell me if correction is ok ? they are error Node_error on last try

I can write test, but how i can mount minikube in the ci ? Tests can't work without

[briantist]

@briantist can you please relaunch the CI and tell me if correction is ok ? they are error Node_error on last try

Sure I've launched the CI for the latest commit but canceled the queued runs for the other commits. New contributors need approval to run on every push. I strongly recommend getting set up to run the tests locally. All the tests we run in CI are able to be run on a local machine, and there's instructions in the contributor guide, it will really make turnaround time a lot faster for you. If you have any trouble getting set up, let me know.

I don't have much left in me this weekend to take a deeper look at this yet, but I will as soon as I get more free time.

I can write test, but how i can mount minikube in the ci ? Tests can't work without

Forgive me for being unfamiliar with minikube but from the looks of it, it's something we could possibly spin up as part of the integration tests. We already have Vault, tinyproxy, and MMock running, each in their own containers.

From what little I cold gather, minikube is not really designed to run in a container. But we could launch it separately, as long as we ensure it works in both local and CI environments.

Looking around a bit, it seems like k3d (https://docs.ansible.com/ansible/devel/collections/community/hashi_vault/docsite/contributor_guide.html#mocking-integrations) might be better for this purpose? It looks to be designed to run in a container itself, so this would be even easier to run maybe.

To add a new piece of software like this, you'll want to look at the localenv developer guide, and at the current docker-based localenv that we use to spin up the current set of containers.

---

But there is an easier option too, we can mock the integration with MMock. We already do this for other things that weren't so easy to set up (AWS auth and LDAP auth). You can see the mmock configs here: https://github.com/ansible-collections/community.hashi_vault/tree/main/tests/integration/targets/setup_localenv_docker/templates/mmock

And if you run this locally, you'll be able to connect to MMock's console port in your web browser and get a real-time view of requests which can help tune it.

I can help with that if you have the server response from running the real command against a Vault server (with everything sensitive redacted/replaced).

The basic idea is that the integration tests point at MMock, and MMock is set up to proxy those requests to the test Vault server, unless there's a config for the request that matches a given path (or other criteria), and for those requests, we can return a mock response.

---

Integration tests aside, you can start writing unit tests right away, which can cover a lot of corner cases more easily than integration tests can, and they will not/can not talk to any external services.

Grabbing a real response as described above is useful for unit tests as well as MMock, you can see the other ones we have here: https://github.com/ansible-collections/community.hashi_vault/tree/main/tests/unit/fixtures

And we use these as mock returns in the unit tests as well.

---

Hopefully there's plenty here that you can get started on, even if I can't put a lot of deep time into this just yet, I will try my best to answer questions and keep you moving along!

[chris93111]

Hi @briantist can you please review this before i try to work with CI

[briantist]

A change was made in core a few hours ago that will cause sanity test failures (unrelated to your PR). I have a PR up to address those changes:

• https://github.com/ansible-collections/community.hashi_vault/pull/226

With that merged, please rebase to ensure sanity passes.

[chris93111]

@briantist i thinks it's ok ?

[briantist]

Looks like you've got one sanity nit in there, I recommend setting your editor to trim trailing whitespace, insert final newline (single), and trim final newlines (these are roughly the names of the settings in vscode, not sure what they are called in other editors).

Also again, running tests locally will really help you find these things quickly... and will be invaluable when you start adding the integration and unit tests.

If there's any way I can help you get your local setup going, let me know!

[codecov[bot]]

Codecov Report

Merging #220 (d5f070b) into main (f036ced) will decrease coverage by 1.64%. The diff coverage is 41.93%.

:exclamation: Current head d5f070b differs from pull request most recent head 39b6f09. Consider uploading reports for the commit 39b6f09 to get more accurate results

@@            Coverage Diff             @@
##             main     #220      +/-   ##
==========================================
- Coverage   98.46%   96.82%   -1.65%     
==========================================
  Files          71       57      -14     
  Lines        3458     2549     -909     
  Branches      301      189     -112     
==========================================
- Hits         3405     2468     -937     
- Misses         44       67      +23     
- Partials        9       14       +5     

Flag	Coverage Δ
env_docker-default	96.82% <41.93%> (-1.65%)	:arrow_down:
integration	79.44% <41.93%> (-2.18%)	:arrow_down:
sanity	38.61% <38.70%> (-0.67%)	:arrow_down:
target_ansible-doc	100.00% <ø> (ø)
target_auth_approle	89.47% <ø> (ø)
target_auth_aws_iam	50.00% <ø> (ø)
target_auth_cert	86.36% <ø> (ø)
target_auth_jwt	91.30% <ø> (ø)
target_auth_ldap	89.47% <ø> (ø)
target_auth_none	100.00% <ø> (ø)
target_auth_token	71.42% <ø> (-1.65%)	:arrow_down:
target_auth_userpass	85.71% <ø> (ø)
target_connection_options	74.76% <ø> (ø)
target_controller	74.74% <41.93%> (-8.94%)	:arrow_down:
target_filter_vault_login_token	77.77% <ø> (ø)
target_import	38.61% <38.70%> (-0.67%)	:arrow_down:
target_lookup_hashi_vault	81.33% <ø> (ø)
target_lookup_vault_ansible_settings	?
target_lookup_vault_kv1_get	?
target_lookup_vault_kv2_get	?
target_lookup_vault_login	100.00% <ø> (+11.42%)	:arrow_up:
target_lookup_vault_read	90.00% <ø> (ø)
target_lookup_vault_token_create	82.97% <ø> (+4.79%)	:arrow_up:
target_lookup_vault_write	58.34% <41.93%> (-0.92%)	:arrow_down:
target_module_utils	94.71% <41.93%> (-2.00%)	:arrow_down:
target_module_vault_kv1_get	?
target_module_vault_kv2_get	?
target_module_vault_login	93.93% <ø> (+10.60%)	:arrow_up:
target_module_vault_pki_generate_certificate	78.26% <ø> (ø)
target_module_vault_read	92.10% <ø> (+6.73%)	:arrow_up:
target_module_vault_token_create	90.00% <ø> (ø)
target_module_vault_write	56.70% <41.93%> (-1.55%)	:arrow_down:
target_modules	69.30% <41.93%> (-9.36%)	:arrow_down:
units	92.85% <41.93%> (-2.75%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
plugins/doc_fragments/auth.py	100.00% <ø> (ø)
plugins/module_utils/_auth_method_k8s.py	40.00% <40.00%> (ø)
plugins/module_utils/_authenticator.py	100.00% <100.00%> (ø)
plugins/lookup/vault_read.py	95.00% <0.00%> (-5.00%)	:arrow_down:
.../plugin_utils/base/test_hashi_vault_lookup_base.py	96.66% <0.00%> (-3.34%)	:arrow_down:
plugins/modules/vault_login.py	96.96% <0.00%> (-3.04%)	:arrow_down:
...s/module_utils/authentication/test_auth_aws_iam.py	98.29% <0.00%> (-1.71%)	:arrow_down:
plugins/lookup/vault_token_create.py	95.74% <0.00%> (-0.62%)	:arrow_down:
plugins/module_utils/_auth_method_token.py	97.95% <0.00%> (-0.12%)	:arrow_down:
... and 65 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[briantist]

Hi @chris93111 , are you still interested in pursuing this? I wanted to let you know that version 3.0.0 of the collection will be released in the coming weeks, with some breaking changes. Support for Ansible 2.9 and 2.10 will be dropped and won't be tested against anymore either.

If you'd like to get this into a release that still supports 2.9/2.10, it would be best to get this moving along for the best chance of that. I will need to get 3.0.0 out in time for the deadline for inclusion into the Ansible 6 community package, so I don't really have any leeway when to release that.

If you're not concerned about that then no worries and no rush, just wanted to give you a heads-up.

[chris93111]

hello @briantist Yes of course! I need to look at the important changes brought by the non support of 2.9 and 2.10 I also have to look at the CI

[briantist]

hello @briantist Yes of course! I need to look at the important changes brought by the non support of 2.9 and 2.10 I also have to look at the CI

Great! There's nothing very important you need to know about removal of 2.9/2.10 support for your implementation, I only mention it in case you need this collection with one of those versions, or you would like to get kubernetes auth released in a version of this collection that users of those versions can trust for use.

I don't actually expect the first release of 3.0.0 to be broken in 2.9 or 2.10, but since we will stop testing against those versions, any new change in 3.x.x may break compatibility, so I expect anyone using 2.9 or 2.10 to pin to community.hashi_vault < 3.0.0.

---

Getting the unit tests and especially integration tests set up for this is absolutely going to be the most time consuming part of getting this ready to be merged. I believe it is entirely possible to do, but the timeline will depend on mostly on how much time you can devote to it, because I am very overloaded right now.

I can be more effective now by helping your efforts and steering you in the right direction, rather than trying to implement myself, I think.

In my opinion, the fastest way to get this to a point where we can merge, is to use the MMock method in integration tests. That could give us reasonable confidence, and will not require setting up kubernetes. We can then look up at real kubernetes of some kind as a new PR.

[briantist]

@chris93111 I'm not sure what happened with https://github.com/ansible-collections/community.hashi_vault/pull/220/commits/d49061d98741c7d03453797e17285bf51e2321b5 but the test results don't look right; it seems like perhaps some changes from main are missing.