search

ansible-collections / community.hashi_vault

Ansible collection for managing and working with HashiCorp Vault.
https://docs.ansible.com/ansible/devel/collections/community/hashi_vault/index.html
GNU General Public License v3.0
80 stars 58 forks source link

Database Secrets Engine: Connection Management and Role Management from Ansible #425

Closed M4rt1nCh closed 3 months ago

M4rt1nCh commented 5 months ago
SUMMARY

Database Secrets Engine: Connection Management and Role Management from Ansible

ISSUE TYPE

We manage our infrastructure as code and run plenty (hundreds) of databases. Each should be managed from Hashicorp Vault, meaning:

1.) Create a connection to the database with initial credentials from ansible 2.) Enable root certificate rotation from ansible 3.) Automatically create dynamic / static roles for each connection following a certain naming scheme from ansible

Of course, as I can add / edit the existing configuration, it would be also awesome to be able to delete existing artifacts. From my point of view, nothing from the three steps can be achieved as of today using any of the existing hashi_vault modules.

COMPONENT NAME

Possible module names:

briantist commented 5 months ago

Hi @M4rt1nCh , thanks for opening this feature request! Is this something you're interested in contributing as well?

This collection is mostly a wrapper around hvac, so as long as functionality exists in hvac it should be possible to add to the collection without too much trouble, we just want to take care to ensure the interface is idiomatic to ansible as we do it.

We would probably name modules to align with their hvac counterparts, for the most part:

M4rt1nCh commented 5 months ago

Hi @briantist. Sure, let me see how far I get with the contribution part. I'll let you know whenever support is needed.

M4rt1nCh commented 5 months ago

Hey @briantist. Generally, a couple of modules should be doable. A couple of constraints:

Thanks for getting back to me :-)

briantist commented 5 months ago

setting up the environment locally for running integration tests is cumbersome - at least to me since I used to use molecule for integration testing rather than ansible-test integration.

Please take a look at the contributor guide for information on setting up your local test environment. Docker will be required for integration tests. I also recommend docker for running the unit and sanity tests, though technically you can run those with other methods supported by ansible-test.

It should be pretty easy to get set up to run all those tests locally, but if you run into issues or have questions, please let me know!

This collection has near 100% code coverage, and all new content for the last several years has 100% coverage, but I can also help with adding tests or get you started on how to do that.

re: naming: I think vault_database_list_connections sounds good. If I come up with some reason why it should be named differently I'll let you know but I think it makes sense. Let's not do a deep structure, it's not often used and I think some collections have rolled that decision back recently.

Thanks so much!

M4rt1nCh commented 5 months ago

@briantist I'm trying to create integration tests for the new modules. I've setup my local environment with the role included in the collection and then tried to execute one target: ansible-test integration --docker default --docker-network hashi_vault_default -v lookup_vault_kv2_get. This then fails:

ERROR: Command "/usr/bin/python3.6 /root/ansible/test/lib/ansible_test/_data/quiet_pip.py check --disable-pip-version-check" returned exit status 1.
>>> Standard Output
hvac 1.2.1 has requirement requests<3.0.0,>=2.27.1, but you have requests 2.23.0.

The container created (quay.io/ansible/default-ansible-container 2.9.0) doesn't seem to be orchestrated properly. Is there anything I am missing?

Thanks for your help. Next to that, feel free to provide feedback on the modules created in my fork :-)

briantist commented 5 months ago

What's the output of ansible --version? Ensure you're using a version of ansible-core that's supported by this collection, and you must also use a controller Python version that's supported, see: https://docs.ansible.com/ansible/devel/reference_appendices/release_and_maintenance.html#ansible-core-support-matrix

What is your host OS? If you're on Windows using WSL2 and Docker Desktop, make sure systemd support is enabled in WSL: https://learn.microsoft.com/en-us/windows/wsl/systemd

M4rt1nCh commented 5 months ago

Running Ubuntu 22.04 and python in a virtual environment managed by pyenv.

$ ansible --version
ansible [core 2.16.3]
  config file = /home/$USER/.ansible.cfg
  configured module search path = ['/home/$USER/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/$USER/.pyenv/versions/3.11.7/envs/hvac/lib/python3.11/site-packages/ansible
  ansible collection location = /home/$USER/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/$USER/.pyenv/versions/hvac/bin/ansible
  python version = 3.11.7 (main, Dec 18 2023, 09:37:20) [GCC 11.4.0] (/home/$USER/.pyenv/versions/3.11.7/envs/hvac/bin/python)
  jinja version = 3.1.3
  libyaml = True

So this is - from what I see - a supported setup.

briantist commented 5 months ago

Your original error showed it was using python3.6:

ERROR: Command "/usr/bin/python3.6...

Do you get the same error in that venv? Does the error still refer to the wrong python version?

M4rt1nCh commented 4 months ago

Not sure why, but apparently, the process pulled an outdated container, probably as the local python virtual environment was not configured properly. I was just able to execute the integration testing for one of the existing test cases locally and I'll now start to add integration tests for the modules the merge request will be adding.

briantist commented 4 months ago

Excellent! Glad to hear, do let me know if you run into any other issues.