search

ansible-collections / community.hashi_vault

Ansible collection for managing and working with HashiCorp Vault.
https://docs.ansible.com/ansible/devel/collections/community/hashi_vault/index.html
GNU General Public License v3.0
80 stars 59 forks source link

Unable to use vault_kv2_get lookup with python requests version > 2.27.1 #294

Closed gdadev closed 2 years ago

gdadev commented 2 years ago
SUMMARY

Unable to use vault_kv2_get lookup with python requests version > 2.27.1

ISSUE TYPE
COMPONENT NAME

community.hashi_vault.vault_kv2_get

ANSIBLE VERSION
ansible [core 2.13.2]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/gdadev/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.10/site-packages/ansible
  ansible collection location = /home/gdadev/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.10.5 (main, Aug  1 2022, 07:53:20) [GCC 12.1.0]
  jinja version = 3.1.2
  libyaml = True
COLLECTION VERSION
# /usr/lib/python3.10/site-packages/ansible_collections
Collection            Version
--------------------- -------
community.hashi_vault 3.1.0
CONFIGURATION
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = False
OS / ENVIRONMENT

Host:

LSB Version:    n/a
Distributor ID: EndeavourOS
Description:    EndeavourOS Linux
Release:        rolling
Codename:       rolling

Target:

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.3 LTS
Release:        20.04
Codename:       focal
STEPS TO REPRODUCE
# plugin settings
ansible_hashi_vault_url: 'https://vaulturl:8200'
ansible_hashi_vault_engine_mount_point: 'mountpoint'
ansible_hashi_vault_token: tokenforvault
ansible_hashi_vault_token_validate: no

# lookup some variable from vault
somevar: "{{ lookup('community.hashi_vault.vault_kv2_get', 'path/to/secret').secret.somesecret }}"
ACTUAL RESULTS
The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/ansible/template/__init__.py", line 984, in _lookup
    ran = instance.run(loop_terms, variables=self._available_variables, **kwargs)
  File "/usr/lib/python3.10/site-packages/ansible_collections/community/hashi_vault/plugins/lookup/vault_kv2_get.py", line 228, in run
    raw = client.secrets.kv.v2.read_secret_version(path=term, version=version, mount_point=engine_mount_point)
  File "/home/gdadev/.local/lib/python3.10/site-packages/hvac/api/secrets_engines/kv_v2.py", line 98, in read_secret_version
    return self._adapter.get(
  File "/home/gdadev/.local/lib/python3.10/site-packages/hvac/adapters.py", line 113, in get
    return self.request("get", url, **kwargs)
  File "/home/gdadev/.local/lib/python3.10/site-packages/hvac/adapters.py", line 364, in request
    response = super(JSONAdapter, self).request(*args, **kwargs)
  File "/home/gdadev/.local/lib/python3.10/site-packages/hvac/adapters.py", line 313, in request
    response = self.session.request(
  File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 573, in request
    prep = self.prepare_request(req)
  File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 484, in prepare_request
    p.prepare(
  File "/usr/lib/python3.10/site-packages/requests/models.py", line 369, in prepare
    self.prepare_headers(headers)
  File "/usr/lib/python3.10/site-packages/requests/models.py", line 491, in prepare_headers
    check_header_validity(header)
  File "/usr/lib/python3.10/site-packages/requests/utils.py", line 1037, in check_header_validity
    raise InvalidHeader(
requests.exceptions.InvalidHeader: Header part ('<sometoken>') from {'X-Vault-Token': '<sometoken>'} must be of type str or bytes, not <class 'ansible.parsing.yaml.objects.AnsibleUnicode'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/ansible/template/vars.py", line 98, in __getitem__
    value = self._templar.template(variable)
  File "/usr/lib/python3.10/site-packages/ansible/template/__init__.py", line 871, in template
    result = self.do_template(
  File "/usr/lib/python3.10/site-packages/ansible/template/__init__.py", line 1118, in do_template
    res = self.environment.concat(rf)
  File "/usr/lib/python3.10/site-packages/ansible/template/native_helpers.py", line 44, in ansible_eval_concat
    head = list(islice(nodes, 2))
  File "<template>", line 12, in root
  File "/usr/lib/python3.10/site-packages/jinja2/runtime.py", line 298, in call
    return __obj(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/ansible/template/__init__.py", line 1010, in _lookup
    raise AnsibleError(to_native(msg), orig_exc=e)
ansible.errors.AnsibleError: An unhandled exception occurred while running the lookup plugin 'community.hashi_vault.vault_kv2_get'. Error was a <class 'requests.exceptions.InvalidHeader'>, original message: Header part ('<sometoken>') from {'X-Vault-Token': '<sometoken>'} must be of type str or bytes, not <class 'ansible.parsing.yaml.objects.AnsibleUnicode'>. Header part ('<sometoken>') from {'X-Vault-Token': '<sometoken>'} must be of type str or bytes, not <class 'ansible.parsing.yaml.objects.AnsibleUnicode'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/ansible/plugins/action/template.py", line 138, in run
    resultant = templar.do_template(template_data, preserve_trailing_newlines=True, escape_backslashes=False)
  File "/usr/lib/python3.10/site-packages/ansible/template/__init__.py", line 1116, in do_template
    res = ansible_concat(rf)
  File "/usr/lib/python3.10/site-packages/ansible/template/native_helpers.py", line 88, in ansible_concat
    return ''.join([to_text(v) for v in nodes])
  File "/usr/lib/python3.10/site-packages/ansible/template/native_helpers.py", line 88, in <listcomp>
    return ''.join([to_text(v) for v in nodes])
  File "<template>", line 11, in root
  File "/usr/lib/python3.10/site-packages/ansible/template/__init__.py", line 378, in resolve_or_missing
    val = super(AnsibleContext, self).resolve_or_missing(key)
  File "/usr/lib/python3.10/site-packages/jinja2/runtime.py", line 241, in resolve_or_missing
    return self.parent[key]
  File "/usr/lib/python3.10/site-packages/ansible/template/vars.py", line 103, in __getitem__
    raise AnsibleError("An unhandled exception occurred while templating '%s'. "
ansible.errors.AnsibleError: An unhandled exception occurred while templating '{{ lookup('community.hashi_vault.vault_kv2_get', 'path/to/secret').secret.somesecret }}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: An unhandled exception occurred while running the lookup plugin 'community.hashi_vault.vault_kv2_get'. Error was a <class 'requests.exceptions.InvalidHeader'>, original message: Header part ('<sometoken>') from {'X-Vault-Token': '<sometoken>'} must be of type str or bytes, not <class 'ansible.parsing.yaml.objects.AnsibleUnicode'>. Header part ('<sometoken>') from {'X-Vault-Token': '<sometoken>'} must be of type str or bytes, not <class 'ansible.parsing.yaml.objects.AnsibleUnicode'>
fatal: [target.url]: FAILED! => {
    "changed": false,
    "msg": "AnsibleError: An unhandled exception occurred while templating '{{ lookup('community.hashi_vault.vault_kv2_get', 'path/to/secret').secret.somesecret }}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: An unhandled exception occurred while running the lookup plugin 'community.hashi_vault.vault_kv2_get'. Error was a <class 'requests.exceptions.InvalidHeader'>, original message: Header part ('<sometoken>') from {'X-Vault-Token': '<sometoken>'} must be of type str or bytes, not <class 'ansible.parsing.yaml.objects.AnsibleUnicode'>. Header part ('<sometoken>') from {'X-Vault-Token': '<sometoken>'} must be of type str or bytes, not <class 'ansible.parsing.yaml.objects.AnsibleUnicode'>"
}
briantist commented 2 years ago

Hi @gdadev , this was fixed by #291 but hasn't been released yet, I expect to get a release out over the weekend.

If you'd like to try it now, you can install from the git repo:

ansible-galaxy collection install git+https://github.com/ansible-collections/community.hashi_vault.git,main
briantist commented 2 years ago

@gdadev tagging you because I edited the above (and github doesn't notify on edits)

briantist commented 2 years ago

@gdadev heads-up that version 3.2.0 has been released with the fix for this issue :)

tobiicerb commented 1 year ago

@briantist I have the exact same error but for X-Vault-Namespace. I am using the newest version. It happens when I use the variable ansible_hashi_vault_namespace. It doesn't seem to happen if I prefix the engine_mount_path.

briantist commented 1 year ago

@tobiicerb thank you for this report! would you mind opening a new issue so I can track this separately?

tobiicerb commented 1 year ago

@tobiicerb thank you for this report! would you mind opening a new issue so I can track this separately?

@briantist I would have, but I see now you have an issue that suits. I will follow the other issue. Thank you!