Error in azure_rm_keyvaultsecret

[rdtechie]

When trying to create a Azure Key Vault secret with the latest preview module, I'm receiving this error:

fatal: [localhost]: FAILED! => {
    "changed": false, 
    "module_stderr": "No handlers could be found for logger \"msrest.pipeline.requests\"\nTraceback (most recent call last):\n  File \"<stdin>\", line 114, in <module>\n  File \"<stdin>\", line 106, in _ansiballz_main\n  File \"<stdin>\", line 49, in invoke_module\n
 File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_ASnlKS/__main__.py\", line 214, in <module>\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_ASnlKS/__main__.py\", line 210, in main\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_ASnlKS/__main__.py\", line 124, in __init__\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_ASnlKS/ansible_azure_rm_keyvaultsecret_payload.zip/ansible/module_utils/azure_rm_common.py\", line 315, in __init__\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_ASnlKS/__main__.py\", line 156, in exec_module\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_ASnlKS/__main__.py\", line 191, in get_secret\n  File \"/opt/ansible/venv/ansible_dev_azure/lib/python2.7/site-packages/azure/keyvault/key_vault_client.py\", line 1795, in get_secret\n    response = self._client.send(request, header_parameters, **operation_config)\n  File \"/opt/ansible/venv/ansible_dev_azure/lib/python2.7/site-packages/msrest/service_client.py\", line 219, in send\n    pipeline_response = self.config.pipeline.run(request, **kwargs)\n  File \"/opt/ansible/venv/ansible_dev_azure/lib/python2.7/site-packages/msrest/pipeline/__init__.py\", line 203, in run\n    return first_node.send(pipeline_request, **kwargs)  # type: ignore\n  File \"/opt/ansible/venv/ansible_dev_azure/lib/python2.7/site-packages/msrest/pipeline/__init__.py\", line 156, in send\n    response = self.next.send(request, **kwargs)\n  File \"/opt/ansible/venv/ansible_dev_azure/lib/python2.7/site-packages/msrest/pipeline/requests.py\", line 72, in send\n    return self.next.send(request, **kwargs)\n  File \"/opt/ansible/venv/ansible_dev_azure/lib/python2.7/site-packages/msrest/pipeline/requests.py\", line 137, in send\n    return self.next.send(request, **kwargs)\n  File \"/opt/ansible/venv/ansible_dev_azure/lib/python2.7/site-packages/msrest/pipeline/__init__.py\", line 156, in send\n    response = self.next.send(request, **kwargs)\n  File \"/opt/ansible/venv/ansible_dev_azure/lib/python2.7/site-packages/msrest/pipeline/requests.py\", line 193, in send\n    self.driver.send(request.http_request, **kwargs)\n  File \"/opt/ansible/venv/ansible_dev_azure/lib/python2.7/site-packages/msrest/universal_http/requests.py\", line 328, in send\n    return super(RequestsHTTPSender, self).send(request, **requests_kwargs)\n  File \"/opt/ansible/venv/ansible_dev_azure/lib/python2.7/site-packages/msrest/universal_http/requests.py\", line 137, in send\n    **kwargs)\n  File \"/opt/ansible/venv/ansible_dev_azure/lib/python2.7/site-packages/requests/sessions.py\", line 533, in request\n    resp = self.send(prep, **send_kwargs)\n  File \"/opt/ansible/venv/ansible_dev_azure/lib/python2.7/site-packages/requests/sessions.py\", line 653, in send\n    r = dispatch_hook('response', hooks, r, **kwargs)\n  File \"/opt/ansible/venv/ansible_dev_azure/lib/python2.7/site-packages/requests/hooks.py\", line 31, in dispatch_hook\n    _hook_data = hook(hook_data, **kwargs)\n  File \"/opt/ansible/venv/ansible_dev_azure/lib/python2.7/site-packages/azure/keyvault/custom/key_vault_authentication.py\", line 146, in _handle_401\n    security = self._get_message_security(prep, challenge)\n  File \"/opt/ansible/venv/ansible_dev_azure/lib/python2.7/site-packages/azure/keyvault/custom/key_vault_authentication.py\", line 172, in _get_message_security\n    scheme))\n  File \"/opt/ansible/venv/ansible_dev_azure/lib/python2.7/site-packages/azure/keyvault/custom/key_vault_authentication.py\", line 61, in _auth_callback_compat\n    if len(inspect.getargspec(self._user_callback).args) == 3 \\\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_ASnlKS/__main__.py\", line 133, in auth_callback\nKeyError: 'client_id'\n",
    "module_stdout": "", 
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", 
    "rc": 1
}

I'm using a MSI as authentication method, and this MSI has permissions to the key vault. It's added with the correct permissions to the Access Policy on the Key Vault.

This succeeds from the same machine: az keyvault secret set --subscription <fakenumber> --vault-name vault01 --name secret01 --value secret01

pip freeze:

adal==1.2.1
ansible==2.8.0.dev0
ansible-lint==3.4.23
anyconfig==0.9.7
applicationinsights==0.11.7
argcomplete==1.9.4
arrow==0.13.1
asn1crypto==0.24.0
atomicwrites==1.3.0
attrs==18.2.0
azure-cli-core==2.0.35
azure-cli-nspkg==3.0.2
azure-common==1.1.11
azure-graphrbac==0.40.0
azure-keyvault==1.0.0a1
azure-mgmt-batch==4.1.0
azure-mgmt-cdn==3.0.0
azure-mgmt-compute==4.4.0
azure-mgmt-containerinstance==0.4.0
azure-mgmt-containerregistry==2.0.0
azure-mgmt-containerservice==4.2.2
azure-mgmt-cosmosdb==0.5.1
azure-mgmt-devtestlabs==2.2.0
azure-mgmt-dns==2.1.0
azure-mgmt-hdinsight==0.1.0
azure-mgmt-keyvault==0.40.0
azure-mgmt-marketplaceordering==0.1.0
azure-mgmt-monitor==0.5.2
azure-mgmt-network==2.3.0
azure-mgmt-nspkg==2.0.0
azure-mgmt-rdbms==1.4.1
azure-mgmt-redis==5.0.0
azure-mgmt-resource==1.2.2
azure-mgmt-sql==0.10.0
azure-mgmt-storage==3.1.0
azure-mgmt-trafficmanager==0.50.0
azure-mgmt-web==0.32.0
azure-nspkg==2.0.0
azure-storage==0.35.1
backports.functools-lru-cache==1.5
bcrypt==3.1.6
binaryornot==0.4.4
Cerberus==1.2
certifi==2018.11.29
cffi==1.12.1
chardet==3.0.4
click==6.7
click-completion==0.3.1
colorama==0.3.9
configparser==3.7.1
cookiecutter==1.6.0
cryptography==2.5
enum34==1.1.6
fasteners==0.14.1
flake8==3.5.0
funcsigs==1.0.2
future==0.17.1
git-url-parse==1.2.0
humanfriendly==4.17
idna==2.8
ipaddress==1.0.22
isodate==0.6.0
Jinja2==2.10
jinja2-time==0.2.0
jmespath==0.9.3
knack==0.3.3
MarkupSafe==1.1.0
mccabe==0.6.1
molecule==2.19.0
monotonic==1.5
more-itertools==5.0.0
msrest==0.6.1
msrestazure==0.5.0
ntlm-auth==1.2.0
oauthlib==3.0.1
packaging==19.0
paramiko==2.4.2
pathlib2==2.3.3
pathspec==0.5.9
pbr==4.1.0
pexpect==4.6.0
pluggy==0.8.1
poyo==0.4.2
psutil==5.4.6
ptyprocess==0.6.0
py==1.7.0
pyasn1==0.4.5
pycodestyle==2.3.1
pycparser==2.19
pyflakes==1.6.0
Pygments==2.3.1
PyJWT==1.7.1
PyNaCl==1.3.0
pyOpenSSL==19.0.0
pyparsing==2.3.1
pytest==4.3.0
python-dateutil==2.8.0
python-gilt==1.2.1
pywinrm==0.3.0
PyYAML==3.13
requests==2.21.0
requests-ntlm==1.1.0
requests-oauthlib==1.2.0
scandir==1.9.0
sh==1.12.14
six==1.11.0
tabulate==0.8.2
testinfra==1.16.0
tree-format==0.1.2
typing==3.6.6
urllib3==1.24.1
whichcraft==0.5.2
xmltodict==0.11.0
yamllint==1.11.1

[Fred-sun]

@zikalino Could you help take a look this issue when you're available? Thanks!

[yungezz]

@rdtechie thanks for reporting the issue, we're looking at it.

[yungezz]

Hi could you pls try fix in #215 , resolving your issue?

[rdtechie]

Hi @yungezz that resolves the issue. Can you merge it into master?

[yungezz]

fix was merged into master, close the issue.

[nicolas-marcq]

Hi, I'm still facing the issue. Ansible 2.8.2.

Task:

- name: "Create secret: postgres password"
  azure_rm_keyvaultsecret:
    secret_name: "pg-admin-password"
    secret_value: "my_password"
    keyvault_uri: "{{ vault_uri }}"
  tags: ["kv"]

Full log:

task path: /home/nico/Documents/ifrs17-deployer/ansible/roles/infrastructure/tasks/key-vault.yml:47
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: nico
<127.0.0.1> EXEC /bin/sh -c 'echo ~nico && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/nico/.ansible/tmp/ansible-tmp-1562316228.5673969-202423648538091 `" && echo ansible-tmp-1562316228.5673969-202423648538091="` echo /home/nico/.ansible/tmp/ansible-tmp-1562316228.5673969-202423648538091 `" ) && sleep 0'
Using module file /home/nico/Documents/ifrs17-deployer/ansible/library/azure_rm_keyvaultsecret.py
<127.0.0.1> PUT /home/nico/.ansible/tmp/ansible-local-29615w2d24ndr/tmpz8_d3vum TO /home/nico/.ansible/tmp/ansible-tmp-1562316228.5673969-202423648538091/AnsiballZ_azure_rm_keyvaultsecret.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/nico/.ansible/tmp/ansible-tmp-1562316228.5673969-202423648538091/ /home/nico/.ansible/tmp/ansible-tmp-1562316228.5673969-202423648538091/AnsiballZ_azure_rm_keyvaultsecret.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python3 /home/nico/.ansible/tmp/ansible-tmp-1562316228.5673969-202423648538091/AnsiballZ_azure_rm_keyvaultsecret.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/nico/.ansible/tmp/ansible-tmp-1562316228.5673969-202423648538091/ > /dev/null 2>&1 && sleep 0'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "MSI: wait: 0.1s and retry: 1\nMSI: wait: 0.1s and retry: 2\nMSI: wait: 0.7s and retry: 3\nMSI: wait: 0.7s and retry: 4\nMSI: wait: 3.1s and retry: 5\nMSI: wait: 0.7s and retry: 6\nMSI: wait: 3.1s and retry: 7\nMSI: wait: 0.3s and retry: 8\nMSI: wait: 0.3s and retry: 9\nMSI: wait: 0.7s and retry: 10\nMSI: wait: 204.7s and retry: 11\nMSI: wait: 1.5s and retry: 12\nYour credentials class does not support session injection. Performance will not be at the maximum.\nTraceback (most recent call last):\n  File \"/home/nico/.ansible/tmp/ansible-tmp-1562316228.5673969-202423648538091/AnsiballZ_azure_rm_keyvaultsecret.py\", line 114, in <module>\n    _ansiballz_main()\n  File \"/home/nico/.ansible/tmp/ansible-tmp-1562316228.5673969-202423648538091/AnsiballZ_azure_rm_keyvaultsecret.py\", line 106, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/nico/.ansible/tmp/ansible-tmp-1562316228.5673969-202423648538091/AnsiballZ_azure_rm_keyvaultsecret.py\", line 49, in invoke_module\n    imp.load_module('__main__', mod, module, MOD_DESC)\n  File \"/usr/lib/python3.6/imp.py\", line 235, in load_module\n    return load_source(name, filename, file)\n  File \"/usr/lib/python3.6/imp.py\", line 170, in load_source\n    module = _exec(spec, sys.modules[name])\n  File \"<frozen importlib._bootstrap>\", line 618, in _exec\n  File \"<frozen importlib._bootstrap_external>\", line 678, in exec_module\n  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_9kvx9sj7/__main__.py\", line 228, in <module>\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_9kvx9sj7/__main__.py\", line 224, in main\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_9kvx9sj7/__main__.py\", line 127, in __init__\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_9kvx9sj7/ansible_azure_rm_keyvaultsecret_payload.zip/ansible/module_utils/azure_rm_common.py\", line 325, in __init__\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_9kvx9sj7/__main__.py\", line 141, in exec_module\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_9kvx9sj7/__main__.py\", line 205, in get_secret\n  File \"/usr/local/lib/python3.6/dist-packages/azure/keyvault/key_vault_client.py\", line 1795, in get_secret\n    response = self._client.send(request, header_parameters, **operation_config)\n  File \"/usr/local/lib/python3.6/dist-packages/msrest/service_client.py\", line 219, in send\n    pipeline_response = self.config.pipeline.run(request, **kwargs)\n  File \"/usr/local/lib/python3.6/dist-packages/msrest/pipeline/__init__.py\", line 203, in run\n    return first_node.send(pipeline_request, **kwargs)  # type: ignore\n  File \"/usr/local/lib/python3.6/dist-packages/msrest/pipeline/__init__.py\", line 156, in send\n    response = self.next.send(request, **kwargs)\n  File \"/usr/local/lib/python3.6/dist-packages/msrest/pipeline/requests.py\", line 72, in send\n    return self.next.send(request, **kwargs)\n  File \"/usr/local/lib/python3.6/dist-packages/msrest/pipeline/requests.py\", line 137, in send\n    return self.next.send(request, **kwargs)\n  File \"/usr/local/lib/python3.6/dist-packages/msrest/pipeline/__init__.py\", line 156, in send\n    response = self.next.send(request, **kwargs)\n  File \"/usr/local/lib/python3.6/dist-packages/msrest/pipeline/requests.py\", line 193, in send\n    self.driver.send(request.http_request, **kwargs)\n  File \"/usr/local/lib/python3.6/dist-packages/msrest/universal_http/requests.py\", line 328, in send\n    return super(RequestsHTTPSender, self).send(request, **requests_kwargs)\n  File \"/usr/local/lib/python3.6/dist-packages/msrest/universal_http/requests.py\", line 137, in send\n    **kwargs)\n  File \"/usr/lib/python3/dist-packages/requests/sessions.py\", line 520, in request\n    resp = self.send(prep, **send_kwargs)\n  File \"/usr/lib/python3/dist-packages/requests/sessions.py\", line 637, in send\n    r = dispatch_hook('response', hooks, r, **kwargs)\n  File \"/usr/lib/python3/dist-packages/requests/hooks.py\", line 31, in dispatch_hook\n    _hook_data = hook(hook_data, **kwargs)\n  File \"/usr/local/lib/python3.6/dist-packages/azure/keyvault/custom/key_vault_authentication.py\", line 146, in _handle_401\n    security = self._get_message_security(prep, challenge)\n  File \"/usr/local/lib/python3.6/dist-packages/azure/keyvault/custom/key_vault_authentication.py\", line 172, in _get_message_security\n    scheme))\n  File \"/usr/local/lib/python3.6/dist-packages/azure/keyvault/custom/key_vault_authentication.py\", line 61, in _auth_callback_compat\n    if len(inspect.getargspec(self._user_callback).args) == 3 \\\n  File \"/tmp/ansible_azure_rm_keyvaultsecret_payload_9kvx9sj7/__main__.py\", line 184, in auth_callback\nKeyError: 'client_id'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

Maybe related to MSI and the proxy. I don't see the flag "proxies" in the code like for other class.

[nicolas-marcq]

To fix this on my side (Ubuntu system), I commented out the MSI part.

      # MSI is blocking when using service principal credentials
        # try:
        #     self.log("Get KeyVaultClient from MSI")
        #     credentials = MSIAuthentication(resource='https://vault.azure.net')
        #     return KeyVaultClient(credentials)
        # except Exception:
        #     self.log("Get KeyVaultClient from service principal")