Unable to use `azure_rm_aduser_info` or `azure_rm_adgroup_info`: Insufficient privileges to complete the operation.

[l3ender]

SUMMARY

I am unable to use the azure_rm_adgroup_info or azure_rm_aduser_info modules. They report:

"msg": "failed to get ad group info Insufficient privileges to complete the operation."

I am using a service principal which has Owner access for the subscription and looks like it has sufficient API access for user information:

ISSUE TYPE

• Bug Report

COMPONENT NAME

azure_rm_adgroup_info azure_rm_aduser_info

ANSIBLE VERSION

-> ansible --version
ansible [core 2.11.2]
  config file = /Users/ross/repos/azure-config/ansible.cfg
  configured module search path = ['/Users/ross/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/ross/repos/azure-config/venv/lib/python3.9/site-packages/ansible
  ansible collection location = /Users/ross/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/ross/repos/azure-config/venv/bin/ansible
  python version = 3.9.1 (default, Dec 10 2020, 10:36:35) [Clang 12.0.0 (clang-1200.0.32.27)]
  jinja version = 3.0.1
  libyaml = True

COLLECTION VERSION

-> ansible-galaxy collection list azure.azcollection

# /Users/ross/repos/azure-config/venv/lib/python3.9/site-packages/ansible_collections
Collection         Version
------------------ -------
azure.azcollection 1.7.0

# /Users/ross/.ansible/collections/ansible_collections
Collection         Version
------------------ -------
azure.azcollection 1.7.0

CONFIGURATION

-> ansible-config dump --only-changed
DISPLAY_SKIPPED_HOSTS(/Users/ross/repos/azure-config/ansible.cfg) = False

OS / ENVIRONMENT

Mac OS Big Sur 11.4.

STEPS TO REPRODUCE

I am testing using the following playbook:

---
- name: "Test loading AD group info"
  hosts: localhost
  connection: local
  gather_facts: false

  tasks:
    - name: "Get group info."
      azure.azcollection.azure_rm_adgroup_info:
        tenant: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
        all: true
      register: group_info

EXPECTED RESULTS

I would be able to retrieve results of groups in the tenant.

ACTUAL RESULTS

Module fails with error.

-> ansible-playbook sandbox.yml
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [Test loading AD group info] *******************************************************************************************************************************************************

TASK [Get group info.] ******************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "failed to get ad group info Insufficient privileges to complete the operation."}

PLAY RECAP ******************************************************************************************************************************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Reference: https://github.com/ansible-collections/azure/pull/423.

[l3ender]

@coleneubauer - any ideas?

[Fred-sun]

@coleneubauer - any ideas?

@l3ender Thank you for submitting this issue and we will find out the cause as soon as possible.

[l3ender]

From my research, I've found the Azure Active Directory Graph API permission Directory.Read.All is required to use the modules azure_rm_adgroup_info and azure_rm_aduser_info. Once I added these under the "API Permissions" section of the app registration (service principal), the two modules worked as expected.

This brings up another question, however: when adding the permission, I see the following warning:

This application is using Azure AD Graph API, which is on a deprecation path. Starting June 30th, 2020 we will no longer add any new features to Azure AD Graph API. We strongly recommend that you upgrade your application to use Microsoft Graph API instead of Azure AD Graph API to access Azure Active Directory resources. Learn more

Should a separate issue be opened for this collection using the deprecated APIs?

[Fred-sun]

@l3ender We are planning to upgrade all the APIs, the old ones will no longer be used, so we can ignore the old ones. Thank you very much!

[l3ender]

We are planning to upgrade all the APIs, the old ones will no longer be used, so we can ignore the old ones.

Thank you! When this is done, please note it in the release notes as it is a breaking change in which a different API permission will need to be granted for account/service principal.

Thanks!

[l3ender]

Apologies for the accidental close! I will keep open as perhaps a documentation PR should be added for these modules.

[Fred-sun]

@l3ender Can you set "auth_source: cli" in the playbook to retry? The current 'ad' related modules only support CLI Credentials (az login). Thank you very much!

[l3ender]

@Fred-sun I was able to make the modules work with a credentials file. The issue was that the service principal I was using didn't have the necessary permissions (it seems they aren't granted by default). Once I configured the SP with the above API access, our playbook worked as expected with a credentials file.

I kept the issue open because it seems good to add notes in documentation (either for the modules, authentication setup, or both).

Thank you!

[johnpetersjr]

Sorry for attaching to this old issue - can create a new one if desired -- I use AWX to run all my ansible modules, and use an attached credential type of "Microsoft Azure Resource Manager" that has my Service Principal ID/Secret to authenticate with Azure. From reading comments above, it seems I need to set

auth_source: cli

in my task, which I've never had to do before. This is now causing me grief in the azure_rm_adgroup module task, because it:

Could not retrieve credential from local cache for service principal

Likely because I'm using a remote machine to run this job, which doesn't 'hold' onto my credentials from play to play in the local venv.

Ideas on how to solve this? Without specifying the auth_source on other azcollection modules, I have no issues building Azure Cloud infrastructure. I simply set these env vars and everything works fine:

  vars:
    # Necessary ENV Variables for Azure Resource Manager Credentials
    client_id: "{{ lookup('env', 'AZURE_CLIENT_ID') }}"
    secret: "{{ lookup('env', 'AZURE_SECRET') }}"
    subscription_id: "{{ lookup('env', 'AZURE_SUBSCRIPTION_ID') }}"
    tenant: "{{ lookup('env', 'AZURE_TENANT') }}"

But it seems this particular azure_rm_adgroup module doesn't work that way... Has anyone gotten this module to work from AWX, using a Service Principal Credential?

[l3ender]

@johnpetersjr Sorry for the slow response. I can't speak exactly to using azure_rm_adgroup, but we are using azure_rm_adgroup_info from Ansible Tower without issue. We don't need to specify auth_source in the playbook nor specify any vars with environment lookups. We simply bind the "Microsoft Azure Resource Manager" credential to the job template and it works as expected.

I'm not sure if it matters, but our job template is configured to run on a "localhost" inventory, i.e. the same node as where Tower is installed.

Hope that helps!

[radhikari-arch]

Is auth_source. msi supported for these azure.azcollection.azure_rm_adgroup_info and azure.azcollection.azure_rm_adgroup modules? From the dcoumentaiton, it sure says msi for auth-source. I am getting "failed to get ad group info Access Token missing or malformed." error. cc: @Fred-sun @l3ender @coleneubauer

[radhikari-arch]

I gave the global administrator role in Azure AD for the MSI virtual machine. But still getting that error "failed to get ad group info Access Token missing or malformed."

- name: Return a specific group using object_id
  azure.azcollection.azure_rm_adgroup_info:
    object_id: *****
    tenant: xxx-xxx-xx
    auth_source: msi
  environment:
    AZURE_SUBSCRIPTION_ID: xxx-xxx--xxx
    AZURE_TENANT: xxx-xxxx-xxx-

Versions:

ansible 2.9.27
python version = 3.6.8
azure.azcollection = v1.12.0

Error:

The full traceback is:
  File "/tmp/ansible_azure.azcollection.azure_rm_adgroup_info_payload_i6sayeof/ansible_azure.azcollection.azure_rm_adgroup_info_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_adgroup_info.py", line 236, in exec_module
  File "/usr/local/lib/python3.6/site-packages/azure/graphrbac/operations/groups_operations.py", line 472, in get
    raise models.GraphErrorException(self._deserialize, response)
fatal: [localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "ad_user": null,
            "adfs_authority_url": null,
            "all": false,
            "api_profile": "latest",
            "attribute_name": null,
            "attribute_value": null,
            "auth_source": "msi",
            "cert_validation_mode": null,
            "check_membership": null,
            "client_id": null,
            "cloud_environment": "AzureCloud",
            "log_mode": null,
            "log_path": null,
            "object_id": "xxx-xxx-xxx-xxx",
            "odata_filter": null,
            "password": null,
            "profile": null,
            "return_group_members": false,
            "return_member_groups": false,
            "return_owners": false,
            "secret": null,
            "subscription_id": null,
            "tenant": "xxx-xx-xxx-x"
        }
    },
    "msg": "failed to get ad group info Access Token missing or malformed."
}

Howevere, with the same tenant id and subs, using the msi, create resource module works fine:

- name: Create resource group 
  azure_rm_resourcegroup:
    name: "My-testing"
    location: eastus2
    auth_source: msi
  environment:
    AZURE_SUBSCRIPTION_ID:  xxxxxx
    AZURE_TENANT: xxxxxxxx

[radhikari-arch]

Well, I found a way to add the Directory.Read.All permissions for the MSI from here: https://docs.microsoft.com/en-us/azure/app-service/scenario-secure-app-access-microsoft-graph-as-app?tabs=azure-cli#grant-access-to-microsoft-graph However, even with that still gave the same error: "failed to get ad group info Access Token missing or malformed." I then created an App Registration and gave the Directory.Read.All access on this as well. For this, I got an error: 'ADGroup' object has no attribute 'mail_nickname' Looks like API has some limitations! Also @Fred-sun mentioned above, they are planning to upgrade all APIs. Checking to see if there is any updates on this.

[Fred-sun]

Because all the apis need to be upgraded, it's a big project that we're working on. Thank you very much!

[Fred-sun]

@l3ender @radhikari-arch It has support mscrosoft gaph in PR #1112, Please review! Thank you very much!

[felixmarch]

I tried the codes on Fred-sun:Add_msgrpah_support, it works great :thumbsup: :thumbsup:

---
- hosts: localhost
  gather_facts: false
  tasks:
  - name: Query all user infos
    azure.azcollection.azure_rm_msuser_info:
      all: true
    register: azure_rm_aduser_info
  - debug: var=azure_rm_aduser_info

# ansible-playbook azure_playbook.yml
...
...
PLAY RECAP ***************************************************************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

[Fred-sun]

@felixmarch Thanks for your test! I will push to merge as soon as possible!

[felixmarch]

Hi @Fred-sun I tried azure_rm_msuser to add new user like this

---
- hosts: localhost
  gather_facts: false
  tasks:
  - name: Add Users
    azure.azcollection.azure_rm_msuser:
      user_principal_name: "xxx.xxx@xxxxxx.onmicrosoft.com"
      display_name: "xxx xxx"
      account_enabled: True
      mail_nickname: "xxx xxx"
      password_profile:
        Password: Secret1111!!!!
      mail: "xxx.xxx@xxxxxx.onmicrosoft.com"
      state: "present"

During the run, I got error like this:

# ansible-playbook adduser_azure_playbook.yml -vvvvvv
...
...
1406658 1679888825.94367: ^ failed state is now: HOST STATE: block=2, task=3, rescue=0, always=0, run_state=ITERATING_COMPLETE, fail_state=FAILED_TASKS, pending_setup=False, tasks child state? (None), rescue child state? (None), always child state? (None), did rescue? False, did start at task? False
1406658 1679888825.94374: getting the next task for host localhost
1406658 1679888825.94379: host localhost is done iterating, returning
The full traceback is:
Traceback (most recent call last):
  File "/root/.ansible/tmp/ansible-tmp-1679888822.5047784-1406714-26876904145436/AnsiballZ_azure_rm_msuser.py", line 102, in <module>
    _ansiballz_main()
  File "/root/.ansible/tmp/ansible-tmp-1679888822.5047784-1406714-26876904145436/AnsiballZ_azure_rm_msuser.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/root/.ansible/tmp/ansible-tmp-1679888822.5047784-1406714-26876904145436/AnsiballZ_azure_rm_msuser.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible_collections.azure.azcollection.plugins.modules.azure_rm_msuser', init_globals=None, run_name='__main__', alter_sys=True)
  File "/usr/lib64/python3.6/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib64/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py", line 283, in <module>
  File "/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py", line 279, in main
  File "/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py", line 174, in __init__
  File "/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 473, in __init__
  File "/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py", line 243, in exec_module
  File "/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py", line 267, in user_to_dict
KeyError: 'id'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/root/.ansible/tmp/ansible-tmp-1679888822.5047784-1406714-26876904145436/AnsiballZ_azure_rm_msuser.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/root/.ansible/tmp/ansible-tmp-1679888822.5047784-1406714-26876904145436/AnsiballZ_azure_rm_msuser.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/root/.ansible/tmp/ansible-tmp-1679888822.5047784-1406714-26876904145436/AnsiballZ_azure_rm_msuser.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.azure.azcollection.plugins.modules.azure_rm_msuser', init_globals=None, run_name='__main__', alter_sys=True)\n  File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n    mod_name, mod_spec, pkg_name, script_name)\n  File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py\", line 283, in <module>\n  File \"/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py\", line 279, in main\n  File \"/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py\", line 174, in __init__\n  File \"/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py\", line 473, in __init__\n  File \"/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py\", line 243, in exec_module\n  File \"/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py\", line 267, in user_to_dict\nKeyError: 'id'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}
1406658 1679888825.94493: no more pending results, returning what we have
1406658 1679888825.94502: results queue empty
...
...

Any steps I miss here? :thinking:

[Fred-sun]

@felixmarch I added a judgment on the creation failure and returned an error message, please try again! Thank you very much!

[Fred-sun]

@felixmarch I added a judgment on the creation failure and returned an error message, please try again! Thank you very much!

[felixmarch]

Cool, able to identify the error detail now. 👍 👍

# ANSIBLE_STDOUT_CALLBACK=yaml ansible-playbook adduser_azure_playbook.yml -vvvvvv
...
...
'Create ad user fail, Msg {''code'': ''Request_BadRequest'', ''message'': "Invalid value specified for property ''mailNickname'' of resource ''User''.", ''details'': [{''code'': ''InvalidValue'', ''message'': "Invalid value specified for property ''mailNickname'' of resource ''User''.", ''target'': ''mailNickname''}], ''innerError'': {''date'': ''2023-03-27T09:32:38'', ''request-id'': ''c503367b-17aa-4b0d-ba66-1d7c94755161'', ''client-request-id'': ''3042f0ca-a323-44a4-8bd8-6f7de2d9c797''}}'
...
...

Further check, it was caused by unsupported characters in mailNickname, similar to what reported in this forum

After removing the offending chars, it works fine then.

Also observed certain attributes like given_name, surname, usage_location were no longer supported.

    "msg": "Unsupported parameters for (azure.azcollection.azure_rm_msuser) module: given_name, surname, usage_location Supported parameters include: account_enabled, ad_user, adfs_authority_url, api_profile, auth_source, cert_validation_mode, ct, display_name, log_mode, log_path, mail, mail_nickname, object_id, password, password_profile, profile, secret, state, subscription_id, tenant, thumbprint, user_principal_name, x509_certificate_path"

It may be good in future to have custom attributes similar to what is provided in the community.windows.win_domain_user module. :blush:

[felixmarch]

Is this "give_name" typos? (not "given_name"?) :thinking:

• plugins/modules/azure_rm_msuser.py
• plugins/modules/azure_rm_msuser_info.py

[felixmarch]

I noticed the update logic (the "should_update" flag in azure_rm_aduser.py) was not implemented yet in azure_rm_msuser.py.

So, when I tried to update the "display_name" against existing "user_principal_name", I got error like this:

"msg": "Create ad user fail, Msg {'code': 'Request_BadRequest', 'message': 'Property netId is invalid.', 'details': [{'code': 'GenericError', 'message': 'Property netId is invalid.', 'target': 'netId'}], 'innerError': {'date': '2023-03-27T10:08:01', 'request-id': '6fdf7664-0de8-4809-9a8b-db2a8d9ab683', 'client-request-id': 'd50ac719-3fae-4d7f-b8e7-f92d0489716f'}}"

Hopefully the incoming work will address it :blush:

[Fred-sun]

@felixmarch Thanks for your advice, The spelling errors has update. This module does not support updating for the time being. If there is a good way, it will be improved. Thanks!

[Fred-sun]

commit as bellow!

https://github.com/ansible-collections/azure/pull/1112/commits/2fb6a6827fdbaa35ed687194967e8b46e746b421

[Fred-sun]

@l3ender azure-graphrbac has beed deprecated, and migrate to msgraph-sdk. It has support in v2.1.0. I will closed it! If you have any question. Please repoen a new! Thank you very much!