search

ansible-collections / azure

Development area for Azure Collections
https://galaxy.ansible.com/azure/azcollection
GNU General Public License v3.0
245 stars 327 forks source link

MSAL Token Cache Multiple Token Problem - Enterprise Azure Account #918

Open siddallhotmail opened 2 years ago

siddallhotmail commented 2 years ago
SUMMARY

Whilst using an enterprise azure login (multiple subscriptions under a single tenant). The auth_sources property when set to use CLI fails with User 'xx' does not exist in MSAL token cache.

Using a standard azure subscription / 1-2-1 logon the azure modules work fine.

I suspect that this is because of the multiple tokens I obtain when logging in using the (now depreciated) function in azure_rm_common.py

self.log('Retrieving credentials from Azure CLI profile') cli_credentials = self._get_azure_cli_credentials(subscription_id=params.get('subscription_id')) return cli_credentials

Which in turn calls get_cli_profile() which is part of the azure-sdk-for-python

These functions are depreciated now ... and the documentation suggests moving to the Azure Identity client library for Python.

https://docs.microsoft.com/en-us/python/api/azure-common/azure.common.credentials?view=azure-python

I think the crux of the issue is that due to the multiple tokens received when using the AZ LOGIN command, we would need to be able to specify the subscription ID and tenant ID to correctly obtain the token from the cache.

ISSUE TYPE
COMPONENT NAME

azure_rm_common.py

ANSIBLE VERSION
ansible [core 2.13.1]
  config file = /home/g3.martin.siddall@hosting.civica.com/ccsautomation/05_provision_developer/ansible.cfg
  configured module search path = ['/home/g3.martin.siddall@hosting.civica.com/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/g3.martin.siddall@hosting.civica.com/ansible_stable/lib/python3.8/site-packages/ansible
  ansible collection location = /home/g3.martin.siddall@hosting.civica.com/ccsautomation/00_common/shared/resources/collections
  executable location = /home/g3.martin.siddall@hosting.civica.com/ansible_stable/bin/ansible
  python version = 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0]
  jinja version = 3.1.2
  libyaml = True
COLLECTION VERSION
azure.azcollection            1.12.0
CONFIGURATION
COLLECTIONS_PATHS(/home/g3.martin.siddall@hosting.civica.com/ccsautomation/05_provision_developer/ansible.cfg) = ['/home/g3.martin.siddall@hosting.civica.com/ccsautomation/>
DEFAULT_FORKS(/home/g3.martin.siddall@hosting.civica.com/ccsautomation/05_provision_developer/ansible.cfg) = 30
DEFAULT_HOST_LIST(/home/g3.martin.siddall@hosting.civica.com/ccsautomation/05_provision_developer/ansible.cfg) = ['/home/g3.martin.siddall@hosting.civica.com/ccsautomation/>
DISPLAY_SKIPPED_HOSTS(/home/g3.martin.siddall@hosting.civica.com/ccsautomation/05_provision_developer/ansible.cfg) = False
HOST_KEY_CHECKING(/home/g3.martin.siddall@hosting.civica.com/ccsautomation/05_provision_developer/ansible.cfg) = False
LOCALHOST_WARNING(/home/g3.martin.siddall@hosting.civica.com/ccsautomation/05_provision_developer/ansible.cfg) = False
OS / ENVIRONMENT

Ubuntu 20.04

STEPS TO REPRODUCE

You will require a AZ Login which has multiple tenants under the same subscription id.

Example of tokens obtained when logging in., you will note tenant id is the same ... but has multiple subscriptions.


  {
    "cloudName": "AzureCloud",
    "id": "xxxxxxx-8ec9-4bf7-8f4f-fba9ed4069ed",
    "isDefault": true,
    "name": "Visual Studio Professional",
    "state": "Enabled",
    "tenantId": "xxxxxxx-727e-413f-b5b8-cd0f27c02b6b",
    "user": {
      "name": "example.user@test.domain",
      "type": "user"
    }
  },
  {
    "cloudName": "AzureCloud",
    "id": "xxxxxxx-9d7d-4450-901a-e16f09f9fd1a",
    "isDefault": false,
    "name": "Visual Studio Professional",
    "state": "Enabled",
    "tenantId": "xxxxxxx-727e-413f-b5b8-cd0f27c02b6b",
    "user": {
      "name": "example.user@test.domain",
      "type": "user"
    }
  },
  {
    "cloudName": "AzureCloud",
    "id": "xxxxxxx-5cad-4521-99f6-3d7ad1d8a384",
    "isDefault": false,
    "name": "Visual Studio Professional",
    "state": "Enabled",
    "tenantId": "xxxxxxx-727e-413f-b5b8-cd0f27c02b6b",
    "user": {
      "name": "example.user@test.domain",
      "type": "user"
    }
  },
  {
    "cloudName": "AzureCloud",
    "id": "xxxxxxx-016a-43ee-8bd4-ce97f73afc27",
    "isDefault": false,
    "name": "Visual Studio Professional",
    "state": "Enabled",
    "tenantId": "xxxxxxx-727e-413f-b5b8-cd0f27c02b6b",
    "user": {
      "name": "example.user@test.domain",
      "type": "user"
    }
  }
]

Running the following module against the localhost.

azure_rm_resourcegroup_info:
      auth_source: cli
      subscription_id: xxxxx-5cad-4521-99f6-3d7ad1d8a384
      tenant: xxxxx-727e-413f-b5b8-cd0f27c02b6b
EXPECTED RESULTS

Azure CLI profile cannot be loaded - User 'xxxx' does not exist in MSAL token cache.

Fred-sun commented 2 years ago

@martinpsiddall If you use 'az login', can you specify the subscription_id you want to use and retry? Thank you very much!


az login
az account set -s *************
Fred-sun commented 2 years ago

kindly ping!

siddallgoogle commented 2 years ago

Apologies Fred, been away on holiday.

Unfortunately the suggested work-around presents the same MSAL error.

siddallgoogle commented 2 years ago

Any further suggestions / debugging I can do ?

Fred-sun commented 2 years ago

@MartinSiddall After you log in, you need to specify which account subscription you use, such as "az account set-s sub_use_id". Thank you!

siddallgoogle commented 2 years ago

Hi Fred this was the same suggested az login commands you made on the 14th July.

As per my previous reply this has the same MSAL login error

Azure CLI profile cannot be loaded - User 'xxxx' does not exist in MSAL token cache. Run az login."

Fred-sun commented 2 years ago

"****" means the subscription ID you want to use. Thank you very much!

siddallgoogle commented 2 years ago

I have no idea what you're referring to the User 'xxxx' bit is me removing my email address.

I have tried the command you've suggested but they result in the same error.

14th July

az login az account set -s *****

Does not work.

Today, the same command in your comment is basically the same suggestion as the 14th of July.

"az account set-s sub_use_id"

Again results in the MSAL error.

Thanks.

Fred-sun commented 2 years ago

I would like to know the ID after "az account set -s" that you specified to execute, and did you execute this command successfully?

siddallgoogle commented 2 years ago

The command worked fine, with the relevant subscription ID.

Again I've obfuscated the beginning of the ID from the first post as this is a public site.

az login az account set -s xxxxxxx-9d7d-4450-901a-e16f09f9fd1a

As an example.

Thanks

Fred-sun commented 2 years ago

I will try again, Multiple accounts for one tenant.

siddallgoogle commented 2 years ago

Yep thats what we / the team will have.

Just as a FYI using an incorrect subscription id results in

The subscription of 'xxxx' doesn't exist in cloud 'AzureCloud'.

So the set -s works fine.

Additionally the az account show ... also shows the correct subscription ID ... active subscription.

Fred-sun commented 1 year ago

Yep thats what we / the team will have.

Just as a FYI using an incorrect subscription id results in

The subscription of 'xxxx' doesn't exist in cloud 'AzureCloud'.

So the set -s works fine.

Additionally the az account show ... also shows the correct subscription ID ... active subscription.

Do you mean that the problem has been solved?

siddallgoogle commented 1 year ago

No the problem still persists, just confirming that the az commands work. We still have the MSAL token cache error message.

Cyclonit commented 1 year ago

I have exactly the same issue. I would like to use the azurerm_inventory plugin. The plugin initializes just fine, but when executing the following command I get an error:

$ ansible-inventory -i dev.azure_rm.yml --graph
[WARNING]:  * Failed to parse /mnt/c/Projects/XXXXXXXX/ansible/dev.azure_rm.yml with script plugin: problem running /mnt/c/Projects/XXXXXXXX/ansible/dev.azure_rm.yml --list ([Errno 8] Exec format error: '/mnt/c/Projects/XXXXXXXX/ansible/dev.azure_rm.yml')
[WARNING]:  * Failed to parse /mnt/c/Projects/XXXXXXXX/ansible/dev.azure_rm.yml with auto plugin: Azure CLI profile cannot be loaded - User 'my.user@company.com' does not exist in MSAL token cache. Run `az login`.
[WARNING]:  * Failed to parse /mnt/c/Projects/XXXXXXXX/ansible/dev.azure_rm.yml with yaml plugin: Plugin configuration YAML file, not YAML inventory
[WARNING]:  * Failed to parse /mnt/c/Projects/XXXXXXXX/ansible/dev.azure_rm.yml with ini plugin: Invalid host pattern 'plugin:' supplied, ending in ':' is not allowed, this character is reserved to provide a port.
[WARNING]: Unable to parse /mnt/c/Projects/XXXXXXXX/ansible/dev.azure_rm.yml as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available

I login the following way. But the error persists. Executing commands using the Azure CLI works without issue.

$ az login
$ az account set --subscription <SUBSCRIPTION_ID>

I did some searching around, and found another issue which I believe might be related to this one:

688 Could not retrieve credential from local cache for service principal ***. Please run 'az login' for this service principal.

It suggests using different combinations of Azure CLI and the Python plugin azure-cli-core. Apparently there is a mismatch on how they store/access tokens in some versions. However, neither of the suggested combinations of versions worked for me. Additionaly the issue appears to be outdated, as the requirements for this collection reference fairly new versions.

siddallgoogle commented 1 year ago

Just for reference my pip / apt package versions

azure-cli-core==2.34.0 azure-cli-telemetry==1.0.6 azure-common==1.1.11 azure-containerregistry==1.0.0 azure-core==1.24.2 azure-graphrbac==0.61.1 azure-identity==1.7.0 azure-keyvault==1.0.0a1 azure-mgmt-advisor==1.0.1 azure-mgmt-apimanagement==0.2.0 azure-mgmt-applicationinsights==0.1.1 azure-mgmt-authorization==0.51.1 azure-mgmt-automation==1.0.0 azure-mgmt-batch==5.0.1 azure-mgmt-batchai==2.0.0 azure-mgmt-billing==0.2.0 azure-mgmt-cdn==3.0.0 azure-mgmt-cognitiveservices==3.0.0 azure-mgmt-commerce==1.0.1 azure-mgmt-compute==26.1.0 azure-mgmt-consumption==2.0.0 azure-mgmt-containerinstance==1.4.0 azure-mgmt-containerregistry==9.1.0 azure-mgmt-containerservice==9.1.0 azure-mgmt-core==1.3.0 azure-mgmt-cosmosdb==0.15.0 azure-mgmt-datafactory==2.0.0 azure-mgmt-datalake-analytics==0.6.0 azure-mgmt-datalake-nspkg==2.0.0 azure-mgmt-datalake-store==0.5.0 azure-mgmt-datamigration==1.0.0 azure-mgmt-devspaces==0.1.0 azure-mgmt-devtestlabs==3.0.0 azure-mgmt-dns==8.0.0 azure-mgmt-eventgrid==1.0.0 azure-mgmt-eventhub==2.0.0 azure-mgmt-hanaonazure==0.1.1 azure-mgmt-hdinsight==0.1.0 azure-mgmt-iotcentral==0.1.0 azure-mgmt-iothub==0.7.0 azure-mgmt-iothubprovisioningservices==0.2.0 azure-mgmt-keyvault==1.1.0 azure-mgmt-loganalytics==1.0.0 azure-mgmt-logic==3.0.0 azure-mgmt-machinelearningcompute==0.4.1 azure-mgmt-managedservices==1.0.0 azure-mgmt-managementgroups==0.2.0 azure-mgmt-managementpartner==0.1.1 azure-mgmt-maps==0.1.0 azure-mgmt-marketplaceordering==0.1.0 azure-mgmt-media==1.0.1 azure-mgmt-monitor==3.0.0 azure-mgmt-msi==0.2.0 azure-mgmt-network==19.1.0 azure-mgmt-notificationhubs==2.0.0 azure-mgmt-nspkg==2.0.0 azure-mgmt-policyinsights==0.1.0 azure-mgmt-powerbiembedded==2.0.0 azure-mgmt-privatedns==0.1.0 azure-mgmt-rdbms==1.9.0 azure-mgmt-recoveryservices==0.4.0 azure-mgmt-recoveryservicesbackup==0.6.0 azure-mgmt-redis==13.0.0 azure-mgmt-relay==0.1.0 azure-mgmt-reservations==0.2.1 azure-mgmt-resource==10.2.0 azure-mgmt-scheduler==2.0.0 azure-mgmt-search==8.0.0 azure-mgmt-servicebus==0.5.3 azure-mgmt-servicefabric==0.2.0 azure-mgmt-signalr==0.1.1 azure-mgmt-sql==3.0.1 azure-mgmt-storage==19.0.0 azure-mgmt-subscription==0.2.0 azure-mgmt-trafficmanager==0.50.0 azure-mgmt-web==6.1.0 azure-nspkg==2.0.0 azure-storage==0.36.0 azure-storage-blob==2.1.0 azure-storage-common==2.1.0

azure-cli/focal-updates,now 2.0.81+ds-4ubuntu0.2 all [installed] python3-azure-cli-core/focal-updates,now 2.0.81+ds-4ubuntu0.2 all [installed,automatic] python3-azure-cli-telemetry/focal-updates,now 2.0.81+ds-4ubuntu0.2 all [installed,automatic] python3-azure-cli-testsdk/focal-updates 2.0.81+ds-4ubuntu0.2 all python3-azure-cli/focal-updates,now 2.0.81+ds-4ubuntu0.2 all [installed,automatic]

siddallgoogle commented 1 year ago

So fiddling about with versions i've managed to get my login issue fixed.

This is a similar problem

https://github.com/ansible-collections/azure/issues/688

MS Article on registering repos ... removing of azure-cli apt package

https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt

Packages to obtain

pip install azure-cli-core==2.30 sudo apt-get install azure-cli=2.30.0-1~focal

Make sure that you az logout and then use az login --use-device-code

siddallgoogle commented 1 year ago

Also works with latest packages

PIP azure-cli-core==2.34.0

APT azure-cli/focal,now 2.40.0-1~focal

soerennielsen commented 3 months ago

The problem can be caused by having access to multiple tenants with different accounts and then az login will just pick one of them, and if it picks the wrong one you get the "does not exist in MSAL token cache. Run az login." error.

Solution is simple, specify the tenant at az login. You cannot specify a subscription there, so any attempted resolution above that talks about subscriptions is not going to work.

Fix: az login --tenant yourtenant.onmicrosoft.com