azure dynamic inventory issue

[abhaysrivastava83]

Hello all,

I am facing issue with dynamic inventory for azure getting following error ansible-inventory -i test.azure_rm.yaml --graph -vvv ansible-inventory [core 2.13.7] config file = /etc/ansible/ansible.cfg configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python3/dist-packages/ansible ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections executable location = /usr/bin/ansible-inventory python version = 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] jinja version = 3.0.3 libyaml = True Using /etc/ansible/ansible.cfg as config file host_list declined parsing /root/test.azure_rm.yaml as it did not pass its verify_file() method script declined parsing /root/test.azure_rm.yaml as it did not pass its verify_file() method Using inventory plugin 'ansible_collections.azure.azcollection.plugins.inventory.azure_rm' to process inventory source '/root/test.azure_rm.yaml' toml declined parsing /root/test.azure_rm.yaml as it did not pass its verify_file() method [WARNING]: * Failed to parse /root/test.azure_rm.yaml with auto plugin: Failed to get credentials. Either pass as parameters, set environment variables, define a profile in ~/.azure/credentials, or install Azure CLI and log in (az login). File "/usr/lib/python3/dist-packages/ansible/inventory/manager.py", line 290, in parse_source plugin.parse(self._inventory, self._loader, source, cache=cache) File "/usr/lib/python3/dist-packages/ansible/plugins/inventory/auto.py", line 59, in parse plugin.parse(inventory, loader, path, cache=cache) File "/root/.ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 220, in parse self._credential_setup() File "/root/.ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 241, in _credential_setup self.azure_auth = AzureRMAuth(*auth_options) File "/root/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1479, in init self.fail("Failed to get credentials. Either pass as parameters, set environment variables, " File "/root/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1605, in fail self._fail_impl(msg) File "/root/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1608, in _default_fail_impl raise AzureRMAuthException(msg) [WARNING]: Failed to parse /root/test.azure_rm.yaml with yaml plugin: Plugin configuration YAML file, not YAML inventory File "/usr/lib/python3/dist-packages/ansible/inventory/manager.py", line 290, in parse_source plugin.parse(self._inventory, self._loader, source, cache=cache) File "/usr/lib/python3/dist-packages/ansible/plugins/inventory/yaml.py", line 114, in parse raise AnsibleParserError('Plugin configuration YAML file, not YAML inventory') [WARNING]: * Failed to parse /root/test.azure_rm.yaml with ini plugin: Invalid host pattern 'plugin:' supplied, ending in ':' is not allowed, this character is reserved to provide a port. File "/usr/lib/python3/dist-packages/ansible/inventory/manager.py", line 290, in parse_source plugin.parse(self._inventory, self._loader, source, cache=cache) File "/usr/lib/python3/dist-packages/ansible/plugins/inventory/ini.py", line 136, in parse raise AnsibleParserError(e) [WARNING]: Unable to parse /root/test.azure_rm.yaml as an inventory source [WARNING]: No inventory was parsed, only implicit localhost is available @all: |--@ungrouped:

we can do az vm list az vm list | wc -l 1341

my azure inventory is below cat test.azure_rm.yaml plugin: azure.azcollection.azure_rm auth_source: auto include_vm_resource_groups:

'*' vars: ansible_python_interpreter: /usr/bin/python3 plugin: azure.azcollection.azure_rm auth_source: auto include_vm_resource_groups:

'*' vars: ansible_python_interpreter: /usr/bin/python3

[akira6592]

@abhaysrivastava83

Failed to get credentials. Either pass as parameters, set environment variables, define a profile in ~/.azure/credentials, or install Azure CLI and log in (az login).

It looks like your credentials are not set.

https://docs.ansible.com/ansible/latest/scenario_guides/guide_azure.html#providing-credentials-to-azure-modules

[abhaysrivastava83]

Thanks Akira,

I already tried with az login and keep credentials file inside ~/.azure/credentials, as well as I tried with environmental variables also but still having same issue.

On Wed, 18 Jan 2023 at 18:03, Akira Yokochi @.***> wrote:

@abhaysrivastava83 https://github.com/abhaysrivastava83

Failed to get credentials. Either pass as parameters, set environment variables, define a profile in ~/.azure/credentials, or install Azure CLI and log in (az login).

It looks like your credentials are not set.

https://docs.ansible.com/ansible/latest/scenario_guides/guide_azure.html#providing-credentials-to-azure-modules

— Reply to this email directly, view it on GitHub https://github.com/ansible-collections/azure/issues/1067#issuecomment-1386981914, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHMW6WKBWMVAYBJKCLWNWXTWS7PKTANCNFSM6AAAAAAT62BVMM . You are receiving this because you were mentioned.Message ID: @.***>

-- Regards, Abhay Srivastava

Mob-9160512000

[Fred-sun]

@abhaysrivastava83 Or you can try it this way: if logging in through 'az login', set "auth_source: cli", and if setting credential variables through environment variables, set" auth_source: env". Thanks!

[Fred-sun]

@abhaysrivastava83 Is the format of your test script correct? Please help to provide the correct format use cases, thank you!

plugin: azure.azcollection.azure_rm
auth_source: auto
include_vm_resource_groups:
  - '*'

[Fred-sun]

kindly ping!

[abhaysrivastava83]

Yes I am using the same format.

On Fri, 3 Mar 2023 at 11:33, Fred-sun @.***> wrote:

kindly ping!

— Reply to this email directly, view it on GitHub https://github.com/ansible-collections/azure/issues/1067#issuecomment-1453020571, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHMW6WJ6HAQLBMVQMHTFZ6DW2GCR5ANCNFSM6AAAAAAT62BVMM . You are receiving this because you were mentioned.Message ID: @.***>

-- Regards, Abhay Srivastava

Mob-9160512000

[Fred-sun]

@akira6592 But judging from your mistake, which is your script format problem, can you provide a fully formatted use case? This helps solve the problem! Thank you very much!

[akira6592]

Perhaps the above is a message to @abhaysrivastava83 .

[abhaysrivastava83]

Thanks, I'll check it out.

On Wed, 22 Mar 2023 at 05:06, Akira Yokochi @.***> wrote:

Perhaps the above is a message to @abhaysrivastava83 https://github.com/abhaysrivastava83 .

— Reply to this email directly, view it on GitHub https://github.com/ansible-collections/azure/issues/1067#issuecomment-1478728220, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHMW6WNBMNJLKZ37IHI5AN3W5I3QFANCNFSM6AAAAAAT62BVMM . You are receiving this because you were mentioned.Message ID: @.***>

-- Regards, Abhay Srivastava

Mob-9160512000

[jmlp1]

I am truly straggling with a similar error using Azure Dynamic Inventories in an Ubuntu VM running in Azure, any given help will be very much appreciated. Looks like credentials have not been applied, created env variables to ansible azure secrets and related ids, but does not look like applying, or I might be reading the log wrongly

Config Steps

I created an Ubuntu VM, installed pre-requisites and Ansible

I created an Azure Service Principal and assigned rbac roles, exported the usual values to env variables

Created the below files

azure credentials cat /home/azureuser/.azure/credentials [default] subscription_id=0000000000000000000000000000 client_id=0000000000000000000000000000000000 secret=0000000000000000000000000000000000000 tenant=0000000000000000000000000000000000000

ansible.cfg cat /etc/ansible/ansible.cfg [defaults] inventory=/home/azureuser/.ansible/inventory_azure_rm.yml [inventory] enable_plugins = host_list, script, auto, yaml, ini, toml

inventory_azure_rm.yml cat /home/azureuser/.ansible/inventory_azure_rm.yml plugin: azure.azcollection.azure_rm auth_source: credential_file include_vm_resource_groups:

• devops

azure_rm.ini cat /home/azureuser/.ansible/azure_rm.ini [azure] include_powerstate=yes group_by_resource_group=yes group_by_location=yes group_by_security_group=yes group_by_tag=yes

From the VM launched ansible-inventory -vvvv -i /home/azureuser/.ansible/inventory_azure_rm.yml --list

Reply and Error ansible-inventory [core 2.14.4] config file = /etc/ansible/ansible.cfg configured module search path = ['/home/azureuser/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python3/dist-packages/ansible ansible collection location = /home/azureuser/.ansible/collections:/usr/share/ansible/collections executable location = /usr/bin/ansible-inventory python version = 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] (/usr/bin/python3) jinja version = 3.0.3 libyaml = True Using /etc/ansible/ansible.cfg as config file setting up inventory plugins host_list declined parsing /home/azureuser/.ansible/inventory_azure_rm.yml as it did not pass its verify_file() method script declined parsing /home/azureuser/.ansible/inventory_azure_rm.yml as it did not pass its verify_file() method Loading collection azure.azcollection from /home/azureuser/.ansible/collections/ansible_collections/azure/azcollection Using inventory plugin 'ansible_collections.azure.azcollection.plugins.inventory.azure_rm' to process inventory source '/home/azureuser/.ansible/inventory_azure_rm.yml' toml declined parsing /home/azureuser/.ansible/inventory_azure_rm.yml as it did not pass its verify_file() method [WARNING]: * Failed to parse /home/azureuser/.ansible/inventory_azure_rm.yml with auto plugin: name 'azure_cloud' is not defined File "/usr/lib/python3/dist-packages/ansible/inventory/manager.py", line 293, in parse_source plugin.parse(self._inventory, self._loader, source, cache=cache) File "/usr/lib/python3/dist-packages/ansible/plugins/inventory/auto.py", line 59, in parse plugin.parse(inventory, loader, path, cache=cache) File "/home/azureuser/.ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 221, in parse self._credential_setup() File "/home/azureuser/.ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 242, in _credential_setup self.azure_auth = AzureRMAuth(*auth_options) File "/home/azureuser/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1529, in init self._cloud_environment = azure_cloud.AZURE_PUBLIC_CLOUD # SDK default [WARNING]: Failed to parse /home/azureuser/.ansible/inventory_azure_rm.yml with yaml plugin: Plugin configuration YAML file, not YAML inventory File "/usr/lib/python3/dist-packages/ansible/inventory/manager.py", line 293, in parse_source plugin.parse(self._inventory, self._loader, source, cache=cache) File "/usr/lib/python3/dist-packages/ansible/plugins/inventory/yaml.py", line 114, in parse raise AnsibleParserError('Plugin configuration YAML file, not YAML inventory') [WARNING]: * Failed to parse /home/azureuser/.ansible/inventory_azure_rm.yml with ini plugin: Invalid host pattern 'plugin:' supplied, ending in ':' is not allowed, this character is reserved to provide a port. File "/usr/lib/python3/dist-packages/ansible/inventory/manager.py", line 293, in parse_source plugin.parse(self._inventory, self._loader, source, cache=cache) File "/usr/lib/python3/dist-packages/ansible/plugins/inventory/ini.py", line 137, in parse raise AnsibleParserError(e) [WARNING]: Unable to parse /home/azureuser/.ansible/inventory_azure_rm.yml as an inventory source [WARNING]: No inventory was parsed, only implicit localhost is available { "_meta": { "hostvars": {} }, "all": { "children": [ "ungrouped" ] }

[Fred-sun]

@akira6592 @jmlp1 Can you help provide pictures of the use cases? This will make it easier to find formatting problems, because from your error, it is the script problem, or just copy the use case below, thank you!

plugin: azure.azcollection.azure_rm
auth_source: auto
include_vm_resource_groups:
  - '*'

[jmlp1]

@Fred-sun Hope this is what you are asking

I tried with auth_source combinations as auto and others, but not vail, I have spent an awful amount of time just trying to setup this Dynamic Inventory and have not even started to test or get any practice application of this suite

[akira6592]

It seems the above comment for @Fred-sun. not me.

[jmlp1]

@akira6592 Well spotted, thanks rectified

[cycotron69]

I think the Python version of Azure CLI is needed. If you are using Azure CLI from MS, try running pip3 install azure-cli instead.

[jmlp1]

@cycotron69 I installed the AZ CLI from MS Indeed, I removed and installed the pip3 version with the suggested command, which is amazing, scan installed versions and leftovers and install the pip3 version instead.

pip3 install azure-cli

Same issue

[jmlp1]

@akira6592 @cycotron69 @Fred-sun @abhaysrivastava83 Managed to fix my issue and got a full inventory ref: https://stackoverflow.com/questions/71256239/azure-rm-ansible-plugin-fails-to-parse-dynamic-inventory/71266343#71266343

First I removed all pip3 azure cli modules which I just had installed, then ran the below

ansible-galaxy collection install azure.azcollection && pip install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt

After successfully running de above in that order re-ran the inventory and just worked

[Fred-sun]

@jmlp1 Thanks for your feedback!

[Fred-sun]

@abhaysrivastava83 Dy you still have this problems?

[flmmartins]

Hello,

I did what was mentioned in this post and the dynamic inventory works fine. I removed all my pip pre-installed packages and ran:

ansible-galaxy collection install azure.azcollection && pip install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt

If I upgrade/install azure-cli everything breaks again. I need az-cli in order to login and also to ssh. My inventory files has the lines mentioned before:

plugin: azure.azcollection.azure_rm
auth_source: auto
include_vm_resource_groups:
  - '*'

I think there's some incompatibility with azure-cli. Here is what happens when I use ansible-inventory command after doing a pip install azure-cli. I have also tried to pin the version of azure-cli to 2.34.0 which is the one installed by you but no success.

Azure CLI version: 2.58.0 Collection version: 2.2.0

Even if I az login before error still happens

$ ansible-inventory -vvvv -i inventory/bla.yaml --list

Using /Users/f.martins/work/solutions/clp/cd/devops/db-platform/ansible/ansible.cfg as config file setting up inventory plugins Loading collection ansible.builtin from host_list declined parsing /Users/f.martins/work/solutions/clp/cd/devops/db-platform/ansible/inventory/subs_azure_rm.yaml as it did not pass its verify_file() method script declined parsing /Users/f.martins/work/solutions/clp/cd/devops/db-platform/ansible/inventory/subs_azure_rm.yaml as it did not pass its verify_file() method Loading collection azure.azcollection from /Users/f.martins/.ansible/collections/ansible_collections/azure/azcollection Using inventory plugin 'ansible_collections.azure.azcollection.plugins.inventory.azure_rm' to process inventory source '/Users/f.martins/work/solutions/clp/cd/devops/db-platform/ansible/inventory/subs_azure_rm.yaml' toml declined parsing /Users/f.martins/work/solutions/clp/cd/devops/db-platform/ansible/inventory/subs_azure_rm.yaml as it did not pass its verify_file() method [WARNING]: * Failed to parse /Users/f.martins/work/solutions/clp/cd/devops/db-platform/ansible/inventory/subs_azure_rm.yaml with auto plugin: name 'AzureCliCredential' is not defined File "/Users/f.martins/.pyenv/versions/3.12.2/envs/db-platform/lib/python3.12/site-packages/ansible/inventory/manager.py", line 293, in parse_source plugin.parse(self._inventory, self._loader, source, cache=cache) File "/Users/f.martins/.pyenv/versions/3.12.2/envs/db-platform/lib/python3.12/site-packages/ansible/plugins/inventory/auto.py", line 59, in parse plugin.parse(inventory, loader, path, cache=cache) File "/Users/f.martins/.ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 231, in parse self._credential_setup() File "/Users/f.martins/.ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 253, in _credential_setup self.azure_auth = AzureRMAuth(*auth_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/f.martins/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1423, in init self.credentials = self._get_credentials( ^^^^^^^^^^^^^^^^^^^^^^ File "/Users/f.martins/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1663, in _get_credentials cli_credentials = self._get_azure_cli_credentials(subscription_id=params.get('subscription_id')) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/f.martins/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1622, in _get_azure_cli_credentials az_cli = AzureCliCredential() ^^^^^^^^^^^^^^^^^^ [WARNING]: Failed to parse /Users/f.martins/work/solutions/clp/cd/devops/db-platform/ansible/inventory/subs_azure_rm.yaml with yaml plugin: Plugin configuration YAML file, not YAML inventory File "/Users/f.martins/.pyenv/versions/3.12.2/envs/db-platform/lib/python3.12/site-packages/ansible/inventory/manager.py", line 293, in parse_source plugin.parse(self._inventory, self._loader, source, cache=cache) File "/Users/f.martins/.pyenv/versions/3.12.2/envs/db-platform/lib/python3.12/site-packages/ansible/plugins/inventory/yaml.py", line 114, in parse raise AnsibleParserError('Plugin configuration YAML file, not YAML inventory') [WARNING]: * Failed to parse /Users/f.martins/work/solutions/clp/cd/devops/db-platform/ansible/inventory/subs_azure_rm.yaml with ini plugin: Invalid host pattern '---' supplied, '---' is normally a sign this is a YAML file. File "/Users/f.martins/.pyenv/versions/3.12.2/envs/db-platform/lib/python3.12/site-packages/ansible/inventory/manager.py", line 293, in parse_source plugin.parse(self._inventory, self._loader, source, cache=cache) File "/Users/f.martins/.pyenv/versions/3.12.2/envs/db-platform/lib/python3.12/site-packages/ansible/plugins/inventory/ini.py", line 138, in parse raise AnsibleParserError(e) [WARNING]: Unable to parse /Users/f.martins/work/solutions/clp/cd/devops/db-platform/ansible/inventory/subs_azure_rm.yaml as an inventory source [WARNING]: No inventory was parsed, only implicit localhost is available { "_meta": { "hostvars": {} }, "all": { "children": [ "ungrouped" ] } }

The workaround I have for this now is to: run azure-cli in a different virtualenv then azure-collection packages =/

Can anyone help me figure this out? Thanks!

[Fred-sun]

@abhaysrivastava83 Yes, there is an incompatibility problem and we don't have a good solution yet. But now for an alternative, First install 'azure-cli==2.34 ', then 'az login', and finally install 'azure.azcollection' and' pip3 install -r ~/ requirements-azure.txt ', Thank you!

[flmmartins]

Hello @Fred-sun ! Thanks for answering! Yes, but the problem is that after installing the azcollection libs azure-cli becomes unusable. Check logs below. Anyhow good thing that you are aware of that =)

az login

The command failed with an unexpected error. Here is the traceback: argument _command_package: conflicting subparser: login Traceback (most recent call last): File "/Users/f.martins/.pyenv/versions/3.12.2/envs/db-platform/lib/python3.12/site-packages/knack/cli.py", line 231, in invoke cmd_result = self.invocation.execute(args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/f.martins/.pyenv/versions/3.12.2/envs/db-platform/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 560, in execute self.parser.load_command_table(self.commands_loader) File "/Users/f.martins/.pyenv/versions/3.12.2/envs/db-platform/lib/python3.12/site-packages/azure/cli/core/parser.py", line 100, in load_command_table command_parser = subparser.add_parser(command_verb, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/f.martins/.pyenv/versions/3.12.2/lib/python3.12/argparse.py", line 1214, in addparser raise ArgumentError(self, ('conflicting subparser: %s') % name) argparse.ArgumentError: argument _command_package: conflicting subparser: login

[HarishHary]

Hello, anyone has found a solution for this?

[flmmartins]

Hello, anyone has found a solution for this?

There's no solution but as a workaround you can create a virtual environment with pyenv virtualenv for example. It has to be a clear virtualenv with no packages installed and do:

ansible-galaxy collection install azure.azcollection && pip install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt

Everytime you need to run the ansible code you need to activate this virtualenv.

[Fred-sun]

@HarishHary There is no effective way to solve the problem caused by package dependency. But that's what I usually do, to make the two compatible. Thank you!

First update the system, sudo apt update
Second: install pip3 and specify the azure-cli version, pip3 install azure-cli==2.34.0
Again: login 'az login --tenant=****'
Finally, install aze.azcollection and its dependencies.  ansible-galaxy collection install azure.azcollection && pip install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements.txt

[Fred-sun]

@abhaysrivastava83 The azure-cli-core has been updated to v2.61.0（relate PR #1593). Could you try again?