search

ansible-collections / community.general

Ansible Community General Collection
https://galaxy.ansible.com/ui/repo/published/community/general/
GNU General Public License v3.0
814 stars 1.49k forks source link

Unable to use json_query with hostvars #3706

Closed tomkivlin closed 2 years ago

tomkivlin commented 2 years ago

Summary

When I try to query hostvars output (once converted to json), the json_query filters don't work. I've tested on https://jmespath.org/ and the query is sound. I am using the nb_inventory inventory plugin.

NB I originally opened on ansible/ansible but it got closed: https://github.com/ansible/ansible/issues/76289

Issue Type

Bug Report

Component Name

json_query

Ansible Version

$ ansible --version
ansible 2.10.5
  config file = /Users/kivlint/Documents/GitHub/vmware-automation/ansible/ansible.cfg
  configured module search path = ['/Users/kivlint/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/kivlint/.virtualenvs/ansible-dev/lib/python3.7/site-packages/ansible
  executable location = /Users/kivlint/.virtualenvs/ansible-dev/bin/ansible
  python version = 3.7.4 (v3.7.4:e09359112e, Jul  8 2019, 14:54:52) [Clang 6.0 (clang-600.0.57)]

Community.general Version

$ ansible-galaxy collection list community.general
# /Users/kivlint/.ansible/collections/ansible_collections
Collection        Version
----------------- -------
community.general 1.2.0  

# /Users/kivlint/.virtualenvs/ansible-dev/lib/python3.7/site-packages/ansible_collections
Collection        Version
----------------- -------
community.general 1.3.6

Configuration

$ ansible-config dump --only-changed
DEFAULT_LOAD_CALLBACK_PLUGINS(/Users/kivlint/Documents/GitHub/vmware-automation/ansible/ansible.cfg) = True
DEFAULT_STDOUT_CALLBACK(/Users/kivlint/Documents/GitHub/vmware-automation/ansible/ansible.cfg) = yaml
INVENTORY_ENABLED(/Users/kivlint/Documents/GitHub/vmware-automation/ansible/ansible.cfg) = ['yaml', 'netbox.netbox.nb_inventory']

OS / Environment

Mac OS 11.6.1

Steps to Reproduce

Running the following task produces what appears to be normal JSON that I can query elsewhere without issue:

- debug:
    msg: "{{ hostvars[inventory_hostname] | to_nice_json }}"

Example snippet of the JSON output:

{
        "ansible_check_mode": false,
        "ansible_config_file": "/Users/kivlint/Documents/GitHub/vmware-automation/ansible/ansible.cfg",
        "ansible_diff_mode": false,
        "ansible_facts": {},
        "ansible_forks": 5,
        "ansible_host": "10.50.130.13",
        "ansible_inventory_sources": [
            "/Users/kivlint/Documents/GitHub/vmware-automation/ansible/inventory"
        ],
        "interfaces": [
            {
                "cable": {
                    "id": 647,
                    "label": "",
                    "url": "http://netbox/api/dcim/cables/647/"
                },
                "connected_endpoint": {
                    "cable": 647,
                    "connection_status": {
                        "label": "Connected",
                        "value": true
                    },
                    "device": {
                        "display_name": "pos-1-leaf-1",
                        "id": 40,
                        "name": "pos-1-leaf-1",
                        "url": "http://netbox/api/dcim/devices/40/"
                    },
                    "id": 3270,
                    "name": "02/2",
                    "url": "http://netbox/api/dcim/interfaces/3270/"
                },
                "connected_endpoint_type": "dcim.interface",
                "connection_status": {
                    "label": "Connected",
                    "value": true
                },
                "count_ipaddresses": 0,
                "description": "",
                "device": {
                    "display_name": "CASPOSR1BDAT003",
                    "id": 252,
                    "name": "CASPOSR1BDAT003",
                    "url": "http://netbox/api/dcim/devices/252/"
                },
                "enabled": true,
                "id": 3434,
                "ip_addresses": [],
                "label": "",
                "lag": null,
                "mac_address": "20:67:7C:00:36:A0",
                "mgmt_only": false,
                "mode": null,
                "mtu": null,
                "name": "PCI1.1",
                "tagged_vlans": [],
                "tags": [
                    {
                        "color": "9e9e9e",
                        "id": 29,
                        "name": "maas-if-ens2f0",
                        "slug": "maas-if-ens2f0",
                        "url": "http://netbox/api/extras/tags/29/"
                    }
                ],
                "type": {
                    "label": "SFP+ (10GE)",
                    "value": "10gbase-x-sfpp"
                },
                "untagged_vlan": null,
                "url": "http://netbox/api/dcim/interfaces/3434/"
            },
            {
                "cable": {
                    "id": 748,
                    "label": "IL0059-C",
                    "url": "http://netbox/api/dcim/cables/748/"
                },
                "connected_endpoint": {
                    "cable": 748,
                    "connection_status": {
                        "label": "Connected",
                        "value": true
                    },
                    "device": {
                        "display_name": "pos-1-leaf-1",
                        "id": 40,
                        "name": "pos-1-leaf-1",
                        "url": "http://netbox/api/dcim/devices/40/"
                    },
                    "id": 3751,
                    "name": "04/2",
                    "url": "http://netbox/api/dcim/interfaces/3751/"
                },
                "connected_endpoint_type": "dcim.interface",
                "connection_status": {
                    "label": "Connected",
                    "value": true
                },
                "count_ipaddresses": 0,
                "description": "",
                "device": {
                    "display_name": "CASPOSR1BDAT003",
                    "id": 252,
                    "name": "CASPOSR1BDAT003",
                    "url": "http://netbox/api/dcim/devices/252/"
                },
                "enabled": true,
                "id": 3473,
                "ip_addresses": [],
                "label": "",
                "lag": null,
                "mac_address": null,
                "mgmt_only": false,
                "mode": null,
                "mtu": null,
                "name": "PCI1.2",
                "tagged_vlans": [],
                "tags": [
                    {
                        "color": "9e9e9e",
                        "id": 33,
                        "name": "maas-if-ens2f1",
                        "slug": "maas-if-ens2f1",
                        "url": "http://netbox/api/extras/tags/33/"
                    }
                ],
                "type": {
                    "label": "SFP+ (10GE)",
                    "value": "10gbase-x-sfpp"
                },
                "untagged_vlan": null,
                "url": "http://netbox/api/dcim/interfaces/3473/"
            }
        ]
    }

However when I try and query that JSON, I get the errors:

- debug:
    msg: "{{ hostvars[inventory_hostname] | to_nice_json | community.general.json_query(interfaces[?name=='PCI1.1'].mac_address) }}"

Even something simple doesn't work:

- debug:
    msg: "{{ hostvars[inventory_hostname] | to_nice_json | community.general.json_query(ansible_host) }}"

Expected Results

I expect to be presented with the value that the query results in, e.g.

[
  "aa:bb:cc:dd:ee:ff"
]

(This is what https://jmespath.org/ produces.)

Actual Results

If I don't use single quotes in the json_query, I receive the following error:

fatal: [CASPOSR1BDAT003]: FAILED! => 
  msg: |-
    Error in jmespath.search in json_query filter plugin:
    unhashable type: 'dict'

If I do use single quotes in the json_query, I just get a blank output (no failure).



### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct
ansibullbot commented 2 years ago

Files identified in the description:

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

tomkivlin commented 2 years ago

Feel free to close if this isn't an issue with json_query, I have found a different solution to my needs: https://stackoverflow.com/questions/69953172/in-ansible-how-to-query-hostvars-to-get-a-specific-value-of-a-key-from-a-list-i

felixfontein commented 2 years ago 
- debug:
    msg: "{{ hostvars[inventory_hostname] | to_nice_json | community.general.json_query(interfaces[?name=='PCI1.1'].mac_address) }}"
- debug:
    msg: "{{ hostvars[inventory_hostname] | to_nice_json | community.general.json_query(ansible_host) }}"

Please note that your queries are missing quotes. You need to quote what you pass to json_query, it expects a string.

tomkivlin commented 2 years ago

@felixfontein many thanks. So the correct syntax, respectively, should be:

- debug:
    msg: "{{ hostvars[inventory_hostname] | to_nice_json | community.general.json_query('interfaces[?name==`PCI1.1`].mac_address') }}"
- debug:
    msg: "{{ hostvars[inventory_hostname] | to_nice_json | community.general.json_query('ansible_host') }}"
felixfontein commented 2 years ago

@tomkivlin I think so. Otherwise it will definitely not work. (Whether it works with these I don't know, I don't use the json_query filter.)

flowerysong commented 2 years ago

That's definitely not the right syntax, since json_query operates on data structures, but you're passing in a JSON string. You at a minimum need to remove to_nice_json.

russoz commented 2 years ago

@tomkivlin could you please confirm if this is still an issue? TIA

tomkivlin commented 2 years ago

@russoz apologies for the delay. I can confirm that as @flowerysong suggests, I needed to remove to_nice_json and as @felixfontein suggested, I needed quotes. So the following both work:

- debug:
        msg: "{{ hostvars[inventory_hostname] | community.general.json_query('interfaces[?name==`PCI1.1`].mac_address') }}"
- debug:
        msg: "{{ hostvars[inventory_hostname] | community.general.json_query('ansible_host') }}"

Worth noting again, that for my use-case (getting info about the host the play is running against), there is a simpler way as described here (thanks again to @flowerysong, I assume): https://stackoverflow.com/a/69958617/10556530

russoz commented 2 years ago

Thanks @tomkivlin!! Glad that it worked for you! :-)