with_path cannot be yes

[usscarter]

SUMMARY

When attempting to set with_path: yes, you get the following error:

[WARNING]: * Failed to parse /home/user/ansiblevmware/inventories/production/filename.vmware.yml with ansible_collections.community.vmware.plugins.inventory.vmware_vm_inventory plugin: Invalid empty/false group name provided:

ISSUE TYPE

• Bug Report

COMPONENT NAME

vmware_vm_inventory.py

ANSIBLE VERSION

/usr/lib/python2.7/dist-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.
  from cryptography.exceptions import InvalidSignature
ansible 2.9.14
  config file = /home/User/ansiblevmware/ansible.cfg
  configured module search path = [u'/home/User/ansiblevmware/library']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.17 (default, Jul 20 2020, 15:37:01) [GCC 7.5.0]

CONFIGURATION

/usr/lib/python2.7/dist-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.
  from cryptography.exceptions import InvalidSignature
CACHE_PLUGIN_CONNECTION(/home/user/ansiblevmware/ansible.cfg) = ~/.cache/
DEFAULT_CALLBACK_WHITELIST(/home/user/ansiblevmware/ansible.cfg) = [u'profile_tasks', u'timer']
DEFAULT_FILTER_PLUGIN_PATH(/home/User/ansiblevmware/ansible.cfg) = [u'/home/User/ansiblevmware/filter_plugins']
DEFAULT_HOST_LIST(/home/User/ansiblevmware/ansible.cfg) = [u'/home/User/ansiblevmware/inventories/production']
DEFAULT_MODULE_PATH(/home/User/ansiblevmware/ansible.cfg) = [u'/home/User/ansiblevmware/library']
DEFAULT_MODULE_UTILS_PATH(/home/User/ansiblevmware/ansible.cfg) = [u'/home/User/ansiblevmware/module_utils']
DEFAULT_ROLES_PATH(/home/User/ansiblevmware/ansible.cfg) = [u'/home/User/.ansible/roles', u'/usr/share/ansible/roles', u'/etc/ansible/roles', u'/home/User/ansiblevmware/roles']
DEFAULT_STDOUT_CALLBACK(/home/User/ansiblevmware/ansible.cfg) = yaml
DEFAULT_VAULT_PASSWORD_FILE(/home/User/ansiblevmware/ansible.cfg) = /opt/playbooks/.vaultpwd
HOST_KEY_CHECKING(/home/User/ansiblevmware/ansible.cfg) = False
INTERPRETER_PYTHON(/home/User/ansiblevmware/ansible.cfg) = auto
INVENTORY_ENABLED(/home/User/ansiblevmware/ansible.cfg) = [u'community.vmware.vmware_vm_inventory']
SYSTEM_WARNINGS(/home/User/ansiblevmware/ansible.cfg) = False

OS / ENVIRONMENT

Ubuntu 18.04.5 LTS

STEPS TO REPRODUCE

plugin: community.vmware.vmware_vm_inventory
strict: False
hostname: 
username: 
password:
validate_certs: False
with_tags: yes
with_path: yes
hostnames:
- config.name
groups:
  kwd: True
properties:
    - 'name'
    - 'config.instanceUuid'
    - 'config.hardware.numCPU'
    - 'config.template'
    - 'config.name'
    - 'config.uuid'
    - 'guest.hostName'
    - 'guest.ipAddress'
    - 'guest.guestId'
    - 'guest.guestState'
    - 'summary.runtime.powerState'
    - 'config.guestId'
    - 'guest.toolsVersionStatus'
compose:
  ansible_host: guest.ipAddress
  folder: path.split('/')[-1].lower()
  subnet: (guest.ipAddress + '/24') | ipv4('network')
keyed_groups:
  - key: folder
    prefix: kwd
filters:
- runtime.powerState == "poweredOn"
- '"Z-VRA" not in config.name'

EXPECTED RESULTS

Expect an inventory return

ACTUAL RESULTS

[WARNING]: * Failed to parse /home/User/ansiblevmware/inventories/production/file.vmware.yml with ansible_collections.community.vmware.plugins.inventory.vmware_vm_inventory plugin: Invalid empty/false group name provided:

ansible-inventory --graph

[Akasurde]

@usscarter Thanks for reporting this. I am not able to reproduce this at my setup

...
        }
    },
    "all": {
        "children": [
            "Datacenters",
            "kwd",
            "kwd_discovered_virtual_machine",
            "kwd_prod",
            "ungrouped"
        ]
    },
    "kwd": {
        "hosts": [
            "N\u00e4gemist-s.example.com",
            "VMware-vR-Appliance-7.6.0.317-13027280_OVF10",
            "caddyshack",
            "centos_7",
            "kickstart_vm",
            "xd"
        ]
    },
    "kwd_discovered_virtual_machine": {
        "hosts": [
            "N\u00e4gemist-s.example.com",
            "VMware-vR-Appliance-7.6.0.317-13027280_OVF10",
            "caddyshack"
        ]
    },
    "kwd_prod": {
        "hosts": [
            "centos_7",
            "kickstart_vm",
            "xd"
        ]
    }
}

Since you are using paths, can you check if there is some weird path or Unicode value in the path variable?

this line prints invalid group name - https://github.com/ansible/ansible/blob/cf0cd4b50bbabc5b2f8c36c63fc7b064c94f2040/lib/ansible/inventory/data.py#L173

Also, can you enable strict: True in the above inventory configuration? This will give more information about the error.

[usscarter]

So when running it with Strict: True I am left with a few things.

The first error that pops up is:

[WARNING]:  * Failed to parse /home/User/ansiblevmware/inventories/production/file.vmware.yml with
ansible_collections.community.vmware.plugins.inventory.vmware_vm_inventory plugin: Could not evaluate runtime.powerState == "poweredOn" as host filters - 'runtime' is undefined

If I comment out the filters to get past that error, I get:

[WARNING]:  * Failed to parse /home/User/ansiblevmware/inventories/production/file.vmware.yml with
ansible_collections.community.vmware.plugins.inventory.vmware_vm_inventory plugin: Could not set subnet for host VM-NAME: 'dict object' has no attribute 'ipAddress'

That VM is powered off, so I don't know what it is trying to do exactly there.

[usscarter]

I fiddled with it some more, and it seems like what i may be disliking is VAPP's that seem to exist in the vcenters I had tried until that point. I targeted a vCenter with no VAPP's and it worked fine.

[usscarter]

I also am notcing an issue when I have my config like this, where the groups are comnig out with a prefix of "_" IE: '_security_dmz' when its creating groups.

Could it be something going on with the path.split?

compose:
  ansible_host: guest.ipAddress
  folder: path.split('/')[-1].lower()

keyed_groups:
  - key: folder
    seperator: ''

[ansibullbot]

cc @GBrawl @Tomorrow9 @alongchamps @bedecarroll @castawayegr @chrrrles @dav1x @davidmhewitt @digifuchsi @dx0xm @equelin @garbled1 @goneri @gyorgypeter @imjoseangel @jjahns @kamsz @karstenjakobsen @lparkes @mariolenz @nafpliot-ibm @nerzhul @ngp @oboukili @pdellaert @pgbidkar @ritzk @rmin @scottd018 @sky-joker @stravassac @sumkincpp @victron @vmwjoseph @warthog9 @woshihaoren click here for bot help

[jdranchman]

Basically the same problem here. When with_path is no, it works. With it on it fails. I'm also using

TRANSFORM_INVALID_GROUP_CHARS(/opt/ansible/ansible.cfg) = silently

Changing that to always doesn't print any errors. With debug on:

ansible-inventory [core 2.11.3]
  config file = /opt/ansible/ansible.cfg
  configured module search path = ['/opt/ansible/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/dist-packages/ansible
  ansible collection location = /opt/ansible/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible-inventory
  python version = 3.8.10 (default, Jun  2 2021, 10:49:15) [GCC 9.4.0]
  jinja version = 2.10.1
  libyaml = True
Using /opt/ansible/ansible.cfg as config file
setting up inventory plugins
Loading collection community.vmware from /opt/ansible/.ansible/collections/ansible_collections/community/vmware

[WARNING]:  * Failed to parse /opt/ansible/.vmware.yml with
ansible_collections.community.vmware.plugins.inventory.vmware_vm_inventory
plugin: Invalid empty/false group name provided:
  File "/usr/local/lib/python3.8/dist-packages/ansible/inventory/manager.py", line 290, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/opt/ansible/.ansible/collections/ansible_collections/community/vmware/plugins/inventory/vmware_vm_inventory.py", line 748, in parse
    cacheable_results = self._populate_from_source()
  File "/opt/ansible/.ansible/collections/ansible_collections/community/vmware/plugins/inventory/vmware_vm_inventory.py", line 860, in _populate_from_source
    self._populate_host_properties(host_properties, host)
  File "/opt/ansible/.ansible/collections/ansible_collections/community/vmware/plugins/inventory/vmware_vm_inventory.py", line 929, in _populate_host_properties
    c_group = self.inventory.add_group(c_name)
  File "/usr/local/lib/python3.8/dist-packages/ansible/inventory/data.py", line 176, in add_group
    raise AnsibleError("Invalid empty/false group name provided: %s" % group)

plugin: community.vmware.vmware_vm_inventory
strict: yes
hostname: 172.31.110.20
username: dcgov\sa_ansible_RO
password: 4#t@4Cwe8Gn7
validate_certs:  no
with_tags: yes
with_path: no
hostnames:
  - config.name | lower
keyed_groups:
  - key: summary.runtime.powerState
    separator: ''
properties:
  - 'all'
filter:
  - 'runtime.powerState == "poweredOn"'
  - '"Template" not in config.name'`

Note: Running the "old" vmware-vm-inventory.py script on the same vcenter returns the paths just fine. Example for one guest:

      "parent": {
          "name": "Lab",
          "parent": {
            "name": "NonProduction",
            "parent": {
              "name": "Unix"
            }
          }
        }

With spaces in our vmware folder names I get - as expected:

       "parent": {
          "name": "Database_Oracle",
          "parent": {
            "name": "Production",
            "parent": {
              "name": "Unix"
            }
          }

We have templates (that are not powered on) that also give the no ipAddress error as mentioned earlier in this thread. With or without the filter.

[jdranchman]

REedit...

Pulling in all properties and setting with_path gives me this:

ansible_collections.community.vmware.plugins.inventory.vmware_vm_inventory
plugin: Invalid empty/false group name provided:
  File "/usr/local/lib/python3.8/dist-packages/ansible/inventory/manager.py", line 290, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/opt/ansible/.ansible/collections/ansible_collections/community/vmware/plugins/inventory/vmware_vm_inventory.py", line 748, in parse
    cacheable_results = self._populate_from_source()
  File "/opt/ansible/.ansible/collections/ansible_collections/community/vmware/plugins/inventory/vmware_vm_inventory.py", line 860, in _populate_from_source
    self._populate_host_properties(host_properties, host)
  File "/opt/ansible/.ansible/collections/ansible_collections/community/vmware/plugins/inventory/vmware_vm_inventory.py", line 926, in _populate_host_properties
    c_group = self.inventory.add_group(c_name)
  File "/usr/local/lib/python3.8/dist-packages/ansible/inventory/data.py", line 176, in add_group
    raise AnsibleError("Invalid empty/false group name provided: %s" % group)

Obviously there is a folder in my vcenter that data.py doesn't like but it doesn't print what it might be. There were no "complaints" by the to_safe_group_name method when running with -vvvv

I have this in my ansible.cfg for this specific test: force_valid_group_names=always

[nextnely]

I am also experiencing the same issue. For me, it happens when a vApp is present in vCenter, and with_path is set to yes.

[Zappelphilipp]

issue still exists. has somebody found a workaround or fix for that?

EDIT: For us the problem looks a bit different but also only appears by using with_path:

# big block of vm data of tanzu vm
type: DirectoryOrCreate\\n          name: ca-certs\\n---\\napiVersion:
apiregistration.k8s.io/v1\\nkind: APIService\\nmetadata:\\n  labels:\\n    k8s-app:
metrics-server\\n  name: v1beta1.metrics.k8s.io\\nspec:\\n  group:
metrics.k8s.io\\n  groupPriorityMinimum: 100\\n  insecureSkipTLSVerify: false\\n
service:\\n    name: metrics-server\\n    namespace: kube-system\\n  version:
v1beta1\\n  versionPriority: 100\\n  caBundle: \\"{{ .MetricsServerCA }}\\"\\n"}.
unexpected '.'
  File "/usr/local/lib/python3.9/site-packages/ansible/inventory/manager.py", line 293, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/usr/local/lib/python3.9/site-packages/ansible/plugins/inventory/auto.py", line 59, in parse
    plugin.parse(inventory, loader, path, cache=cache)
  File "/usr/share/ansible/collections/ansible_collections/community/vmware/plugins/inventory/vmware_vm_inventory.py", line 675, in parse
    cacheable_results = self._populate_from_source()
  File "/usr/share/ansible/collections/ansible_collections/community/vmware/plugins/inventory/vmware_vm_inventory.py", line 786, in _populate_from_source
    host = self._get_hostname(host_properties, hostnames, strict=strict)
  File "/usr/share/ansible/collections/ansible_collections/community/vmware/plugins/inventory/vmware_vm_inventory.py", line 811, in _get_hostname
    raise AnsibleError(
[WARNING]: Unable to parse /runner/inventory/vmware_vm_inventory.yml as an
inventory source
ERROR! No inventory was parsed, please check your configuration and options.

[DorBreger]

Suffer from this issue as well, any help?