bigip_device_info fails to collect gtm-servers if there are objects with space in name

pbystrov commented 1 year ago

COMPONENT NAME

bigip_device_info, gtm-servers subset

Environment

ANSIBLE VERSION

ansible [core 2.11.11]
  config file = /home/pbystrov-a/.ansible.cfg
  configured module search path = ['/home/pbystrov-a/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/pbystrov-a/venv/f5/lib/python3.7/site-packages/ansible
  ansible collection location = /home/pbystrov-a/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/pbystrov-a/venv/f5/bin/ansible
  python version = 3.7.3 (default, Oct 31 2022, 14:04:00) [GCC 8.3.0]
  jinja version = 2.11.3
  libyaml = True

BIGIP VERSION

Sys::Version
Main Package
  Product     BIG-IP
  Version     14.1.4
  Build       0.0.11
  Edition     Final
  Date        Thu Feb 11 19:05:03 PST 2021

CONFIGURATION

None

OS / ENVIRONMENT

Distributor ID: Debian
Description:    Debian GNU/Linux 10 (buster)
Release:        10
Codename:       buster

SUMMARY

bigip_device_info gtm-servers subset gathering fails if there are objects with spaces in name. This is not working since version 1.14.0

STEPS TO REPRODUCE

- name: Reproduce bigip_device_info gtm-server subset gathering issue
  hosts: localhost
  gather_facts: no
  vars:
    provider:
      server: "{{ lookup('env', 'F5_GTM_DEVICE') }}"
      transport: rest
      user: "{{ lookup('env', 'F5_GTM_USER') }}"
      password: "{{ lookup('env', 'F5_GTM_PASSWORD') }}"
      validate_certs: no

  tasks:
    - name: Gather gtm-servers subset
      delegate_to: localhost
      f5networks.f5_modules.bigip_device_info:
        provider: "{{ provider }}"
        gather_subset:
          - gtm-servers
      register: _bigip_gtm_servers_facts

EXPECTED RESULTS

List of gtm-server objects saved to _bigip_gtm_servers_facts variable

ACTUAL RESULTS

ansible-playbook [core 2.11.11]
  config file = /home/pbystrov-a/.ansible.cfg
  configured module search path = ['/home/pbystrov-a/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/pbystrov-a/venv/f5/lib/python3.7/site-packages/ansible
  ansible collection location = /home/pbystrov-a/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/pbystrov-a/venv/f5/bin/ansible-playbook
  python version = 3.7.3 (default, Oct 31 2022, 14:04:00) [GCC 8.3.0]
  jinja version = 2.11.3
  libyaml = True
Using /home/pbystrov-a/.ansible.cfg as config file
setting up inventory plugins
host_list declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
script declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
auto declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Parsed /etc/ansible/hosts inventory source with ini plugin
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
Loading collection f5networks.f5_modules from /home/pbystrov-a/.ansible/collections/ansible_collections/f5networks/f5_modules
Loading callback plugin default of type stdout, v2.0 from /home/pbystrov-a/venv/f5/lib/python3.7/site-packages/ansible/plugins/callback/default.py
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: bigip-gtm-bug-2.yml *******************************************************************************************************
Positional arguments: playbooks/bigip-gtm-bug-2.yml
verbosity: 4
connection: smart
timeout: 10
become_method: sudo
tags: ('all',)
inventory: ('/etc/ansible/hosts',)
forks: 5
1 plays in playbooks/bigip-gtm-bug-2.yml

PLAY [Reproduce bigip_device_info gtm-server subset gathering issue] ****************************************************************
META: ran handlers

TASK [Gather gtm-servers subset] ****************************************************************************************************
task path: /home/pbystrov-a/code/playbooks/bigip-gtm-bug-2.yml:14
<localhost> Using network group action f5networks.f5_modules.bigip for f5networks.f5_modules.bigip_device_info
Loading collection ansible.netcommon from /home/pbystrov-a/.ansible/collections/ansible_collections/ansible/netcommon
<localhost> connection transport is rest
<localhost> ANSIBLE_NETWORK_IMPORT_MODULES: disabled
<localhost> ANSIBLE_NETWORK_IMPORT_MODULES: module execution time may be extended
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: pbystrov-a
<localhost> EXEC /bin/sh -c 'echo ~pbystrov-a && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/pbystrov-a/.ansible/tmp `"&& mkdir "` echo /home/pbystrov-a/.ansible/tmp/ansible-tmp-1671204752.8187318-46883-18349925898396 `" && echo ansible-tmp-1671204752.8187318-46883-18349925898396="` echo /home/pbystrov-a/.ansible/tmp/ansible-tmp-1671204752.8187318-46883-18349925898396 `" ) && sleep 0'
Using module file /home/pbystrov-a/.ansible/collections/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py
<localhost> PUT /home/pbystrov-a/.ansible/tmp/ansible-local-46877colub61n/tmpgg7n0izp TO /home/pbystrov-a/.ansible/tmp/ansible-tmp-1671204752.8187318-46883-18349925898396/AnsiballZ_bigip_device_info.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/pbystrov-a/.ansible/tmp/ansible-tmp-1671204752.8187318-46883-18349925898396/ /home/pbystrov-a/.ansible/tmp/ansible-tmp-1671204752.8187318-46883-18349925898396/AnsiballZ_bigip_device_info.py && sleep 0'
<localhost> EXEC /bin/sh -c '/home/pbystrov-a/venv/f5/bin/python /home/pbystrov-a/.ansible/tmp/ansible-tmp-1671204752.8187318-46883-18349925898396/AnsiballZ_bigip_device_info.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/pbystrov-a/.ansible/tmp/ansible-tmp-1671204752.8187318-46883-18349925898396/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
  File "/tmp/ansible_f5networks.f5_modules.bigip_device_info_payload_7dm06txs/ansible_f5networks.f5_modules.bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 18352, in main
  File "/tmp/ansible_f5networks.f5_modules.bigip_device_info_payload_7dm06txs/ansible_f5networks.f5_modules.bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 18029, in exec_module
  File "/tmp/ansible_f5networks.f5_modules.bigip_device_info_payload_7dm06txs/ansible_f5networks.f5_modules.bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 18136, in execute_managers
  File "/tmp/ansible_f5networks.f5_modules.bigip_device_info_payload_7dm06txs/ansible_f5networks.f5_modules.bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 10910, in exec_module
  File "/tmp/ansible_f5networks.f5_modules.bigip_device_info_payload_7dm06txs/ansible_f5networks.f5_modules.bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 10920, in _exec_module
  File "/tmp/ansible_f5networks.f5_modules.bigip_device_info_payload_7dm06txs/ansible_f5networks.f5_modules.bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 7631, in to_return
  File "/tmp/ansible_f5networks.f5_modules.bigip_device_info_payload_7dm06txs/ansible_f5networks.f5_modules.bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 10840, in virtual_servers
  File "/tmp/ansible_f5networks.f5_modules.bigip_device_info_payload_7dm06txs/ansible_f5networks.f5_modules.bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 10740, in _process_vs_stats
  File "/tmp/ansible_f5networks.f5_modules.bigip_device_info_payload_7dm06txs/ansible_f5networks.f5_modules.bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 10731, in _read_virtual_stats_from_device
fatal: [localhost -> localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "gather_subset": [
                "gtm-servers"
            ],
            "partition": "Common",
            "provider": {
                "auth_provider": null,
                "no_f5_teem": null,
                "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "server": "-REDACTED-",
                "server_port": null,
                "timeout": null,
                "transport": "rest",
                "user": "-REDACTED-",
                "validate_certs": false
            }
        }
    },
    "msg": "b'{\"code\":404,\"message\":\"Object not found - host_domain_com - CA_HTTPS\",\"errorStack\":[],\"apiError\":1}'"
}

PLAY RECAP **************************************************************************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

KrithikaChidambaram commented 1 year ago

Hi, thanks for reporting. Internal tracking ID for this issue is: INFRAANO-1045, thanks!

urohit011 commented 1 year ago

Hi @pbystrov , which object has space in the name so I can test the device_info module with that. I tried to reproduce the issue by fetching info for a gtm server with name like "foo gtm server" and it worked fine without any error.

pbystrov commented 1 month ago

Hi @urohit011, sorry for delay in my reply 😖 Since my initial report we have updated our BIG-IP DNS (GTM) device as well as execution environment, but still encounter this error.

# list gtm server "host.domain.com - CA"
gtm server "host.domain.com - CA" {
    addresses {
        11.22.33.44 {
            device-name "host.domain.com - CA"
        }
    }
    datacenter VX
    devices {
        "host.domain.com - CA" {
            addresses {
                11.22.33.44 { }
            }
        }
    }
    monitor gateway_icmp
    product generic-host
    virtual-servers {
        "host_domain_com - CA_HTTPS" {
            destination 11.22.33.44:https
        }
    }
}