search

F5Networks / f5-ansible

Imperative Ansible modules for F5 BIG-IP products
GNU General Public License v3.0
376 stars 231 forks source link

X-F5-Auth-Token has expired on versions greater than 1.13.0 #2410

Open jtudini91 opened 4 months ago

jtudini91 commented 4 months ago
COMPONENT NAME

f5networks.f5_modules.bigip_device_info

Environment

Dev/QA/Prod

ANSIBLE VERSION
ansible [core 2.13.13]
  python version = 3.9.18 (main, Sep 22 2023, 17:58:34) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)]
  jinja version = 3.1.3
  libyaml = True
BIGIP VERSION
Sys::Version
Main Package
  Product     BIG-IP
  Version     17.1.1.2
  Build       0.15.10
  Edition     Engineering Hotfix
  Date        Mon Mar 18 09:11:08 PDT 2024

Hotfix List
ID1160805-4   ID1297257-1

Also fails on the following versions

Sys::Version
Main Package
  Product     BIG-IP
  Version     15.1.5.1
  Build       0.0.14
  Edition     Point Release 1
  Date        Sun Mar 27 23:23:42 PDT 2022
CONFIGURATION

N/A

OS / ENVIRONMENT

N/A

SUMMARY

Trying to run any module on 1.14.0 or higher results in the following error fatal: [bigip.example.com -> localhost]: FAILED! => {"changed": false, "msg": "b'{\"code\":401,\"message\":\"X-F5-Auth-Token has expired.\",\"referer\":\"10.x.x.x\",\"restOperationId\":14981760,\"kind\":\":resterrorresponse\"}'"}

STEPS TO REPRODUCE

Running the playbook to collect the device info

- name: Standard GTM Retirement
  hosts: all
  gather_facts: false
  collections:
    - f5networks.f5_modules
  vars:
    bigip_app_facts: "{{ lookup('file', 'f5_output.json') | from_json }}"

  tasks:
    - name: Collect details from BIG-IP
      bigip_device_info:
        gather_subset:
          - gtm-servers
          # - all
          # - monitors
          # - profiles
          # - client-ssl-profiles
          # - ssl-certs
          # - ssl-keys
          # - ltm-pools
          # - virtual-servers
          # - nodes
          # - irules
          # - server-ssl-profiles
        provider: "{{ provider }}"
      register: bigip_app_facts
      delegate_to: localhost

    - name: Copy
      ansible.builtin.copy:
        content: "{{ bigip_app_facts | to_nice_json }}"
        dest: f5_output.json
      delegate_to: localhost
EXPECTED RESULTS

I expect to see output with virtual servers

ACTUAL RESULTS
ansible-playbook [core 2.13.13]
  config file = /home/user/F5-Orchestration/ansible.cfg
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/user/.local/lib/python3.9/site-packages/ansible
  ansible collection location = /home/user/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/user/.local/bin/ansible-playbook
  python version = 3.9.18 (main, Sep 22 2023, 17:58:34) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)]
  jinja version = 3.1.3
  libyaml = True
Using /home/user/F5-Orchestration/ansible.cfg as config file
setting up inventory plugins
host_list declined parsing /home/user/F5-Orchestration/tmp_inv/inv.yml as it did not pass its verify_file() method
script declined parsing /home/user/F5-Orchestration/tmp_inv/inv.yml as it did not pass its verify_file() method
Parsed /home/user/F5-Orchestration/tmp_inv/inv.yml inventory source with yaml plugin
Loading collection f5networks.f5_modules from /home/user/.ansible/collections/ansible_collections/f5networks/f5_modules
[WARNING]: Collection f5networks.f5_modules does not support Ansible version 2.13.13
Loading callback plugin default of type stdout, v2.0 from /home/user/.local/lib/python3.9/site-packages/ansible/plugins/callback/default.py
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: tmp.yml **********************************************************************************************************************************************************************************************************************************************************************************************************************
Positional arguments: playbooks/gtm_delete/tmp.yml
verbosity: 4
connection: smart
timeout: 10
become_method: sudo
tags: ('all',)
inventory: ('/home/user/F5-Orchestration/tmp_inv/inv.yml',)
forks: 5
1 plays in playbooks/gtm_delete/tmp.yml

PLAY [Standard GTM Retirement] *********************************************************************************************************************************************************************************************************************************************************************************************************
META: ran handlers
<localhost> Using network group action bigip for bigip_device_info
Loading collection ansible.netcommon from /home/user/.ansible/collections/ansible_collections/ansible/netcommon
<localhost> connection transport is rest
<localhost> ANSIBLE_NETWORK_IMPORT_MODULES: disabled
<localhost> ANSIBLE_NETWORK_IMPORT_MODULES: module execution time may be extended
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: user
<localhost> EXEC /bin/sh -c 'echo ~user&& sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/user/.ansible/tmp `"&& mkdir "` echo /home/user/.ansible/tmp/ansible-tmp-1718129676.2228312-565734-59202318164845 `" && echo ansible-tmp-1718129676.2228312-565734-59202318164845="` echo /home/user/.ansible/tmp/ansible-tmp-1718129676.2228312-565734-59202318164845 `" ) && sleep 0'
Using module file /home/user/.ansible/collections/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py
<localhost> PUT /home/user/.ansible/tmp/ansible-local-565730uhoo__ds/tmpcjajkflm TO /home/user/.ansible/tmp/ansible-tmp-1718129676.2228312-565734-59202318164845/AnsiballZ_bigip_device_info.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/user/.ansible/tmp/ansible-tmp-1718129676.2228312-565734-59202318164845/ /home/user/.ansible/tmp/ansible-tmp-1718129676.2228312-565734-59202318164845/AnsiballZ_bigip_device_info.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /home/user/.ansible/tmp/ansible-tmp-1718129676.2228312-565734-59202318164845/AnsiballZ_bigip_device_info.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/user/.ansible/tmp/ansible-tmp-1718129676.2228312-565734-59202318164845/ > /dev/null 2>&1 && sleep 0'

TASK [Collect details from BIG-IP] *****************************************************************************************************************************************************************************************************************************************************************************************************
task path: /home/user/F5-Orchestration/playbooks/gtm_delete/tmp.yml:17
The full traceback is:
  File "/tmp/ansible_bigip_device_info_payload_t3205qgi/ansible_bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 18659, in main
  File "/tmp/ansible_bigip_device_info_payload_t3205qgi/ansible_bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 18329, in exec_module
  File "/tmp/ansible_bigip_device_info_payload_t3205qgi/ansible_bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 18437, in execute_managers
  File "/tmp/ansible_bigip_device_info_payload_t3205qgi/ansible_bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 11007, in exec_module
  File "/tmp/ansible_bigip_device_info_payload_t3205qgi/ansible_bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 11015, in _exec_module
  File "/tmp/ansible_bigip_device_info_payload_t3205qgi/ansible_bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 11024, in read_facts
  File "/tmp/ansible_bigip_device_info_payload_t3205qgi/ansible_bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 11034, in increment_read
  File "/tmp/ansible_bigip_device_info_payload_t3205qgi/ansible_bigip_device_info_payload.zip/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_device_info.py", line 11056, in read_collection_from_device
fatal: [bigip.example.com-> localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "data_increment": 10,
            "gather_subset": [
                "gtm-servers"
            ],
            "partition": "Common",
            "provider": {
                "auth_provider": "tmos",
                "no_f5_teem": null,
                "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "server": "bigip.example.com",
                "server_port": 443,
                "timeout": 15,
                "transport": "rest",
                "user": "admin",
                "validate_certs": false
            }
        }
    },
    "msg": "b'{\"code\":401,\"message\":\"X-F5-Auth-Token has expired.\",\"referer\":\"10.x.x.x\",\"restOperationId\":14999388,\"kind\":\":resterrorresponse\"}'"
}

PLAY RECAP *****************************************************************************************************************************************************************************************************************************************************************************************************************************
bigip.example.com          : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0
pgouband commented 3 months ago

Hi @jtudini91

I tested with version 1.28 without any issue on BIG-IP version 16.1 and 17.1. The issue is maybe related to your environment.

jtudini91 commented 1 month ago

Hello @pgouband,

So this issue seems to be on devices with a larger configuration. It seems that paging functionality added from v13 -> 14 causes the issues. I see that the data_increment parameter was added in. Is there any recommended value settings for the size of the configuration or hardware?

pgouband commented 1 month ago

Hi @jtudini91,

This setting is useful for setups with large configurations which may take a long time with the default values. While there is no limit to the value that can be specified, note that putting very large values with gather_subset set to meta choices like all might lead to module or device API crash. F5 recommends using data_increment custom values in tandem with partition and a specific gather_subset value for best experience.

rgonzal1 commented 1 month ago

Hey @pgouband,

I've been working with @jtudini91 on this issue. It appears this this issue results from our large configuration due to the stats for each virtual server being pulled during the gathering of the gtm-servers.

The API endpoint /mgmt/tm/gtm/server?expandSubcollections response is ~ 30 seconds.

With our large configuration, there are a large number of virtual servers that are having their /mgmt/tm/gtm/server/<server>/virtual-servers/<virtual server>/stats endpoint queried independently following the completion of the /mgmt/tm/gtm/server?expandSubcollections endpoint.

Is it possible to have stats and the server configuration separated out so that only the gtm-server configuration is pulled when gtm-servers is requested? This would greatly improve the speed of the module and prevent the Auth Timeout issue that we have been experiencing.

jtudini91 commented 3 days ago

Hello @pgouband,

I just wanted to see if you have had anytime to checkout what @rgonzal1 posted and shared.