Closed ksbarnt closed 5 years ago
i think this is related to the string differences between 2.7 and 3.x python, we will address this at some point, try with py3.x and see if this issue persists
FMFA-266
The issue persists under Python 3
[user@it-auto-t-abl1 bugreport]$ ansible-playbook -vvvv -i demo_inventory.yml demo.yml --vault-password-file /home/user/vaultpass.txt --limit 'CMU_Production'
ansible-playbook 2.8.0
config file = None
configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3.6/site-packages/ansible
executable location = /usr/bin/ansible-playbook
python version = 3.6.8 (default, May 2 2019, 20:40:44) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
No config file found; using defaults
setting up inventory plugins
host_list declined parsing /home/user/bugreport/demo_inventory.yml as it did not pass it's verify_file() method
script declined parsing /home/user/bugreport/demo_inventory.yml as it did not pass it's verify_file() method
Parsed /home/user/bugreport/demo_inventory.yml inventory source with yaml plugin
[DEPRECATION WARNING]: bigip_facts is kept for backwards compatibility but usage is discouraged. The module documentation details page may explain more about this rationale.. This feature will be removed in a future release.
Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
Loading callback plugin default of type stdout, v2.0 from /usr/lib/python3.6/site-packages/ansible/plugins/callback/default.py
PLAYBOOK: demo.yml ******************************************************************************************************************************************************************************************************************
Positional arguments: demo.yml
verbosity: 4
connection: smart
timeout: 10
become_method: sudo
tags: ('all',)
inventory: ('/home/user/bugreport/demo_inventory.yml',)
subset: CMU_Production
vault_password_files: ('/home/user/vaultpass.txt',)
forks: 5
1 plays in demo.yml
PLAY [Demonstrate bigip_data_group issue] *******************************************************************************************************************************************************************************************
META: ran handlers
TASK [Create Address Data Group] ****************************************************************************************************************************************************************************************************
task path: /home/user/bugreport/demo.yml:23
Trying secret FileVaultSecret(filename='/home/user/vaultpass.txt') for vault_id=default
Trying secret FileVaultSecret(filename='/home/user/vaultpass.txt') for vault_id=default
Trying secret FileVaultSecret(filename='/home/user/vaultpass.txt') for vault_id=default
<bigip1> connection transport is rest
<bigip1> ESTABLISH LOCAL CONNECTION FOR USER: user
<bigip1> EXEC /bin/sh -c 'echo ~user && sleep 0'
<bigip1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/user/.ansible/tmp/ansible-tmp-1559834375.8247724-106768323629699 `" && echo ansible-tmp-1559834375.8247724-106768323629699="` echo /home/user/.ansible/tmp/ansible-tmp-1559834375.8247724-106768323629699 `" ) && sleep 0'
Using module file /usr/lib/python3.6/site-packages/ansible/modules/network/f5/bigip_data_group.py
<bigip1> PUT /home/user/.ansible/tmp/ansible-local-16570d9q090zp/tmpv177ih4s TO /home/user/.ansible/tmp/ansible-tmp-1559834375.8247724-106768323629699/AnsiballZ_bigip_data_group.py
<bigip1> EXEC /bin/sh -c 'chmod u+x /home/user/.ansible/tmp/ansible-tmp-1559834375.8247724-106768323629699/ /home/user/.ansible/tmp/ansible-tmp-1559834375.8247724-106768323629699/AnsiballZ_bigip_data_group.py && sleep 0'
<bigip1> EXEC /bin/sh -c 'F5_USER=admin F5_PASSWORD=methanol.stiletto.getaway.whorl /usr/bin/python /home/user/.ansible/tmp/ansible-tmp-1559834375.8247724-106768323629699/AnsiballZ_bigip_data_group.py && sleep 0'
<bigip1> EXEC /bin/sh -c 'rm -f -r /home/user/.ansible/tmp/ansible-tmp-1559834375.8247724-106768323629699/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
File "/home/user/.ansible/tmp/ansible-tmp-1559834375.8247724-106768323629699/AnsiballZ_bigip_data_group.py", line 114, in <module>
_ansiballz_main()
File "/home/user/.ansible/tmp/ansible-tmp-1559834375.8247724-106768323629699/AnsiballZ_bigip_data_group.py", line 106, in _ansiballz_main
invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
File "/home/user/.ansible/tmp/ansible-tmp-1559834375.8247724-106768323629699/AnsiballZ_bigip_data_group.py", line 49, in invoke_module
imp.load_module('__main__', mod, module, MOD_DESC)
File "/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py", line 1320, in <module>
File "/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py", line 1313, in main
File "/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py", line 1257, in exec_module
File "/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py", line 782, in exec_module
File "/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py", line 831, in present
File "/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py", line 969, in create
File "/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py", line 1095, in create_on_device
File "/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py", line 1028, in _upload_to_file
File "/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py", line 1021, in upload_file_to_device
File "/tmp/ansible_bigip_data_group_payload_yGCnGe/ansible_bigip_data_group_payload.zip/ansible/module_utils/network/f5/icontrol.py", line 402, in upload_file
TypeError: coercing to Unicode: need string or buffer, _io.StringIO found
fatal: [bigip1]: FAILED! => {
"changed": false,
"module_stderr": "Traceback (most recent call last):\n File \"/home/user/.ansible/tmp/ansible-tmp-1559834375.8247724-106768323629699/AnsiballZ_bigip_data_group.py\", line 114, in <module>\n _ansiballz_main()\n File \"/home/user/.ansible/tmp/ansible-tmp-1559834375.8247724-106768323629699/AnsiballZ_bigip_data_group.py\", line 106, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/home/user/.ansible/tmp/ansible-tmp-1559834375.8247724-106768323629699/AnsiballZ_bigip_data_group.py\", line 49, in invoke_module\n imp.load_module('__main__', mod, module, MOD_DESC)\n File \"/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py\", line 1320, in <module>\n File \"/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py\", line 1313, in main\n File \"/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py\", line 1257, in exec_module\n File \"/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py\", line 782, in exec_module\n File \"/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py\", line 831, in present\n File \"/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py\", line 969, in create\n File \"/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py\", line 1095, in create_on_device\n File \"/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py\", line 1028, in _upload_to_file\n File \"/tmp/ansible_bigip_data_group_payload_yGCnGe/__main__.py\", line 1021, in upload_file_to_device\n File \"/tmp/ansible_bigip_data_group_payload_yGCnGe/ansible_bigip_data_group_payload.zip/ansible/module_utils/network/f5/icontrol.py\", line 402, in upload_file\nTypeError: coercing to Unicode: need string or buffer, _io.StringIO found\n",
"module_stdout": "",
"msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
"rc": 1
}
PLAY RECAP **************************************************************************************************************************************************************************************************************************
bigip1 : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
@wojtek0806 This appears to also affect attempts to update existing external data groups, which I discovered trying to reproduce an issue we're seeing in Ansible 2.7 with bigip_data_group not actually updating the data group if the contents of the external file are different than what's on the BIG-IP boxes (i.e. if the Data Group file gets edited via the config manager) to see if it was fixed in 2.8 or if it was something we should report.
Any guesses on when this might be looked at?
change reverted as fix broke everything else
@wojtek0806 Hello , I am getting this error on ansible 2.8.4 . My playbook looks like this
tasks:
- name: Create a data group of IP addresses from a file
bigip_data_group:
name: portal-pool-wasp-8000_dg
records_src: "/home/devans/strings.txt"
type: string
provider: "{{ provider }}"
state: present
#external_file_name: "strings.txt"
separator: " := "
delegate_to: localhost
I am getting error as below
module_stderr: |-
Traceback (most recent call last):
File "/home/devans/.ansible/tmp/ansible-tmp-1567674839.02-94792716981243/AnsiballZ_bigip_data_group.py", line 114, in <module>
_ansiballz_main()
File "/home/devans/.ansible/tmp/ansible-tmp-1567674839.02-94792716981243/AnsiballZ_bigip_data_group.py", line 106, in _ansiballz_main
invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
File "/home/devans/.ansible/tmp/ansible-tmp-1567674839.02-94792716981243/AnsiballZ_bigip_data_group.py", line 49, in invoke_module
imp.load_module('__main__', mod, module, MOD_DESC)
File "/tmp/ansible_bigip_data_group_payload_eW_lis/__main__.py", line 1320, in <module>
File "/tmp/ansible_bigip_data_group_payload_eW_lis/__main__.py", line 1313, in main
File "/tmp/ansible_bigip_data_group_payload_eW_lis/__main__.py", line 1257, in exec_module
File "/tmp/ansible_bigip_data_group_payload_eW_lis/__main__.py", line 782, in exec_module
File "/tmp/ansible_bigip_data_group_payload_eW_lis/__main__.py", line 831, in present
File "/tmp/ansible_bigip_data_group_payload_eW_lis/__main__.py", line 969, in create
File "/tmp/ansible_bigip_data_group_payload_eW_lis/__main__.py", line 1095, in create_on_device
File "/tmp/ansible_bigip_data_group_payload_eW_lis/__main__.py", line 1028, in _upload_to_file
File "/tmp/ansible_bigip_data_group_payload_eW_lis/__main__.py", line 1021, in upload_file_to_device
File "/tmp/ansible_bigip_data_group_payload_eW_lis/ansible_bigip_data_group_payload.zip/ansible/module_utils/network/f5/icontrol.py", line 402, in upload_file
TypeError: coercing to Unicode: need string or buffer, _io.StringIO found
module_stdout: ''
msg: |-
MODULE FAILURE
See stdout/stderr for the exact error
rc: 1
My file looks like below
cat "/home/devans/strings.txt"
"client1" := "pool1",
"client2" := "pool2",
"client3" := "pool3"
What i am missing ?
you are not using the ansible version which has the fix, 2.9 (to be released in few weeks) or current galaxy role has this fix already
@wojtek0806 Thanks for that info. With role I am getting below error
The full traceback is:
WARNING: The below traceback may *not* be related to the actual failure.
File "/tmp/ansible_bigip_data_group_payload_1vQ3KU/__main__.py", line 291, in <module>
from library.module_utils.network.f5.bigip import F5RestClient
fatal: [server.example.com -> localhost]: FAILED! => changed=false
invocation:
module_args:
delete_data_group_file: null
description: null
external_file_name: null
internal: false
name: portal-pool-wasp-8000_dg
partition: Common
provider:
auth_provider: null
password: VALUE_SPECIFIED_IN_NO_LOG_PARAMETER
proxy_to: null
server: server.example.com
server_port: 443
ssh_keyfile: null
timeout: null
transport: rest
user: f5-ansible
validate_certs: false
records: null
records_src: /home/devans/strings.txt
separator: ' := '
state: present
type: string
msg: 'Unsupported parameters for (bigip_data_group) module: proxy_to found in provider. Supported parameters include: auth_provider, password, server, server_port, ssh_keyfile, timeout, transport, user, validate_certs'
I am using roles first time so not sure if I have to do anything extra ...
this is my playbook looks like
---
- name: setting-node-state-to-disable
hosts: bigips
connection: local
gather_facts: no
vars:
provider:
password: "{{ password }}"
server: server.example.com
user: "{{ username }}"
validate_certs: no
server_port: 443
roles:
- /home/devans/.ansible/roles/f5devcentral.f5ansible
tasks:
- name: Create a data group of IP addresses from a file
bigip_data_group:
name: portal-pool-wasp-8000_dg
records_src: "/home/devans/strings.txt"
type: string
provider: "{{ provider }}"
state: present
#external_file_name: "strings.txt"
separator: " := "
# no_proxy: "{{ inventory_hostname }}"
delegate_to: localhost
Sorry to drag this , if you confirm this is different issue then , I will raise a new one for further discussion.
this is expected cause we had to remove some legacy parameters for 2.9 release, so the role had been updated, you could install current ansible devel version or the 2.9 when the first release candidate is released:
https://docs.ansible.com/ansible/latest/roadmap/ROADMAP_2_9.html
After 2.9 hits we will have moved away from this role to a collection which would be much easier to work with, but for now you need to do the above i am afraid.
ISSUE TYPE
COMPONENT NAME
bigip_data_group
ANSIBLE VERSION
PYTHON VERSION
BIGIP VERSION
ROLE VERSION
CONFIGURATION
Default configuration.
OS / ENVIRONMENT
CentOS Linux release 7.6.1810 (Core)
SUMMARY
As of an upgrade to Ansible 2.8, calls to bigip_data_group to create a new external data group fail with an error stating TypeError: coercing to Unicode: need string or buffer, _io.StringIO found. These same calls succeeded under Ansible 2.7. We generally use the built-in modules rather than the galaxy role/modules, but have tested this both ways with the same result. This is the same sort of behavior shown in Issue 1256, but in our case we're explicitly trying to create external groups.
STEPS TO REPRODUCE
Playbook: demo.yml
Data Group 1: address-dg
Data Group 2: string-dg
EXPECTED RESULTS
I'd expect two External Data Groups to be created, one address, one string.
ACTUAL RESULTS
Module fails with an error.