bigip_command modules does not execute commands via CLI (SSH), always trying via REST

[slavkv]

ISSUE TYPE

• Bug Report

COMPONENT NAME

bigip_command (tried library comes with Ansible and downloaded from your repository)

ANSIBLE VERSION

ansible 2.4.0.0

PYTHON VERSION

Python 2.7.14

BIGIP VERSION

Main Package
  Product  BIG-IP
  Version  11.5.1
  Build    8.0.175

CONFIGURATION

OS / ENVIRONMENT

OSX 10.12.6

SUMMARY

I am trying to run several commands on F5 via SSH. It worked with Ansible 2.2 (we communicated about this some time ago), but does not work with Ansible 2.4

STEPS TO REPRODUCE

Run the following playbook

- name: Collect all configured accounts
  bigip_command:
    server: "{{ device_hostname }}"
    user: "{{ auth_user }}"
    password: "{{ auth_pass }}"
    validate_certs: "false"
    commands:
      - list auth user
    transport: 'cli'
  delegate_to: localhost
  register: result

- debug: var=result

EXPECTED RESULTS

A list of configured users. When I login as 'admin' using SSH and run 'list auth user', I get a list of all configured users.

ACTUAL RESULTS

Module fails saying it is not able to do REST.

I understand that 11.5 may not work with the module REST calls, but module should invoke SSH connection as transport specify 'cli', and it should not do any HTTPS REST calls.

What I also noticed, that there is such code in the module

try:
from ansible.module_utils.f5_utils import run_commands
    HAS_CLI_TRANSPORT = True
except ImportError:
    HAS_CLI_TRANSPORT = False

It can not import 'run_commands' as such does not exists in f5 utils. Can it cause the issue?

The full traceback is:
  File "/var/folders/4k/lxknkpkn0wg4bmmjwrmb5b8mk_gp_p/T/ansible_3cG1Wd/ansible_modlib.zip/ansible/module_utils/f5_utils.py", line 237, in __init__
    f5_product_name, **self._connect_params
  File "/var/folders/4k/lxknkpkn0wg4bmmjwrmb5b8mk_gp_p/T/ansible_3cG1Wd/ansible_modlib.zip/ansible/module_utils/f5_utils.py", line 262, in _get_mgmt_root
    token='tmos'
  File "/usr/local/Cellar/ansible/2.4.0.0/libexec/lib/python2.7/site-packages/f5/bigip/__init__.py", line 124, in __init__
    self.post_configuration_setup()
  File "/usr/local/Cellar/ansible/2.4.0.0/libexec/lib/python2.7/site-packages/f5/bigip/__init__.py", line 91, in post_configuration_setup
    self._get_tmos_version()
  File "/usr/local/Cellar/ansible/2.4.0.0/libexec/lib/python2.7/site-packages/f5/bigip/__init__.py", line 96, in _get_tmos_version
    response = connect.get(base_uri)
  File "/usr/local/Cellar/ansible/2.4.0.0/libexec/lib/python2.7/site-packages/icontrol/session.py", line 258, in wrapper
    response = method(self, REST_uri, **kwargs)
  File "/usr/local/Cellar/ansible/2.4.0.0/libexec/lib/python2.7/site-packages/icontrol/session.py", line 449, in get
    return self.session.get(uri, **kwargs)
  File "/usr/local/Cellar/ansible/2.4.0.0/libexec/lib/python2.7/site-packages/requests/sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/Cellar/ansible/2.4.0.0/libexec/lib/python2.7/site-packages/requests/sessions.py", line 494, in request
    prep = self.prepare_request(req)
  File "/usr/local/Cellar/ansible/2.4.0.0/libexec/lib/python2.7/site-packages/requests/sessions.py", line 437, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "/usr/local/Cellar/ansible/2.4.0.0/libexec/lib/python2.7/site-packages/requests/models.py", line 309, in prepare
    self.prepare_auth(auth, url)
  File "/usr/local/Cellar/ansible/2.4.0.0/libexec/lib/python2.7/site-packages/requests/models.py", line 540, in prepare_auth
    r = auth(self)
  File "/usr/local/Cellar/ansible/2.4.0.0/libexec/lib/python2.7/site-packages/icontrol/authtoken.py", line 207, in __call__
    self.get_new_token(netloc)
  File "/usr/local/Cellar/ansible/2.4.0.0/libexec/lib/python2.7/site-packages/icontrol/authtoken.py", line 113, in get_new_token
    response=response)

fatal: [f5host.local -> localhost]: FAILED! => {
    "changed": false,
    "failed": true,
    "invocation": {
        "module_args": {
            "commands": [
                "list auth user"
            ],
            "interval": 1,
            "match": "all",
            "partition": "Common",
            "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "retries": 10,
            "server": "f5host.local",
            "server_port": 443,
            "state": "present",
            "transport": "cli",
            "user": "admin",
            "validate_certs": false,
            "wait_for": null
        }
    },
    "msg": "404 Unexpected Error: Not Found for uri: https://f5host.local:443/mgmt/shared/authn/login\nText: u'{\"code\":404,\"message\":\"http://localhost:8100/mgmt/shared/authn/login\",\"restOperationId\":56757345,\"errorStack\":[\"com.f5.rest.common.RestWorkerUriNotFoundException: http://localhost:8100/mgmt/shared/authn/login\",\"at com.f5.rest.common.RestServer.trySendInProcess(RestServer.java:231)\",\"at com.f5.rest.common.RestRequestReceiver.dispatchToService(RestRequestReceiver.java:93)\",\"at com.f5.rest.common.RestRequestReceiver.processNext(RestRequestReceiver.java:57)\",\"at com.f5.rest.common.RestHelper$2.run(RestHelper.java:1910)\",\"at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\",\"at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\",\"at java.lang.Thread.run(Thread.java:722)\\\\n\"]}'"

[caphrim007]

@slavkv did you also download the latest module_utils code? that's where we added the run_commands stuff you see

[slavkv]

@caphrim007 I have downloaded whole Library directory from your git repository, but there is no reference about 'run_commands' there.

$ grep run_commands -R ./library/
./library//bigip_command.py:    from ansible.module_utils.f5_utils import run_commands
./library//bigip_command.py:                responses = run_commands(self.client.module, self.want.commands)

As you can see, command module trying to import it, but there is no 'def run_commands' anywhere.

Where exactly do I need to download run_commands from? Do I need to copy it to Ansible module_utils dir?

Thanks!

[caphrim007]

@slavkv oops, I meant the one that is upstream (which will be released with 2.5)

https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/f5_utils.py

The module_utils that you're finding in the f5-ansible repository is not a working product right now; only a playground for me to use in case I need to refactor something in the future.

[slavkv]

@caphrim007 I download this file and put into Ansible path. I can import run_commands now, but it's not working still. Does it suppose to work with Ansible 2.4 or I need to install Ansible 2.5 for that?

The full traceback is:
Traceback (most recent call last):
  File "/var/folders/4k/lxknkpkn0wg4bmmjwrmb5b8mk_gp_p/T/ansible_vqZOBz/ansible_module_bigip_command.py", line 403, in <module>
    main()
  File "/var/folders/4k/lxknkpkn0wg4bmmjwrmb5b8mk_gp_p/T/ansible_vqZOBz/ansible_module_bigip_command.py", line 396, in main
    results = mm.exec_module()
  File "/var/folders/4k/lxknkpkn0wg4bmmjwrmb5b8mk_gp_p/T/ansible_vqZOBz/ansible_module_bigip_command.py", line 251, in exec_module
    changed = self.execute()
  File "/var/folders/4k/lxknkpkn0wg4bmmjwrmb5b8mk_gp_p/T/ansible_vqZOBz/ansible_module_bigip_command.py", line 275, in execute
    responses = run_commands(self.client.module, self.want.commands)
  File "/var/folders/4k/lxknkpkn0wg4bmmjwrmb5b8mk_gp_p/T/ansible_vqZOBz/ansible_modlib.zip/ansible/module_utils/f5_utils.py", line 152, in run_commands
  File "/var/folders/4k/lxknkpkn0wg4bmmjwrmb5b8mk_gp_p/T/ansible_vqZOBz/ansible_modlib.zip/ansible/module_utils/connection.py", line 66, in exec_command
  File "/usr/local/opt/python/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
TypeError: argument must be string or read-only character buffer, not None

fatal: [f5host.local -> localhost]: FAILED! => {
    "changed": false,
    "failed": true,
    "module_stderr": "Traceback (most recent call last):\n  File \"/var/folders/4k/lxknkpkn0wg4bmmjwrmb5b8mk_gp_p/T/ansible_vqZOBz/ansible_module_bigip_command.py\", line 403, in <module>\n    main()\n  File \"/var/folders/4k/lxknkpkn0wg4bmmjwrmb5b8mk_gp_p/T/ansible_vqZOBz/ansible_module_bigip_command.py\", line 396, in main\n    results = mm.exec_module()\n  File \"/var/folders/4k/lxknkpkn0wg4bmmjwrmb5b8mk_gp_p/T/ansible_vqZOBz/ansible_module_bigip_command.py\", line 251, in exec_module\n    changed = self.execute()\n  File \"/var/folders/4k/lxknkpkn0wg4bmmjwrmb5b8mk_gp_p/T/ansible_vqZOBz/ansible_module_bigip_command.py\", line 275, in execute\n    responses = run_commands(self.client.module, self.want.commands)\n  File \"/var/folders/4k/lxknkpkn0wg4bmmjwrmb5b8mk_gp_p/T/ansible_vqZOBz/ansible_modlib.zip/ansible/module_utils/f5_utils.py\", line 152, in run_commands\n  File \"/var/folders/4k/lxknkpkn0wg4bmmjwrmb5b8mk_gp_p/T/ansible_vqZOBz/ansible_modlib.zip/ansible/module_utils/connection.py\", line 66, in exec_command\n  File \"/usr/local/opt/python/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py\", line 228, in meth\n    return getattr(self._sock,name)(*args)\nTypeError: argument must be string or read-only character buffer, not None\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 0
}

[caphrim007]

@slavkv I have not tried it with 2.4. It was only recently added by an external contributor as part of the 2.5 release. Do you have a development environment that you can try a 2.5 devel installation to see if that is the case?

[caphrim007]

@jmighion any thoughts on this?

[slavkv]

@caphrim007 I am testing with 2.4 in staging because 2.4 is officially stable release now, and then I am going to roll it in production. I will try to use 2.5 just to check it works later, but my main goal is to make it work with 2.4.

What minimal F5 version do I need to run to make sure F5 modules work fine? 11.5 does REST and service is running, but looks like it's very old and not working with modules.

[caphrim007]

it should not matter the version of BIG-IP as it should just be normal SSH. so, assuming your SSH works as you expect.

Are you logging in as the root user? using the bash shell? or tmsh shell?

[slavkv]

Which F5 version I need for modules to work with REST?

ADMIN user with tmsh shell. It works with Ansible 2.2.

And yes, ssh works fine from my host.

[caphrim007]

for using REST i recommend 12.0.0 or greater. We don't test on anything earlier than that. If it works on earlier versions, then you got lucky

[slavkv]

@caphrim007 cheers.

[jmighion]

CLI transport only works in 2.5. The error TypeError: argument must be string or read-only character buffer, not None was because it's missing plugins. If you're trying to backport it all into 2.4, there's a few more files to copy and even then you'd have to make some manual changes to those. The PR for cli transport in bigip_command https://github.com/ansible/ansible/pull/30391

[caphrim007]

ahh yes the action plugins. good call @jmighion i forgot about that.

[jmighion]

I would not recommend backporting it unless you are comfortable making all the changes in that PR and the subsequent PRs for bigip_command.

[caphrim007]

@slavkv if you must have these features in your 2.4 installation, there are a number of additional dependencies that @jmighion mentioned in his PR there including an action plugin, terminal plugin and additions to your NETWORK_GROUP_MODULES that needs to happen. This is not something we will backport into 2.4.

[slavkv]

@caphrim007 Thanks. I will wait for 2.5 then.