search

F5Networks / f5-ansible

Imperative Ansible modules for F5 BIG-IP products
GNU General Public License v3.0
376 stars 231 forks source link

bigip_command - 400 Unexpected Error on UCS creation #1207

Closed writemike closed 5 years ago

writemike commented 5 years ago
ISSUE TYPE
COMPONENT NAME

bigip_command

ANSIBLE VERSION
ansible 2.7.6
  config file = /home/mholland/ansible.cfg
  configured module search path = [u'/home/mholland/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /bin/ansible
  python version = 2.7.5 (default, Oct 30 2018, 23:45:53) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
PYTHON VERSION
Python 2.7.5
BIGIP VERSION
Sys::Version
Main Package
  Product     BIG-IP
  Version     14.1.0
  Build       0.0.116
  Edition     Final
  Date        Wed Nov 14 18:41:56 PST 2018
CONFIGURATION
cat ansible.cfg
[defaults]
inventory      = ./hosts

cat ./hosts
[bigip]
10.200.1.10

[bigiq]
10.200.1.128
OS / ENVIRONMENT

Ansible is running from CentOS 7 Docker Instance and BIG-IP is a VE running in VMware Fusion Professional Version 11.0.2 (10952296).

SUMMARY

Intermittently responding with 400 Unexpected Error to "tmsh save sys ucs /var/tmp/tmp_test.ucs" when executed from bigip_command module. The UCS file is always successfully created on the BIG-IP, even when we see the error message.

[root@bigip-14-1:Active:Standalone] tmp # pwd
/var/tmp
[root@bigip-14-1:Active:Standalone] tmp # ls -al *.ucs
-rw-r--r--. 1 root root 86587205 Feb  6 11:06 tmp_test.ucs
[root@bigip-14-1:Active:Standalone] tmp #

[root@bigip-14-1:Active:Standalone] tmp # time tmsh save sys ucs /var/tmp/tmp_test.ucs
Saving active configuration...
/var/tmp/tmp_test.ucs is saved.

real    1m16.768s
user    0m18.503s
sys 0m37.522s
STEPS TO REPRODUCE

Run playbook below.

---
- name: Test bigip_command Module
  hosts: bigip
  connection: local
  gather_facts: no
  vars:
    provider:
      password: admin
      server: 10.200.1.10
      user: admin
      validate_certs: no
      server_port: 443
      timeout: 60

  tasks:
    - name: Delete the ucs file
      bigip_command:
        provider: "{{provider}}"
        commands:
          - tmsh delete sys ucs /var/tmp/tmp_test.ucs
      delegate_to: localhost

    - name: Create the ucs file
      bigip_command:
        provider: "{{provider}}"
        wait_for: result[0] contains saved
        commands:
          - tmsh save sys ucs /var/tmp/tmp_test.ucs
      delegate_to: localhost:
EXPECTED RESULTS
ansible-playbook bigip_command-test.yaml -vvv
ansible-playbook 2.7.6
  config file = /home/mholland/ansible.cfg
  configured module search path = [u'/home/mholland/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /bin/ansible-playbook
  python version = 2.7.5 (default, Oct 30 2018, 23:45:53) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
Using /home/mholland/ansible.cfg as config file
/home/mholland/hosts did not meet host_list requirements, check plugin documentation if this is unexpected
/home/mholland/hosts did not meet script requirements, check plugin documentation if this is unexpected
Parsed /home/mholland/hosts inventory source with ini plugin

PLAYBOOK: bigip_command-test.yaml ***************************************************************************************************************************************************************************************
1 plays in bigip_command-test.yaml

PLAY [Test bigip_command Module] **************************************************************************************************************************************************************************************
META: ran handlers

TASK [Delete the ucs file] **********************************************************************************************************************************************************************************************
task path: /home/mholland/bigip_command-test.yaml:16
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: mholland
<localhost> EXEC /bin/sh -c 'echo ~mholland && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/mholland/.ansible/tmp/ansible-tmp-1549411042.87-139269419657526 `" && echo ansible-tmp-1549411042.87-139269419657526="` echo /home/mholland/.ansible/tmp/ansible-tmp-1549411042.87-139269419657526 `" ) && sleep 0'
Using module file /usr/lib/python2.7/site-packages/ansible/modules/network/f5/bigip_command.py
<localhost> PUT /home/mholland/.ansible/tmp/ansible-local-200WZfEec/tmpyCVBo9 TO /home/mholland/.ansible/tmp/ansible-tmp-1549411042.87-139269419657526/AnsiballZ_bigip_command.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/mholland/.ansible/tmp/ansible-tmp-1549411042.87-139269419657526/ /home/mholland/.ansible/tmp/ansible-tmp-1549411042.87-139269419657526/AnsiballZ_bigip_command.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python2 /home/mholland/.ansible/tmp/ansible-tmp-1549411042.87-139269419657526/AnsiballZ_bigip_command.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/mholland/.ansible/tmp/ansible-tmp-1549411042.87-139269419657526/ > /dev/null 2>&1 && sleep 0'
 [WARNING]: Using "write" commands is not idempotent. You should use a module that is specifically made for that. If such a module does not exist, then please file a bug. The command in question is "delete sys ucs
/var/tmp/tmp_test.ucs..."

changed: [10.200.1.10 -> localhost] => {
    "changed": true,
    "executed_commands": [
        "tmsh -c \\\"delete sys ucs /var/tmp/tmp_test.ucs\\\""
    ],
    "invocation": {
        "module_args": {
            "auth_provider": null,
            "chdir": null,
            "commands": [
                "tmsh delete sys ucs /var/tmp/tmp_test.ucs"
            ],
            "interval": 1,
            "match": "all",
            "password": null,
            "provider": {
                "auth_provider": null,
                "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "server": "10.200.1.10",
                "server_port": 443,
                "ssh_keyfile": null,
                "timeout": 60,
                "transport": "rest",
                "user": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "validate_certs": false
            },
            "retries": 10,
            "server": null,
            "server_port": null,
            "transport": "rest",
            "user": null,
            "validate_certs": null,
            "wait_for": null,
            "warn": true
        }
    },
    "stdout": [
        "/var/tmp/tmp_test.ucs is deleted"
    ],
    "stdout_lines": [
        [
            "/var/tmp/tmp_test.ucs is deleted"
        ]
    ]
}

TASK [Create the ucs file] **********************************************************************************************************************************************************************************************
task path: /home/mholland/bigip_command-test.yaml:23
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: mholland
<localhost> EXEC /bin/sh -c 'echo ~mholland && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/mholland/.ansible/tmp/ansible-tmp-1549411044.23-234119228990792 `" && echo ansible-tmp-1549411044.23-234119228990792="` echo /home/mholland/.ansible/tmp/ansible-tmp-1549411044.23-234119228990792 `" ) && sleep 0'
Using module file /usr/lib/python2.7/site-packages/ansible/modules/network/f5/bigip_command.py
<localhost> PUT /home/mholland/.ansible/tmp/ansible-local-200WZfEec/tmphP4_rg TO /home/mholland/.ansible/tmp/ansible-tmp-1549411044.23-234119228990792/AnsiballZ_bigip_command.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/mholland/.ansible/tmp/ansible-tmp-1549411044.23-234119228990792/ /home/mholland/.ansible/tmp/ansible-tmp-1549411044.23-234119228990792/AnsiballZ_bigip_command.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python2 /home/mholland/.ansible/tmp/ansible-tmp-1549411044.23-234119228990792/AnsiballZ_bigip_command.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/mholland/.ansible/tmp/ansible-tmp-1549411044.23-234119228990792/ > /dev/null 2>&1 && sleep 0'
 [WARNING]: Using "write" commands is not idempotent. You should use a module that is specifically made for that. If such a module does not exist, then please file a bug. The command in question is "save sys ucs
/var/tmp/tmp_test.ucs..."

ok: [10.200.1.10 -> localhost] => {
    "changed": false,
    "executed_commands": [
        "tmsh -c \\\"save sys ucs /var/tmp/tmp_test.ucs\\\""
    ],
    "invocation": {
        "module_args": {
            "auth_provider": null,
            "chdir": null,
            "commands": [
                "tmsh save sys ucs /var/tmp/tmp_test.ucs"
            ],
            "interval": 1,
            "match": "all",
            "password": null,
            "provider": {
                "auth_provider": null,
                "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "server": "10.200.1.10",
                "server_port": 443,
                "ssh_keyfile": null,
                "timeout": 60,
                "transport": "rest",
                "user": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "validate_certs": false
            },
            "retries": 1,
            "server": null,
            "server_port": null,
            "transport": "rest",
            "user": null,
            "validate_certs": null,
            "wait_for": [
                "result[0] contains saved"
            ],
            "warn": true
        }
    },
    "stdout": [
        "Saving active configuration...\n/var/tmp/tmp_test.ucs is saved."
    ],
    "stdout_lines": [
        [
            "Saving active configuration...",
            "/var/tmp/tmp_test.ucs is saved."
        ]
    ]
}
META: ran handlers
META: ran handlers

PLAY RECAP **************************************************************************************************************************************************************************************************************
10.200.1.10                : ok=2    changed=1    unreachable=0    failed=0
ACTUAL RESULTS

Both tasks always complete successfully. The delete UCS task always responds without any errors, but the create UCS task sends the 400 error most of the time. The UCS file is always successfully created on the BIG-IP in /var/tmp/. Any workarounds in the task to not get the error (interval, retries, wait_for)? I have tested these optional parameters and saw no change in response.

$ ansible-playbook bigip_command-test.yaml -vvv
ansible-playbook 2.7.6
  config file = /home/mholland/ansible.cfg
  configured module search path = [u'/home/mholland/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /bin/ansible-playbook
  python version = 2.7.5 (default, Oct 30 2018, 23:45:53) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
Using /home/mholland/ansible.cfg as config file
/home/mholland/hosts did not meet host_list requirements, check plugin documentation if this is unexpected
/home/mholland/hosts did not meet script requirements, check plugin documentation if this is unexpected
Parsed /home/mholland/hosts inventory source with ini plugin

PLAYBOOK: bigip_command-test.yaml ***************************************************************************************************************************************************************************************
1 plays in bigip_command-test.yaml

PLAY [Test bigip_command Module] **************************************************************************************************************************************************************************************
META: ran handlers

TASK [Delete the ucs file] **********************************************************************************************************************************************************************************************
task path: /home/mholland/bigip_command-test.yaml:16
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: mholland
<localhost> EXEC /bin/sh -c 'echo ~mholland && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/mholland/.ansible/tmp/ansible-tmp-1549411155.11-182929480933295 `" && echo ansible-tmp-1549411155.11-182929480933295="` echo /home/mholland/.ansible/tmp/ansible-tmp-1549411155.11-182929480933295 `" ) && sleep 0'
Using module file /usr/lib/python2.7/site-packages/ansible/modules/network/f5/bigip_command.py
<localhost> PUT /home/mholland/.ansible/tmp/ansible-local-258T7tmrU/tmpAiF39X TO /home/mholland/.ansible/tmp/ansible-tmp-1549411155.11-182929480933295/AnsiballZ_bigip_command.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/mholland/.ansible/tmp/ansible-tmp-1549411155.11-182929480933295/ /home/mholland/.ansible/tmp/ansible-tmp-1549411155.11-182929480933295/AnsiballZ_bigip_command.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python2 /home/mholland/.ansible/tmp/ansible-tmp-1549411155.11-182929480933295/AnsiballZ_bigip_command.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/mholland/.ansible/tmp/ansible-tmp-1549411155.11-182929480933295/ > /dev/null 2>&1 && sleep 0'
 [WARNING]: Using "write" commands is not idempotent. You should use a module that is specifically made for that. If such a module does not exist, then please file a bug. The command in question is "delete sys ucs
/var/tmp/tmp_test.ucs..."

changed: [10.200.1.10 -> localhost] => {
    "changed": true,
    "executed_commands": [
        "tmsh -c \\\"delete sys ucs /var/tmp/tmp_test.ucs\\\""
    ],
    "invocation": {
        "module_args": {
            "auth_provider": null,
            "chdir": null,
            "commands": [
                "tmsh delete sys ucs /var/tmp/tmp_test.ucs"
            ],
            "interval": 1,
            "match": "all",
            "password": null,
            "provider": {
                "auth_provider": null,
                "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "server": "10.200.1.10",
                "server_port": 443,
                "ssh_keyfile": null,
                "timeout": 60,
                "transport": "rest",
                "user": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "validate_certs": false
            },
            "retries": 10,
            "server": null,
            "server_port": null,
            "transport": "rest",
            "user": null,
            "validate_certs": null,
            "wait_for": null,
            "warn": true
        }
    },
    "stdout": [
        "/var/tmp/tmp_test.ucs is deleted"
    ],
    "stdout_lines": [
        [
            "/var/tmp/tmp_test.ucs is deleted"
        ]
    ]
}

TASK [Create the ucs file] **********************************************************************************************************************************************************************************************
task path: /home/mholland/bigip_command-test.yaml:23
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: mholland
<localhost> EXEC /bin/sh -c 'echo ~mholland && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/mholland/.ansible/tmp/ansible-tmp-1549411156.5-24245637598403 `" && echo ansible-tmp-1549411156.5-24245637598403="` echo /home/mholland/.ansible/tmp/ansible-tmp-1549411156.5-24245637598403 `" ) && sleep 0'
Using module file /usr/lib/python2.7/site-packages/ansible/modules/network/f5/bigip_command.py
<localhost> PUT /home/mholland/.ansible/tmp/ansible-local-258T7tmrU/tmpYSyG0x TO /home/mholland/.ansible/tmp/ansible-tmp-1549411156.5-24245637598403/AnsiballZ_bigip_command.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/mholland/.ansible/tmp/ansible-tmp-1549411156.5-24245637598403/ /home/mholland/.ansible/tmp/ansible-tmp-1549411156.5-24245637598403/AnsiballZ_bigip_command.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python2 /home/mholland/.ansible/tmp/ansible-tmp-1549411156.5-24245637598403/AnsiballZ_bigip_command.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/mholland/.ansible/tmp/ansible-tmp-1549411156.5-24245637598403/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
WARNING: The below traceback may *not* be related to the actual failure.
  File "/tmp/ansible_bigip_command_payload_CePnUD/__main__.py", line 723, in main
    results = mm.exec_module()
  File "/tmp/ansible_bigip_command_payload_CePnUD/__main__.py", line 660, in exec_module
    result = manager.exec_module()
---
  File "/tmp/ansible_bigip_command_payload_CePnUD/__main__.py", line 445, in exec_module
    raise F5ModuleError(str(e))

fatal: [10.200.1.10 -> localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "auth_provider": null,
            "chdir": null,
            "commands": [
                "tmsh save sys ucs /var/tmp/tmp_test.ucs"
            ],
            "interval": 1,
            "match": "all",
            "password": null,
            "provider": {
                "auth_provider": null,
                "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "server": "10.200.1.10",
                "server_port": 443,
                "ssh_keyfile": null,
                "timeout": 60,
                "transport": "rest",
                "user": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "validate_certs": false
            },
            "retries": 1,
            "server": null,
            "server_port": null,
            "transport": "rest",
            "user": null,
            "validate_certs": null,
            "wait_for": [
                "result[0] contains saved"
            ],
            "warn": true
        }
    },
    "msg": "400 Unexpected Error: Bad Request for uri: https://10.200.1.10:443/mgmt/tm/util/bash/\nText: u'{\"code\":400,\"message\":\"remoteSender:10.200.1.1, method:POST \",\"originalRequestBody\":\"{\\\\\"utilCmdArgs\\\\\": \\\\\"-c \\\\\\\\\\\\\"tmsh -c \\\\\\\\\\\\\\\\\\\\\\\\\\\\\"save sys ucs /var/tmp/tmp_test.ucs\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\"\\\\\", \\\\\"command\\\\\": \\\\\"run\\\\\"}\",\"referer\":\"10.200.1.1\",\"restOperationId\":6821211,\"kind\":\":resterrorresponse\"}'"
}
    to retry, use: --limit @/home/mholland/bigip_command-test.retry

PLAY RECAP **************************************************************************************************************************************************************************************************************
10.200.1.10                : ok=1    changed=1    unreachable=0    failed=1
wojtek0806 commented 5 years ago

@writemike

None of the modules have been tested on 14.x, looking at the error output:

"utilCmdArgs\\\\\": \\\\\"-c \\\\\\\\\\\\\"tmsh -c \\\\\\\\\\\\\\\\\\\\\\\\\\\\\"save sys ucs /var/tmp/tmp_test.ucs

Can you check in the audit logs, what is the exact command being run, cause I do not think having this: utilCmdArgs: "-c "tmsh -c save sys ucs"is what it should look like.

I guess I could look at maybe having an option to just SAVE the ucs file/not download added to bigip_ucs_fetch module, as we have endpoints to save/load UCS in the REST api. I need to think about it.

writemike commented 5 years ago

We saw the same result on the customer's BIG-IP running 12.1.4. 14.1 is just what they have running in their lab for repro. Here are the logs: Audit log:

Feb  7 16:03:45 bigip-14-1.lab.local notice tmsh[13672]: 01420002:5: AUDIT - pid=13672 user=admin folder=/Common module=(tmos)# status=[Command OK] cmd_data=save sys ucs /var/tmp/tmp_test.ucs
Feb  7 16:03:45 bigip-14-1.lab.local notice icrd_child[28513]: 01420002:5: AUDIT - pid=28513 user=admin folder=/Common module=(tmos)# status=[Command OK] cmd_data=run util bash -c "tmsh -c \"save sys ucs /var/tmp/tmp_test.ucs\""

restjavad.0.log:

[WARNING][1356][06 Feb 2019 03:04:24 UTC][8100/mgmt ForwarderPassThroughWorker] URI:http://localhost:8100/mgmt/tm/util/bash/, Referrer:10.200.1.1, Method:POST, Exception:java.util.concurrent.TimeoutException: remoteSender:10.200.1.1, method:POST
        at com.f5.rest.common.RestWorker.logAndFailExpiredOperation(RestWorker.java:3113)
        at com.f5.rest.common.RestWorker.checkForExpiredOperations(RestWorker.java:3102)
        at com.f5.rest.common.RestServer.checkAndExpirePendingWorkerOperations(RestServer.java:1263)
        at com.f5.rest.common.RestServer.access$200(RestServer.java:44)
        at com.f5.rest.common.RestServer$3.run(RestServer.java:1236)
        at com.f5.rest.common.ScheduleTaskManager$2$1.run(ScheduleTaskManager.java:123)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
        at java.lang.Thread.run(Thread.java:748)

[WARNING][1357][06 Feb 2019 03:04:24 UTC][8100/tm/auth/user PublicWorker] URI:http://localhost:8100/tm/util/bash/, Referrer:10.200.1.1, Method:POST, Exception:java.util.concurrent.TimeoutException: remoteSender:10.200.1.1, method:POST
        at com.f5.rest.common.RestWorker.logAndFailExpiredOperation(RestWorker.java:3113)
        at com.f5.rest.common.RestWorker.checkForExpiredOperations(RestWorker.java:3102)
        at com.f5.rest.common.RestServer.checkAndExpirePendingWorkerOperations(RestServer.java:1263)
        at com.f5.rest.common.RestServer.access$200(RestServer.java:44)
        at com.f5.rest.common.RestServer$3.run(RestServer.java:1236)
        at com.f5.rest.common.ScheduleTaskManager$2$1.run(ScheduleTaskManager.java:123)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
        at java.lang.Thread.run(Thread.java:748)

[SEVERE][1359][06 Feb 2019 03:04:24 UTC][ChildWrapper] Timed out waiting for POST request at http://localhost:8100/tm/util/bash/. Killing child process 31560
[I][1360][06 Feb 2019 03:04:24 UTC][ChildWrapper] Killing IcrdChild process 31560
[I][1362][06 Feb 2019 03:04:24 UTC][ProcessManager] Enabling transaction for child 12701332-d920-4fca-9d19-7a23b74486f6.

Here is the equivalent curl test. Maybe it is an iControl REST issue?

curl -X POST \
  https://10.200.1.10/mgmt/tm/util/bash/ \
  -H 'Content-Type: application/json' \
  -H 'X-F5-Auth-Token: WDLVYSJ5OXRZPM2OXFULLJYL4R' \
  -H 'cache-control: no-cache' \
  -d '{
    "command": "run",
    "utilCmdArgs": "-c \"tmsh -c \\\"save sys ucs /var/tmp/tmp_test.ucs\\\"\""
}'

and result:

400 Bad Request
Time:70649 ms
Size:893 B
{
    "code": 400,
    "message": "remoteSender:10.200.1.1, method:POST ",
    "originalRequestBody": "{\n    \"command\": \"run\",\n    \"utilCmdArgs\": \"-c \\\"tmsh -c \\\\\\\"save sys ucs /var/tmp/tmp_test.ucs\\\\\\\"\\\"\"\n}",
    "referer": "10.200.1.1",
    "restOperationId": 7985444,
    "kind": ":resterrorresponse"
}
caphrim007 commented 5 years ago

This is happening because of the hard coded timeout in the REST API server code.

This problem is handled for you, or should be, in the bigip_ucs_fetch module.

This cannot be fixed in the command module because it's a core bug of the rest API. Even in the ucs_fetch module, we work around it by ignoring the timeout error and ps'ing the processes on box until the UCS process disappears.

Suggest you try the bigip_ucs_fetch module

writemike commented 5 years ago

Thanks for the update! We tried the bigip_ucs_fetch module originally, but it was running really slow until we tried the fix for "bigip_ucs_fetch hangs #1152" which has now solved that problem. We will go back to using the bigip_ucs_fetch module.