search

F5Networks / f5-ansible-bigip

Declarative Ansible collection for managing F5 BIG-IP/BIG-IQ.
36 stars 17 forks source link

Cannot create SSLO HTTP service proxy_type as Transparent with existing self IP #82

Open imohmmad opened 8 months ago

imohmmad commented 8 months ago

COMPONENT NAME bigip_sslo_service_http

Environment ANSIBLE VERSION ansible [core 2.16.2] config file = /python-env/POD-ONBOARDING/ansible.cfg configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /python-env/ansible-9.1.0/lib/python3.11/site-packages/ansible ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections executable location = /python-env/ansible-9.1.0/bin/ansible python version = 3.11.4 (main, Dec 7 2023, 15:43:41) [GCC 12.3.0] (/python-env/ansible-9.1.0/bin/python3) jinja version = 3.1.3 libyaml = True

BIGIP VERSION Sys::Version Main Package Product BIG-IP Version 17.1.1.1 Build 0.68.2

CONFIGURATION

OS / ENVIRONMENT Ubuntu 20.04.3

SUMMARY unable to create SSLO http service with proxy_type transparent while utilizing existing self_ip.

STEPS TO REPRODUCE Create the following SSLO http service

EXPECTED RESULTS

should be able to create http service with proxy type as transparent

ACTUAL RESULTS

"msg": "CREATE operation error: a04ed767-cdec-4516-859e-8b1a8f001985 : [OrchestratorConfigProcessor] Deployment failed for Error: [BaseHAConfigProcessor (ServiceBaseHAConfigProcessor)] TypeError: Cannot read property 'lastIndexOf' of undefined" }

pgouband commented 7 months ago

Hi,

Thanks for reporting. Added to the backlog and internal tracking ID for this request is: INFRAANO-1535.

megamattzilla commented 6 months ago

Hello!

Is there an ETA for this fix?

Thanks!

megamattzilla commented 3 months ago

I retested today on latest ansible version and bigip modules and I can still recreate the error:

Error:

TASK [Create Transparent HTTP service] **************************************************************************************************
fatal: [192.168.1.102]: FAILED! => {"changed": false, "msg": "CREATE operation error: bc5ffa65-faa6-4085-b2d4-e4ade90c3367 : [OrchestratorConfigProcessor] Deployment failed for Error: [BaseHAConfigProcessor (ServiceBaseHAConfigProcessor)] TypeError: Cannot read property 'lastIndexOf' of undefined"}

Playbook

- name: Create HTTP service
  hosts: "sslo17"
  connection: httpapi
  gather_facts: false
  any_errors_fatal: true

  tasks:

      - name: Test Module
        f5networks.f5_bigip.bigip_sslo_service_http:
          name: "proxy1a"
          state: "absent"

      - name: Create Transparent HTTP service
        f5networks.f5_bigip.bigip_sslo_service_http:
          name: "HTTP-TRANSPARENT"
          proxy_type: "transparent"
          service_down_action: "ignore"
          auto_manage: false
          use_exist_selfip: true
          snat: "none"
          ip_family: "ipv4"
          devices_to:
            vlan: "/Common/Internal1"
            self_ip: 10.10.1.1
            netmask: 255.255.255.0
          devices_from:
            vlan: "/Common/Internal2"
            self_ip: 10.10.2.2
            netmask: 255.255.255.0
          devices:
          - ip: "10.10.1.2"

Ansible version:

ansible-playbook --version                                     
ansible-playbook [core 2.17.2]
  config file = /Documents/GitHub/local-ansible/ansible.cfg
  configured module search path = ['/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /.pyenv/versions/3.10.14/envs/myenv/lib/python3.10/site-packages/ansible
  ansible collection location = /.ansible/collections:/usr/share/ansible/collections
  executable location = /.pyenv/versions/myenv/bin/ansible-playbook
  python version = 3.10.14 (main, Aug  5 2024, 09:03:11) [Clang 15.0.0 (clang-1500.3.9.4)] (/.pyenv/versions/3.10.14/envs/myenv/bin/python)
  jinja version = 3.1.4
  libyaml = True
  ansible-galaxy collection list  | grep -i bigip           
f5networks.f5_bigip                      3.6.2
prateekramani commented 3 months ago

Hi @megamattzilla,

I was able to reproduce the issue, and it appears to originate from the BIG-IP side. I have been in communication with the BIG-IP team, and they have indicated that this fix will be included in the upcoming BIG-IP v17.5 release, which may take some time. If it is a critical issue, please raise this issue with the BIG-IP team and if you find any problem please let us know.

Thanks.

megamattzilla commented 3 months ago

Could you provide the F5 BZ ID please? I'll add that to the list we are tracking.

Its worth mentioning we can configure the same type of HTTP transparent proxy service (with existing self-IPs) in the Big-IP GUI fine.

The Big-IP GUI is able to configure the appropriate service, but the F5 Rest API cannot?

prateekramani commented 2 months ago

Here's the BZ ID - 1633173. Thanks

megamattzilla commented 2 months ago

Hi @prateekramani,

Thanks for the info!

Are we sure the root cause of this issue is BZ ID 1633173?

Is it possible both issues have similar, vague error messages?

The reason I ask is there is not very many details in BZ 1633173, the few details it mentions is "POST Upgrade" which there is no upgrade involved here.

I'm also able to create the expected HTTP transparent service fine using the GUI with all the same values I'm trying with the ansible module:

2024-08-26_09-58-30

Interestingly enough, if I re-run the ansible module now (with same values I used to successfully created the service in the GUI) the ansible module errors out with a different message.

If the ansible module is doing everything correctly, I would expect it to say "no changes":

This is the ansible output when I run it after creating the service successfully via the GUI:

fatal: [192.168.1.102]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "auth_offload": null,
            "auto_manage": false,
            "devices": [
                {
                    "ip": "10.10.1.2",
                    "port": null
                }
            ],
            "devices_from": {
                "interface": null,
                "netmask": "255.255.255.0",
                "self_ip": "10.10.2.2",
                "tag": null,
                "vlan": "/Common/Internal2"
            },
            "devices_to": {
                "interface": null,
                "netmask": "255.255.255.0",
                "self_ip": "10.10.1.1",
                "tag": null,
                "vlan": "/Common/Internal1"
            },
            "dump_json": false,
            "ip_family": "ipv4",
            "monitor": null,
            "name": "HTTP-TRANSPARENT",
            "port_remap": null,
            "proxy_type": "transparent",
            "rules": null,
            "service_down_action": "ignore",
            "snat": "none",
            "snat_list": null,
            "snat_pool": null,
            "state": "present",
            "timeout": 300,
            "use_exist_selfip": true,
            "vendor_info": null
        }
    },
    "msg": "Self-IPs are immutable. You must delete and recreate the service to change the self-IPs."
}

Why is the ansible module trying to change the self-IPs? Seems like there is a problem with the flag "use_exist_selfip": true.

megamattzilla commented 2 months ago

Repeating the same test- create the HTTP-TRANSPARENT service successfully via the GUI and then running the ansible module with same settings- the ansible module fails even when I add dump_json: true. 

fatal: [192.168.1.102]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "auth_offload": null,
            "auto_manage": false,
            "devices": [
                {
                    "ip": "10.10.1.2",
                    "port": null
                }
            ],
            "devices_from": {
                "interface": null,
                "netmask": "255.255.255.0",
                "self_ip": "10.10.2.2",
                "tag": null,
                "vlan": "/Common/Internal2"
            },
            "devices_to": {
                "interface": null,
                "netmask": "255.255.255.0",
                "self_ip": "10.10.1.1",
                "tag": null,
                "vlan": "/Common/Internal1"
            },
            "dump_json": true,
            "ip_family": "ipv4",
            "monitor": null,
            "name": "HTTP-TRANSPARENT",
            "port_remap": null,
            "proxy_type": "transparent",
            "rules": null,
            "service_down_action": "ignore",
            "snat": "none",
            "snat_list": null,
            "snat_pool": null,
            "state": "present",
            "timeout": 300,
            "use_exist_selfip": true,
            "vendor_info": null
        }
    },
    "msg": "Self-IPs are immutable. You must delete and recreate the service to change the self-IPs."
}

This should return "no changes".

megamattzilla commented 2 months ago

Hi @prateekramani ,

This is definitely looking like an ansible modules issue.

When the ansible module generates the POST request to /mgmt/shared/iapp/blocks the JSON item for "id": "f5-ssl-orchestrator-service" has something wrong with it.

I replaced the JSON item for "id": "f5-ssl-orchestrator-service" from a working configuration (creating the same service via the GUI) and now the JSON document ansible is sending properly creates the service (in postman).

The ansible playbook configuration here generates the following POST request: 

POST /mgmt/shared/iapp/blocks/ HTTP/1.1
Accept-Encoding: identity
Content-Length: 4534
Host: 10.6.1.10:8443
User-Agent: Python-urllib/3.10
Content-Type: application/json
X-F5-Auth-Token: REDACTED
Connection: close

{"name": "sslo_obj_SERVICE_CREATE_ssloS_HTTP-TRANSPARENT", "inputProperties": [{"id": "f5-ssl-orchestrator-operation-context", "type": "JSON", "value": {"version": 11.1, "partition": "Common", "strictness": false, "operationType": "CREATE", "deploymentName": "ssloS_HTTP-TRANSPARENT", "deploymentType": "SERVICE"}}, {"id": "f5-ssl-orchestrator-network", "type": "JSON", "value": []}, {"id": "f5-ssl-orchestrator-service", "type": "JSON", "value": {"customService": {"name": "ssloS_HTTP-TRANSPARENT", "serviceType": "http-proxy", "serviceSpecific": {"name": "ssloS_HTTP-TRANSPARENT", "proxyType": "Transparent", "authOffload": false}, "connectionInformation": {"fromBigipNetwork": {"name": "toNetwork", "vlan": {"path": "/Common/Internal1-Stovall-62", "create": false, "modify": false, "selectedValue": "/Common/Internal1-Stovall-62", "networkVlanValue": ""}, "routeDomain": {"id": 0, "create": false}, "selfIpConfig": {"create": false, "modify": false, "autoValue": "198.19.96.7/25", "selectedValue": "", "selfIp": "10.10.1.1", "netmask": "255.255.255.0", "floating": false, "HAstaticIpMap": []}, "networkBlockId": ""}, "toBigipNetwork": {"name": "fromNetwork", "vlan": {"path": "/Common/Internal2-Stovall-63", "create": false, "modify": false, "selectedValue": "/Common/Internal2-Stovall-63", "networkVlanValue": ""}, "routeDomain": {"id": 0, "create": false}, "selfIpConfig": {"create": false, "modify": false, "autoValue": "198.19.96.245/25", "selectedValue": "", "selfIp": "10.10.2.2", "netmask": "255.255.255.0", "floating": false, "HAstaticIpMap": []}, "networkBlockId": ""}}, "snatConfiguration": {"clientSnat": "None", "snat": {"referredObj": "", "ipv4SnatAddresses": [], "ipv6SnatAddresses": []}}, "loadBalancing": {"devices": [{"ip": "10.10.1.2", "port": 80}], "monitor": {"fromSystem": "/Common/gateway_icmp"}}, "initialIpFamily": "ipv4", "ipFamily": "ipv4", "isAutoManage": false, "portRemap": false, "httpPortRemapValue": 80, "serviceDownAction": "ignore", "iRuleList": [], "managedNetwork": {"serviceType": "http-proxy", "ipFamily": "ipv4", "isAutoManage": false, "ipv4": {"serviceType": "http-proxy", "ipFamily": "ipv4", "serviceSubnet": "10.10.1.0", "serviceIndex": 0, "subnetMask": "255.255.255.0", "toServiceNetwork": "10.10.1.0", "toServiceMask": "255.255.255.0", "toServiceSelfIp": "10.10.1.1", "fromServiceNetwork": "10.10.2.0", "fromServiceMask": "255.255.255.0", "fromServiceSelfIp": "10.10.2.2"}, "operation": "RESERVEANDCOMMIT"}}, "fromVlanNetworkObj": {"create": false, "modify": false, "networkError": false}, "toVlanNetworkObj": {"create": false, "modify": false, "networkError": false}, "fromNetworkObj": {"name": "ssloN_HTTP-TRANSPARENT_in", "partition": "Common", "strictness": false, "vlan": {"create": false, "modify": false, "name": "ssloN_HTTP-TRANSPARENT_in", "path": "/Common/Internal1-Stovall-62", "networkError": false, "interface": [], "tag": 0, "networkInterface": "", "networkTag": 0}, "selfIpConfig": {"create": false, "modify": false, "selfIp": "10.10.1.1", "netmask": "255.255.255.0", "floating": false, "HAstaticIpMap": []}, "routeDomain": {"id": 0, "create": false}}, "toNetworkObj": {"name": "ssloN_HTTP-TRANSPARENT_out", "partition": "Common", "strictness": true, "vlan": {"create": false, "modify": false, "name": "ssloN_HTTP-TRANSPARENT_out", "path": "/Common/Internal2-Stovall-63", "networkError": false, "interface": [], "tag": 0, "networkInterface": "", "networkTag": 0}, "selfIpConfig": {"create": false, "modify": false, "selfIp": "10.10.2.2", "netmask": "255.255.255.0", "floating": false, "HAstaticIpMap": []}, "routeDomain": {"id": 0, "create": false}}, "vendorInfo": {"name": "Generic HTTP Service"}, "name": "ssloS_HTTP-TRANSPARENT", "partition": "Common", "description": "Type: HTTP", "strictness": false, "useTemplate": false, "serviceTemplate": "", "templateName": "HTTP Service", "previousVersion": 11.1, "version": 11.1}}, {"id": "f5-ssl-orchestrator-service-chain", "type": "JSON", "value": []}, {"id": "f5-ssl-orchestrator-policy", "type": "JSON", "value": []}], "configurationProcessorReference": {"link": "https://localhost/mgmt/shared/iapp/processors/f5-iappslx-ssl-orchestrator-gc"}, "configProcessorTimeoutSeconds": 120, "statsProcessorTimeoutSeconds": 60, "configProcessorAffinity": {"processorPolicy": "LOCAL", "affinityProcessorReference": {"link": "https://localhost/mgmt/shared/iapp/affinity/local"}}, "state": "BINDING", "presentationHtmlReference": {"link": "https://localhost/iapps/f5-iappslx-ssl-orchestrator/sgc/sgcIndex.html"}, "operation": "CREATE"}

I tested sending that same POST request in postman and found that I could replicate the issue- the deploy failed every time with Deployment failed for Error: [BaseHAConfigProcessor (ServiceBaseHAConfigProcessor)] TypeError: Cannot read property 'lastIndexOf' of undefined"

After a few hours of testing, I found the root issue is with the JSON item for "id": "f5-ssl-orchestrator-service".

If I replace the ansible authored JSON item for "id": "f5-ssl-orchestrator-service" with a JSON item from a working environment (obtained by running a tcpdump on loopback interface while creating same service via GUI) it successfully creates the service in postman.

This is the fixed JSON item for "id": "f5-ssl-orchestrator-service":

          {
            "id": "f5-ssl-orchestrator-service",
            "type": "JSON",
            "value": {
              "customService": {
                "connectionInformation": {
                  "fromBigipNetwork": {
                    "name": "toNetwork",
                    "routeDomain": {
                      "create": false,
                      "id": 0,
                      "path": ""
                    },
                    "selfIpConfig": {
                      "autoValue": "198.19.96.7/25",
                      "create": false,
                      "netmask": "255.255.255.0",
                      "selectedValue": "10.10.1.1/24",
                      "selfIp": "10.10.1.1/24"
                    },
                    "vlan": {
                      "create": false,
                      "networkVlanValue": "/Common/Internal1-Stovall-62",
                      "path": "/Common/Internal1-Stovall-62",
                      "selectedValue": ""
                    }
                  },
                  "toBigipNetwork": {
                    "name": "fromNetwork",
                    "routeDomain": {
                      "create": false,
                      "id": 0,
                      "path": ""
                    },
                    "selfIpConfig": {
                      "autoValue": "198.19.96.245/25",
                      "create": false,
                      "netmask": "255.255.255.0",
                      "selectedValue": "10.10.2.2/24",
                      "selfIp": "10.10.2.2/24"
                    },
                    "vlan": {
                      "create": false,
                      "networkVlanValue": "/Common/Internal2-Stovall-63",
                      "path": "/Common/Internal2-Stovall-63",
                      "selectedValue": ""
                    }
                  }
                },
                "controlChannels": [],
                "httpPortRemapValue": "80",
                "iRuleList": [],
                "ipFamily": "ipv4",
                "isAutoManage": false,
                "loadBalancing": {
                  "devices": [
                    {
                      "deviceDescription": "",
                      "ip": "10.10.1.2",
                      "port": 80
                    }
                  ],
                  "monitor": {
                    "fromSystem": "/Common/gateway_icmp"
                  }
                },
                "managedNetwork": {
                  "ipFamily": "ipv4",
                  "ipv4": {
                    "fromServiceMask": "255.255.255.128",
                    "fromServiceNetwork": "198.19.96.128",
                    "fromServiceSelfIp": "198.19.96.245",
                    "ipFamily": "ipv4",
                    "serviceIndex": 0,
                    "serviceSubnet": "198.19.96.0",
                    "serviceType": "http-proxy",
                    "subnetMask": "255.255.255.0",
                    "toServiceMask": "255.255.255.128",
                    "toServiceNetwork": "198.19.96.0",
                    "toServiceSelfIp": "198.19.96.7"
                  },
                  "isAutoManage": false,
                  "serviceType": "http-proxy"
                },
                "name": "ssloS_HTTP-TRANSPARENT",
                "portRemap": false,
                "serviceDownAction": "ignore",
                "serviceEntrySSLProfile": "",
                "serviceReturnSSLProfile": "",
                "serviceSpecific": {
                  "name": "ssloS_HTTP-TRANSPARENT",
                  "proxyType": "Transparent"
                },
                "serviceType": "http-proxy",
                "snatConfiguration": {
                  "clientSnat": "None",
                  "snat": {
                    "ipv4SnatAddresses": [
                      {
                        "ip": ""
                      }
                    ],
                    "ipv6SnatAddresses": [
                      {
                        "ip": ""
                      }
                    ],
                    "referredObj": ""
                  }
                }
              },
              "description": "Type: http-proxy",
              "fromVlanNetworkObj": {
                "create": false,
                "modify": false,
                "networkError": false
              },
              "initialIpFamily": "ipv4",
              "name": "ssloS_HTTP-TRANSPARENT",
              "obRestrictedAttribute": {
                "obId": "79740939-fe71-40a3-bd7f-d5209d13d67f"
              },
              "partition": "Common",
              "previousVersion": "11.0",
              "serviceTemplate": "",
              "strictness": true,
              "toVlanNetworkObj": {
                "create": false,
                "modify": false,
                "networkError": false
              },
              "useTemplate": false,
              "vendorInfo": {
                "name": "Generic HTTP Service"
              },
              "version": "11.0"
            }
          }

There is quite a few differences in the JSON keys between the working and non-working item, but one of them seems to be causing the 'lastIndexOf' of undefined error.

prateekramani commented 2 months ago

Hi @megamattzilla I will try to reproduce the issue and let you know.

megamattzilla commented 2 months ago

Files: bad.json good.json

POST payload ansible is sending fails with error 'lastIndexOf' of undefined: curl -k -X POST -H "Content-Type: application/json" https://{{Big-IP MGMT IP}}/mgmt/shared/iapp/blocks -u admin -d @bad.json -v

POST payload of ansible + fixed JSON is 100% successful on 17.1.1.3: curl -k -X POST -H "Content-Type: application/json" https://{{Big-IP MGMT IP}}/mgmt/shared/iapp/blocks -u admin -d @good.json -v 2024-09-20_10-04-50

The Big-IP BZ 1633173 might affect how ansible was previously constructing the JSON payload, but there appears to be alternative ways to structure the JSON that works on 17.1.1.x.

Could the ansible module construct the JSON payload in the way that works for 17.1.1.x?