DO fails to provision LTM/ASM when there is pre-existing route configuration from the nicswap

[papineni87]

Environment

• Declarative Onboarding Version:1.23.0
• BIG-IP Version:15.1.2. or 16.1

Summary

I am seeing below error when try to POST DO declaration on GCP instance.

{
    "id": "a67ff737-6af2-45df-86f8-57c312a1a427",
    "selfLink": "https://localhost/mgmt/shared/declarative-onboarding/task/a67ff737-6af2-45df-86f8-57c312a1a427",
    "code": 422,
    "status": "ERROR",
    "message": "invalid config - rolled back",
    "errors": [
        "\"type\" may not be specified with \"gateway\"",
        "\"type\" may not be specified with \"gateway\""
    ],
    "result": {
        "class": "Result",
        "code": 422,
        "status": "ERROR",
        "message": "invalid config - rolled back",
        "errors": [
            "\"type\" may not be specified with \"gateway\"",
            "\"type\" may not be specified with \"gateway\""
        ]
    },
    "declaration": {
        "schemaVersion": "1.0.0",
        "class": "Device",
        "async": true,
        "label": "Onboard BIG-IP",
        "Common": {
            "class": "Tenant",
            "myProvisioning": {
                "class": "Provision",
                "ltm": "nominal",
                "asm": "nominal"
            }
        }
    }
}

Steps To Reproduce

Steps to reproduce the behavior:

1. Submit the following declaration to bigip instance on GCP ( after nic swapping )

   
   {
   "schemaVersion": "1.0.0",
   "class": "Device",
   "async": true,
   "label": "Onboard BIG-IP",
   "Common": {
       "class": "Tenant",
   
       "myProvisioning": {
           "class": "Provision",
           "ltm": "nominal",
           "asm": "nominal"
       }
   }
   }

nic swapping commands used to bring up bigip in GCP

cat << 'EOF' >> /config/cloud/nic_swap.sh

!/bin/bash

source /usr/lib/bigstart/bigip-ready-functions echo "before nic swapping" tmsh list sys db provision.1nicautoconfig tmsh list sys db provision.managementeth echo "after nic swapping" bigstart stop tmm tmsh modify sys db provision.managementeth value eth1 tmsh modify sys db provision.1nicautoconfig value disable bigstart start tmm wait_bigip_ready echo "---Mgmt interface setting---" tmsh list sys db provision.managementeth tmsh list sys db provision.1nicautoconfig sed -i "s/iface0=eth0/iface0=eth1/g" /etc/ts/common/image.cfg echo "Done changing interface" echo "Set TMM networks" MGMTADDRESS=$(curl -s -f --retry 10 -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/1/ip) MGMTMASK=$(curl -s -f --retry 10 -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/1/subnetmask) MGMTGATEWAY=$(curl -s -f --retry 10 -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/1/gateway) MGMTNETWORK=$(/bin/ipcalc -n $MGMTADDRESS $MGMTMASK | cut -d= -f2) INT1GATEWAY=$(curl -s -f --retry 10 -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/gateway) echo $MGMTADDRESS echo $MGMTMASK echo $MGMTGATEWAY echo $MGMTNETWORK tmsh modify sys global-settings gui-setup disabled tmsh modify sys global-settings mgmt-dhcp disabled tmsh delete sys management-route all tmsh delete sys management-ip all tmsh create sys management-ip $${MGMTADDRESS}/32 tmsh create sys management-route mgmt_gw network $${MGMTGATEWAY}/32 type interface mtu 1460 tmsh create sys management-route mgmt_net network $${MGMTNETWORK}/$${MGMTMASK} gateway $${MGMTGATEWAY} mtu 1460 tmsh create sys management-route default gateway $${MGMTGATEWAY} mtu 1460 tmsh modify sys global-settings remote-host add { metadata.google.internal { hostname metadata.google.internal addr 169.254.169.254 } } tmsh modify sys management-dhcp sys-mgmt-dhcp-config request-options delete { ntp-servers } tmsh save /sys config reboot

2. Observe the following error response:
```json
{
    "id": "c5a5e4c4-a7a3-470d-a077-08ee43ee01a2",
    "selfLink": "https://localhost/mgmt/shared/declarative-onboarding/task/c5a5e4c4-a7a3-470d-a077-08ee43ee01a2",
    "code": 422,
    "status": "ERROR",
    "message": "invalid config - rolled back",
    "errors": [
        "\"type\" may not be specified with \"gateway\"",
        "\"type\" may not be specified with \"gateway\""
    ],
    "result": {
        "class": "Result",
        "code": 422,
        "status": "ERROR",
        "message": "invalid config - rolled back",
        "errors": [
            "\"type\" may not be specified with \"gateway\"",
            "\"type\" may not be specified with \"gateway\""
        ]
    },
    "declaration": {
        "schemaVersion": "1.0.0",
        "class": "Device",
        "async": true,
        "label": "Onboard BIG-IP",
        "Common": {
            "class": "Tenant",
            "myProvisioning": {
                "class": "Provision",
                "ltm": "nominal",
                "asm": "nominal"
            }
        }
    }
}

Expected Behavior

DO should be able to succeed.

Actual Behavior

Unable to provision ASM via DO. But i can manually provision ASM through GUI

[papineni87]

Below are the management routes configured by nic swaping in gcp instance

bigipuser@(bigip1)(cfg-sync Standalone)(Active)(/Common)(tmos)# list sys management-route 
sys management-route default {
    gateway 10.1.0.1
    mtu 1460
    network default
}
sys management-route mgmt_gw {
    mtu 1460
    network 10.1.0.1/32
    type interface
}
sys management-route mgmt_net {
    gateway 10.1.0.1
    mtu 1460
    network 10.1.0.0/16
}
bigipuser@(bigip1)(cfg-sync Standalone)(Active)(/Common)(tmos)# 

[dstokesf5]

Are you able to provide trace information from this failure?

https://clouddocs.f5.com/products/extensions/f5-declarative-onboarding/latest/declarations/miscellaneous.html#enabling-traces-in-do-responses

[papineni87]

Yes, i have shared it in Jira ticket

[dstokesf5]

Thank you for your feedback. I have added this issue to our internal product backlog as AUTOTOOL-2768.