Remark field should match format \"f5remark\"

azahajkiewicz commented 3 years ago

Environment

Application Services Version: { "version": "3.25.0", "release": "3", "schemaCurrent": "3.25.0", "schemaMinimum": "3.0.0" }
BIG-IP Version: [root@journeys-test-system:/S2-green-P::Active:Standalone] restnoded # cat /VERSION Product: BIG-IP Version: 14.1.3 Build: 0.0.7 Sequence: 14.1.3-0.0.7.0 BaseBuild: 0.0.7 Edition: Final

VELOS platform

Summary

If an object has a description, it is translated into "remark" in AS3 declaration. Description of the object in BIG-IP does not have a specific format, but the "remark" field does. It leads to a situations when the AS3 declaration cannot be deployed successfully, because an error occurs.

Steps To Reproduce

Steps to reproduce the behavior:

Convert following config:


ltm virtual /Common/pem_listeners_ANY_IP {
creation-time 2020-11-26:03:53:43
**description** "Created by Web Configuration Utility for PEM Listener pem_listeners"
destination /Common/10.144.18.36:0
last-modified-time 2020-11-26:03:53:43
mask 255.255.255.255
profiles {
    /Common/classification_pem {
        context clientside
    }
    /Common/ipother { }
    /Common/pem_listeners_pem_profile {
        context clientside
    }
}
source 0.0.0.0/24
translate-address enabled
translate-port disabled
}

ltm rule /Common/validation_rule {

when CLIENT_ACCEPTED {
     set hsludp {[HSL::open -proto UDP -pool /Common/just_pool]
}

}


2. Observe the following error message:

No error from Charon - Charon successfully translates config into AS3, because it does not validate remark format.

Error while deploying AS3 declaration:

Thu, 28 Jan 2021 15:33:11 GMT - warning: [appsvcs] {"status":422,"message":"declaration is invalid","errors":["/tenant1/application_1/pem_listeners_L4_2/remark: should match format \"f5remark\""],"level":"warning"}

{ "code": 422, "errors": [ "/tenant1/application_1/pem_listeners_L4_2/remark: should match format \"f5remark\"" ], "declarationFullId": "", "message": "declaration is invalid" }



### Expected Behavior
Charon should validate "Description" properties of the config objects .

### Actual Behavior
Charon does not validate "Description" properties and translates it into "remark" fields, creating invalid declaration. 
FYI: invalid description in the example is created automatically by the BIG-IP configuration Utility.

mdditt2000 commented 3 years ago

Created the following AS3 declaration

{
    "class": "ADC",
    "schemaVersion": "3.25.0",
    "id": "urn:uuid:e1193921-adcf-47f6-b3b7-6ac43e06443a",
    "label": "Converted Declaration",
    "remark": "Auto-generated by Project Charon",
    "Common": {
        "class": "Tenant",
        "Shared": {
            "class": "Application",
            "template": "shared",
            "pem_listeners_ANY_IP": {
                "translateServerAddress": true,
                "translateServerPort": false,
                "class": "Service_Generic",
                "profileClassification": {
                    "bigip": "/Common/classification_pem"
                },
                "profileIPOther": {
                    "bigip": "/Common/ipother"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.36",
                        "0.0.0.0/24"
                    ]
                ],
                "virtualPort": 0,
                "persistenceMethods": [],
                "snat": "none"
            },
            "validation_rule": {
                "class": "iRule",
                "iRule": {
                    "base64": "ICAgIHdoZW4gQ0xJRU5UX0FDQ0VQVEVEIHsKICAgICAgICAgc2V0IGhzbHVkcCB7W0hTTDo6b3BlbiAtcHJvdG8gVURQIC1wb29sIC9Db21tb24vU2hhcmVkL2p1c3RfcG9vbF0KICAgIH0KfQ=="
                }
            }
        }
    }
}

mdditt2000 commented 3 years ago

Github files from conversion using ACC 1.10 https://github.com/mdditt2000/f5-appsvcs-acc/tree/master/Github/8

mdditt2000 commented 3 years ago

Posted to BIG-IP i am getting the following errors

{
    "id": "1fe80513-d2e9-4afb-aa6a-0d7ad9cecfd8",
    "results": [
        {
            "code": 422,
            "declarationFullId": "",
            "message": "Unable to find /Common/classification_pem for /Common/Shared/pem_listeners_ANY_IP/profileClassification"
        }
    ],
    "declaration": {}
}

This message makes sense as /Common/classification_pem is not on BIG-IP so 422 is valid

mdditt2000 commented 3 years ago

Closing this issue as the declaration is created correctly via ACC

azahajkiewicz commented 3 years ago

Hi Mark, The actual problem here is the description field of the objects, which Charon converts as 'remark'. I have just reproduced the issue with the newest Charon version (f5-appsvcs-acc:1.10.0 image). ACC converted UCS to:

{
    "class": "ADC",
    "schemaVersion": "3.25.0",
    "id": "urn:uuid:d21f85ce-6c90-4de3-855c-e6fd00a78b00",
    "label": "Converted Declaration",
    "remark": "Auto-generated by Project Charon",
    "Common": {
        "class": "Tenant",
        "Shared": {
            "class": "Application",
            "template": "shared",
            "pool_test": {
                "members": [
                    {
                        "addressDiscovery": "static",
                        "servicePort": 22,
                        "serverAddresses": [
                            "10.146.65.121"
                        ],
                        "shareNodes": true
                    }
                ],
                "monitors": [
                    {
                        "bigip": "/Common/tcp_half_open"
                    }
                ],
                "class": "Pool"
            },
            "policy_subs": {
                "rules": [
                    {
                        "name": "rule_subs",
                        "precedence": 10,
                        "modifyHttpHeader": {
                            "headerName": "Encoding",
                            "operation": "insert",
                            "valueContent": "BLABLA"
                        },
                        "DTOSTethering": {
                            "detectDtos": true,
                            "detectTethering": true,
                            "reportDestinationHsl": {
                                "highSpeedLogPublisher": {
                                    "bigip": "/Common/local-db-publisher"
                                }
                            }
                        }
                    }
                ],
                "class": "Enforcement_Policy"
            },
            "policy_unknown": {
                "enable": false,
                "class": "Enforcement_Policy"
            },
            "test_addr_list": {
                "addresses": [
                    "10.144.65.121",
                    "10.145.65.122"
                ],
                "remark": "test_addr_list",
                "class": "Firewall_Address_List"
            }
        }
    },
    "tenant1": {
        "class": "Tenant",
        "application_1": {
            "class": "Application",
            "template": "generic",
            "pem_listeners_ANY_IP": {
                "remark": "application_1",
                "translateServerAddress": true,
                "translateServerPort": false,
                "class": "Service_Generic",
                "profileClassification": {
                    "bigip": "/Common/classification_pem"
                },
                "profileIPOther": {
                    "bigip": "/Common/ipother"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.36",
                        "0.0.0.0/24"
                    ]
                ],
                "virtualPort": 0,
                "persistenceMethods": [],
                "snat": "none"
            },
            "pem_listeners_FAST_L4": {
                "remark": "application_1",
                "layer4": "tcp",
                "translateServerAddress": true,
                "translateServerPort": true,
                "class": "Service_HTTP",
                "profileClassification": {
                    "bigip": "/Common/classification_pem"
                },
                "profileTCP": {
                    "bigip": "/Common/f5-tcp-progressive"
                },
                "profileHTTP": {
                    "bigip": "/Common/http"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.36",
                        "0.0.0.0/32"
                    ]
                ],
                "virtualPort": 80,
                "persistenceMethods": [],
                "snat": "none"
            },
            "pem_listeners_HTTP": {
                "remark": "application_1",
                "layer4": "tcp",
                "translateServerAddress": true,
                "translateServerPort": true,
                "class": "Service_Generic",
                "profileClassification": {
                    "bigip": "/Common/classification_pem"
                },
                "profileIPOther": {
                    "bigip": "/Common/ipother"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.36",
                        "0.0.0.0/24"
                    ]
                ],
                "virtualPort": 80,
                "persistenceMethods": [],
                "snat": "none"
            },
            "pem_listeners_L4_1": {
                "remark": "application_1",
                "layer4": "tcp",
                "translateServerAddress": true,
                "translateServerPort": false,
                "class": "Service_Generic",
                "profileClassification": {
                    "bigip": "/Common/classification_pem"
                },
                "profileIPOther": {
                    "bigip": "/Common/ipother"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.36",
                        "0.0.0.0/24"
                    ]
                ],
                "virtualPort": 0,
                "persistenceMethods": [],
                "snat": "none"
            },
            "pem_listeners_L4_2": {
                "remark": "application_1",
                "layer4": "udp",
                "translateServerAddress": true,
                "translateServerPort": false,
                "class": "Service_Generic",
                "profileClassification": {
                    "bigip": "/Common/classification_pem"
                },
                "profileIPOther": {
                    "bigip": "/Common/ipother"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.36",
                        "0.0.0.0/24"
                    ]
                ],
                "virtualPort": 0,
                "persistenceMethods": [],
                "snat": "none"
            },
            "pem_listeners": {
                "enforcementProfile": {
                    "use": "/tenant1/application_1/pem_listeners_pem_profile"
                },
                "services": [
                    {
                        "use": "/tenant1/application_1/pem_listeners_ANY_IP"
                    },
                    {
                        "use": "/tenant1/application_1/pem_listeners_HTTP"
                    },
                    {
                        "use": "/tenant1/application_1/pem_listeners_L4_1"
                    },
                    {
                        "use": "/tenant1/application_1/pem_listeners_L4_2"
                    }
                ],
                "class": "Enforcement_Listener"
            },
            "pem_listeners_pem_profile": {
                "remark": "\"Created by Web Configuration Utility for PEM Listener pem_listeners\"",
                "connectionOptimizationEnabled": true,
                "connectionOptimizationService": {
                    "use": "none"
                },
                "class": "Enforcement_Profile"
            }
        }
    },
    "tenant2": {
        "class": "Tenant",
        "application_2": {
            "class": "Application",
            "template": "generic",
            "cgnat_vs": {
                "remark": "application_2",
                "layer4": "tcp",
                "translateServerAddress": false,
                "translateServerPort": false,
                "class": "Service_TCP",
                "profileTCP": {
                    "bigip": "/Common/tcp"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.44",
                        "0.0.0.0/24"
                    ]
                ],
                "virtualPort": 0,
                "persistenceMethods": [],
                "snat": "none"
            }
        }
    },
    "tenant3": {
        "class": "Tenant",
        "application_3": {
            "class": "Application",
            "template": "generic",
            "pool_test": {
                "members": [
                    {
                        "addressDiscovery": "static",
                        "servicePort": 22,
                        "serverAddresses": [
                            "10.146.65.121"
                        ],
                        "shareNodes": true
                    }
                ],
                "minimumMembersActive": 1,
                "class": "Pool",
                "minimumMonitors": 1,
                "monitors": [
                    {
                        "bigip": "/Common/tcp_half_open"
                    }
                ]
            },
            "VS_test": {
                "remark": "application_3",
                "layer4": "tcp",
                "pool": "pool_test",
                "translateServerAddress": true,
                "translateServerPort": false,
                "class": "Service_L4",
                "profileL4": {
                    "bigip": "/Common/fastL4"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.39",
                        "0.0.0.0/24"
                    ]
                ],
                "virtualPort": 0,
                "persistenceMethods": [],
                "snat": "none",
                "allowVlans": [
                    {
                        "bigip": "/Common/vlan1"
                    },
                    {
                        "bigip": "/Common/vlan2"
                    },
                    {
                        "bigip": "/Common/vlanGroup_test"
                    }
                ]
            }
        }
    }
}

Response:

{
    "code": 422,
    "errors": [
        "/tenant1/application_1/pem_listeners_pem_profile/remark: should match format \"f5remark\""
    ],
    "declarationFullId": "",
    "message": "declaration is invalid"
}

Please test it with attached (change file name to integrationDeployTest.ucs)

integrationDeployTest.ucs.zip

mdditt2000 commented 3 years ago

@azahajkiewicz so what you are saying is you dont like the way ACC is converting the description to "remark": "application_1" etc?

Looking at integrationDeployTest.ucs i dont see the bigip.conf file used in your example. Please can you share the BIG-IP.conf for the configuration below.

"tenant1": {
        "class": "Tenant",
        "application_1": {
            "class": "Application",
            "template": "generic",
            "pem_listeners_ANY_IP": {
                "remark": "application_1",

azahajkiewicz commented 3 years ago

I like this conversion, but it is not applied in all places:

"pem_listeners_pem_profile": {
                "remark": "\"Created by Web Configuration Utility for PEM Listener pem_listeners\"",

So here the deployment fails. You can try with the ucs:

integrationDeployTest.ucs.zip

p-semenov-f5 commented 3 years ago

https://jira.pdsea.f5net.com/browse/CHARON-223

p-semenov-f5 commented 3 years ago

Finished

azahajkiewicz commented 3 years ago

Hello, error still observed.

journeys.errors.AS3DeclarationDeploymentError: iControl REST error: code: 422, message: declaration is invalid, errors: [‘/testing_tenant/application_1/VS_https/remark: should match format “f5remark”’]

Description of the virtual: “Standard Virtual enabled on 443 with ssl profiles and custom cert/key”

Remark can have up to 64 signs... Can you please review the code and add such restriction while converting description field into remark?

f5devcentral / f5-automation-config-converter