Update to CISv2.1.1 failed with code 422 - partition cannot removed due to remaining objects

joebride commented 3 years ago

Setup Details

CIS Version : 2.1.1
Build: f5networks/k8s-bigip-ctlr:2.1.1
BIGIP Version: Big IP 14.1.2.5 (ENG-Build 0.101.3) AS3 Version: 3.20
Agent Mode: AS3
Orchestration: K8S/OSCP
Orchestration Version: OpenShift v3.11.219
Pool Mode: Cluster Additional Setup details: <Platform/CNI Plugins/ cluster nodes/ etc>

Description

Update from CISv1.14 (or CISv2.0.0) to CISv2.1.1 fails with the following error-messages: 2020/10/14 16:05:16 [ERROR] [AS3] Raw response from Big-IP: map[code:422 declaration:HTML Tag-like Content in the Request URL/Body results:[map[code:422 host:localhost message:declaration failed response:0107082a:3: All objects must be removed from a partition (appaotel1_AS3) before the partition may be removed, type ID (13533) runTime:5014 tenant:appaotel1_AS3]]]

I tested it with and without as3-override-configmap. In both cases it does not work!

Steps To Reproduce

1) use CISv1.14 or CISv2.0.0 as starting point 2) stop CIS 3) change image - parameter properly 4) start CIS with new image-parameter

Expected Result

Update must be performed successfully!

Actual Result

Update does not work!

Diagnostic Information

BIG-IP log: [I][3145939][14 Oct 2020 16:05:16 UTC][ForwarderPassThroughWorker] {"user":"local/XXX","method":"POST","uri":"http://localhost:8100/mgmt/shared/appsvcs/declare/","status":422,"from":"10.171.230.100"}

CIS-parameters containers:

args:
- '--bigip-username=$(BIGIP_USERNAME)'
- '--bigip-password=$(BIGIP_PASSWORD)'
- '--bigip-url=XXX'
- '--bigip-partition=appaotel1'
- '--pool-member-type=cluster'
- '--openshift-sdn-name=/Common/RD_000_TUN_VXL_0001'
- '--log-level=INFO'
- '--log-as3-response=true'
- '--manage-routes=true'
- '--manage-ingress=false'
- '--route-label=XXX'
- '--route-http-vserver=RD_000_VS_RED_APPAOTEL1_80'
- '--route-https-vserver=RD_000_VS_STD_APPAOTEL1_443'
- '--route-vserver-addr=XXX'
- '--agent=as3'
- '--insecure=true'
- '--tls-version=1.3'
- - --cipher-group=/Common/XXX

Observations (if any)

mdditt2000 commented 3 years ago

@joebride please can you review the following link to assist with migration from CIS 1.14 to CIS 2.x. I understand this document describes CCCL to AS3. In your case you are already using AS3.

Step 1

Upgrade CIS 1.x to CIS 2.0 remaining on AS3 agent. CIS 2.0 AS3 is default
Make sure all namespaces that CIS is watching are specified in the CIS deployment

Step 2

Remove agent=AS3 from the CIS deployment
IF BIG-IP is using a self-signed cert add insecure=true to the CIS deployment. -- this is already done
Install AS3-18 which the correct version for CIS 2.0 -- Need CIS AS3-18
Make sure all namespaces that CIS is watching are specified in the CIS deployment

Step 3

Install AS3-21 which the correct version for CIS 2.1.1

Similar to the document i posted except you are not changing the API. https://github.com/mdditt2000/openshift-3-11/blob/master/enviroment/migration/migration.md#upgrading-from-cis-1x-to-cis-21-and-above-using-cccl-to-as3

mdditt2000 commented 3 years ago

The error you are seeing will be removed once going from CIS 2.0 to CIS 2.1. In CIS 2.1 we are removing the _AS3 partition and moving all those objects to the base CIS partition. '--bigip-partition=appaotel1'

joebride commented 3 years ago

the described update-procedure does not work. please contact me.

mdditt2000 commented 3 years ago

GUI objects were getting created in appaotel1_AS3 for AWAF analytics. Since AS3 never created these objects they couldn't get moved by CIS/AS3. Had to resort to remove the objects by changing the bigip.conf file. One we changed to a new bigip.conf file. CIS no longer received the 422.

Second BIG-IP is in progress of getting upgraded from CIS 1.14 to CIS 2.1.1

F5Networks / k8s-bigip-ctlr