Closed pmilot closed 9 months ago
@pmilot The logs shared above has no AS3 data being sent to BIG-IP. Please enable log-as3-response
and share the logs.
Try delete and recreate the CRD.
@trinaths
CIS Log followed by list ltm pool
2023/03/13 14:08:05 [DEBUG] [2023-03-13 14:08:05,018 __main__ DEBUG] config handler woken for reset
2023/03/13 14:08:05 [DEBUG] [2023-03-13 14:08:05,018 __main__ DEBUG] loaded configuration file successfully
2023/03/13 14:08:05 [DEBUG] [2023-03-13 14:08:05,018 __main__ DEBUG] NET Config: {}
2023/03/13 14:08:05 [DEBUG] [2023-03-13 14:08:05,019 __main__ DEBUG] loaded configuration file successfully
2023/03/13 14:08:05 [DEBUG] [2023-03-13 14:08:05,019 __main__ DEBUG] updating tasks finished, took 0.0012447834014892578 seconds
2023/03/13 14:08:11 [DEBUG] Processing Key: &{istio-system TLSProfile statusdemo-tls 0xc00085aea0 Create}
2023/03/13 14:08:11 [INFO] Enqueueing TLSProfile: &{{ } {statusdemo-tls istio-system 159a52c6-f4b7-458d-bfdd-aedef7123bbd 9709435 1 2023-03-13 14:08:11 +0000 UTC <nil> <nil> map[f5cr:true] map[kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"cis.f5.com/v1","kind":"TLSProfile","metadata":{"annotations":{},"labels":{"f5cr":"true"},"name":"statusdemo-tls","namespace":"istio-system"},"spec":{"hosts":["statusdemo.pmilot.ecsdev.entrust.com"],"tls":{"clientSSL":"/Common/k8s_ingress_clientside_tls_profile","reference":"bigip","serverSSL":"/Common/istiogw-serverssl","termination":"reencrypt"}}}
] [] [] [{kubectl-client-side-apply Update cis.f5.com/v1 2023-03-13 14:08:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}},"f:labels":{".":{},"f:f5cr":{}}},"f:spec":{".":{},"f:hosts":{},"f:tls":{".":{},"f:clientSSL":{},"f:reference":{},"f:serverSSL":{},"f:termination":{}}}}}]} {[statusdemo.pmilot.ecsdev.entrust.com] {reencrypt /Common/k8s_ingress_clientside_tls_profile [] /Common/istiogw-serverssl [] bigip}}}
2023/03/13 14:08:11 [INFO] No VirtualServers found in namespace istio-system
2023/03/13 14:08:13 [DEBUG] Enqueueing VirtualServer: &{{ } {istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls istio-system d14cc57a-4eb8-4fcb-a93d-b4dedc9c69dd 9709443 1 2023-03-13 14:08:13 +0000 UTC <nil> <nil> map[f5cr:true] map[kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"cis.f5.com/v1","kind":"VirtualServer","metadata":{"annotations":{},"labels":{"f5cr":"true"},"name":"istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls","namespace":"istio-system"},"spec":{"allowVlans":["/Common/internal"],"host":"statusdemo.pmilot.ecsdev.entrust.com","iRules":["/Common/Shared/k8s_ingress_sni_irule"],"pools":[{"monitor":{"interval":3,"timeout":10,"type":"tcp"},"path":"/","service":"istio-ingressgateway","servicePort":443}],"snat":"auto","tlsProfileName":"statusdemo-tls","virtualServerAddress":"10.xx.xx.170","virtualServerName":"statusdemo-pmilot-ecsdev-entrust-com-tls"}}
] [] [] [{kubectl-client-side-apply Update cis.f5.com/v1 2023-03-13 14:08:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}},"f:labels":{".":{},"f:f5cr":{}}},"f:spec":{".":{},"f:allowVlans":{},"f:host":{},"f:iRules":{},"f:pools":{},"f:snat":{},"f:tlsProfileName":{},"f:virtualServerAddress":{},"f:virtualServerName":{}}}}]} {statusdemo.pmilot.ecsdev.entrust.com 10.xx.xx.170 [] statusdemo-pmilot-ecsdev-entrust-com-tls 0 0 [{ / istio-ingressgateway {0 443 } {tcp 3 10 0 } [] 0 }] statusdemo-tls auto [/Common/internal] [/Common/Shared/k8s_ingress_sni_irule] [] {{ } [] } [] false } { }}
2023/03/13 14:08:13 [DEBUG] Processing Key: &{istio-system VirtualServer istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls 0xc00035d500 Create}
2023/03/13 14:08:13 [DEBUG] Process all the Virtual Servers which share same VirtualServerAddress
2023/03/13 14:08:13 [DEBUG] Processing Virtual Server istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls for port 443
2023/03/13 14:08:13 [DEBUG] Configured rule: {vs_statusdemo_pmilot_ecsdev_entrust_com_istio_ingressgateway_443_istio_system_statusdemo_pmilot_ecsdev_entrust_com statusdemo.pmilot.ecsdev.entrust.com 0 [0xc00047dd40] [0xc0004c0640]}
2023/03/13 14:08:13 [DEBUG] Configured policy: {statusdemo_pmilot_ecsdev_entrust_com_tls_443_statusdemo_pmilot_ecsdev_entrust_com_policy istio-system [forwarding] true [http] [0xc000b16480] /Common/first-match}
2023/03/13 14:08:13 [DEBUG] Processing BIGIP referenced profiles for 'VirtualServer' 'istio-system'/'istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls'
2023/03/13 14:08:13 [DEBUG] Updated BIGIP referenced profiles for 'VirtualServer' 'istio-system'/'istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls'
2023/03/13 14:08:13 [DEBUG] Updated Virtual istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls with TLSProfile statusdemo-tls
2023/03/13 14:08:13 [DEBUG] Finished syncing virtual servers &{TypeMeta:{Kind: APIVersion:} ObjectMeta:{Name:istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls GenerateName: Namespace:istio-system SelfLink: UID:d14cc57a-4eb8-4fcb-a93d-b4dedc9c69dd ResourceVersion:9709443 Generation:1 CreationTimestamp:2023-03-13 14:08:13 +0000 UTC DeletionTimestamp:<nil> DeletionGracePeriodSeconds:<nil> Labels:map[f5cr:true] Annotations:map[kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"cis.f5.com/v1","kind":"VirtualServer","metadata":{"annotations":{},"labels":{"f5cr":"true"},"name":"istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls","namespace":"istio-system"},"spec":{"allowVlans":["/Common/internal"],"host":"statusdemo.pmilot.ecsdev.entrust.com","iRules":["/Common/Shared/k8s_ingress_sni_irule"],"pools":[{"monitor":{"interval":3,"timeout":10,"type":"tcp"},"path":"/","service":"istio-ingressgateway","servicePort":443}],"snat":"auto","tlsProfileName":"statusdemo-tls","virtualServerAddress":"10.xx.xx.170","virtualServerName":"statusdemo-pmilot-ecsdev-entrust-com-tls"}}
] OwnerReferences:[] Finalizers:[] ClusterName: ManagedFields:[{Manager:kubectl-client-side-apply Operation:Update APIVersion:cis.f5.com/v1 Time:2023-03-13 14:08:13 +0000 UTC FieldsType:FieldsV1 FieldsV1:{"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}},"f:labels":{".":{},"f:f5cr":{}}},"f:spec":{".":{},"f:allowVlans":{},"f:host":{},"f:iRules":{},"f:pools":{},"f:snat":{},"f:tlsProfileName":{},"f:virtualServerAddress":{},"f:virtualServerName":{}}}}]} Spec:{Host:statusdemo.pmilot.ecsdev.entrust.com HostGroup: VirtualServerAddress:10.xx.xx.170 AdditionalVirtualServerAddresses:[] IPAMLabel: VirtualServerName:statusdemo-pmilot-ecsdev-entrust-com-tls VirtualServerHTTPPort:0 VirtualServerHTTPSPort:0 Pools:[{Name: Path:/ Service:istio-ingressgateway ServicePort:{Type:0 IntVal:443 StrVal:} NodeMemberLabel: Monitor:{Type:tcp Send: Recv: Interval:3 Timeout:10 TargetPort:0 Name: Reference:} Monitors:[] Rewrite: Balance: WAF: ServiceNamespace: ReselectTries:0 ServiceDownAction: HostRewrite:}] TLSProfileName:statusdemo-tls HTTPTraffic: SNAT:auto WAF: RewriteAppRoot: AllowVLANs:[/Common/internal] IRules:[/Common/Shared/k8s_ingress_sni_irule] ServiceIPAddress:[] PolicyName: PersistenceProfile: ProfileMultiplex: DOS: BotDefense: Profiles:{TCP:{Client: Server:} UDP: HTTP: HTTP2: RewriteProfile: PersistenceProfile: LogProfiles:[] ProfileL4: ProfileMultiplex:} AllowSourceRange:[] HttpMrfRoutingEnabled:false Partition:} Status:{VSAddress: StatusOk:}} (542.405µs)
2023/03/13 14:08:13 [DEBUG] [CCCL] ConfigWriter (0xc0001b1e60) writing section name gtm
2023/03/13 14:08:13 [DEBUG] [CCCL] ConfigWriter (0xc0001b1e60) successfully wrote section (gtm)
2023/03/13 14:08:13 [DEBUG] Wrote gtm config section: map[pmrke2v1-cluster:{map[]}]
2023/03/13 14:08:13 [DEBUG] [AS3] PostManager Accepted the configuration
2023/03/13 14:08:13 [DEBUG] [AS3] posting request to https://10.xx.xx.140/mgmt/shared/appsvcs/declare/pmrke2v1-cluster
2023/03/13 14:08:35 [DEBUG] [2023-03-13 14:08:35,019 __main__ DEBUG] config handler woken for reset
2023/03/13 14:08:35 [DEBUG] [2023-03-13 14:08:35,019 __main__ DEBUG] loaded configuration file successfully
2023/03/13 14:08:35 [DEBUG] [2023-03-13 14:08:35,019 __main__ DEBUG] NET Config: {}
2023/03/13 14:08:35 [DEBUG] [2023-03-13 14:08:35,019 __main__ DEBUG] loaded configuration file successfully
2023/03/13 14:08:35 [DEBUG] [2023-03-13 14:08:35,020 __main__ DEBUG] updating tasks finished, took 0.0008637905120849609 seconds
2023/03/13 14:08:46 [DEBUG] [AS3] Response from BIG-IP: code: 200 --- tenant:Common --- message: no change
2023/03/13 14:08:46 [DEBUG] [AS3] Response from BIG-IP: code: 200 --- tenant:pmrke2v1-cluster --- message: success
2023/03/13 14:08:46 [DEBUG] [AS3] Response from BIG-IP: code: 200 --- tenant:Common --- message: no change
2023/03/13 14:08:46 [DEBUG] Updating VirtualServer Status with { Ok} for resource name:istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls , namespace: istio-system
2023/03/13 14:08:46 [DEBUG] Enqueueing VirtualServer: &{{ } {istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls istio-system d14cc57a-4eb8-4fcb-a93d-b4dedc9c69dd 9709708 1 2023-03-13 14:08:13 +0000 UTC <nil> <nil> map[f5cr:true] map[kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"cis.f5.com/v1","kind":"VirtualServer","metadata":{"annotations":{},"labels":{"f5cr":"true"},"name":"istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls","namespace":"istio-system"},"spec":{"allowVlans":["/Common/internal"],"host":"statusdemo.pmilot.ecsdev.entrust.com","iRules":["/Common/Shared/k8s_ingress_sni_irule"],"pools":[{"monitor":{"interval":3,"timeout":10,"type":"tcp"},"path":"/","service":"istio-ingressgateway","servicePort":443}],"snat":"auto","tlsProfileName":"statusdemo-tls","virtualServerAddress":"10.xx.xx.170","virtualServerName":"statusdemo-pmilot-ecsdev-entrust-com-tls"}}
] [] [] [{kubectl-client-side-apply Update cis.f5.com/v1 2023-03-13 14:08:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}},"f:labels":{".":{},"f:f5cr":{}}},"f:spec":{".":{},"f:allowVlans":{},"f:host":{},"f:iRules":{},"f:pools":{},"f:snat":{},"f:tlsProfileName":{},"f:virtualServerAddress":{},"f:virtualServerName":{}}}} {k8s-bigip-ctlr.real Update cis.f5.com/v1 2023-03-13 14:08:46 +0000 UTC FieldsV1 {"f:status":{".":{},"f:status":{},"f:vsAddress":{}}}}]} {statusdemo.pmilot.ecsdev.entrust.com 10.xx.xx.170 [] statusdemo-pmilot-ecsdev-entrust-com-tls 0 0 [{ / istio-ingressgateway {0 443 } {tcp 3 10 0 } [] 0 }] statusdemo-tls auto [/Common/internal] [/Common/Shared/k8s_ingress_sni_irule] [] {{ } [] } [] false } {None Ok}}
2023/03/13 14:08:46 [DEBUG] Processing Key: &{istio-system VirtualServer istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls 0xc0006f8000 Update}
2023/03/13 14:08:46 [DEBUG] Process all the Virtual Servers which share same VirtualServerAddress
2023/03/13 14:08:46 [DEBUG] Processing Virtual Server istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls for port 443
2023/03/13 14:08:46 [DEBUG] Configured rule: {vs_statusdemo_pmilot_ecsdev_entrust_com_istio_ingressgateway_443_istio_system_statusdemo_pmilot_ecsdev_entrust_com statusdemo.pmilot.ecsdev.entrust.com 0 [0xc000949320] [0xc0008c8e10]}
2023/03/13 14:08:46 [DEBUG] Configured policy: {statusdemo_pmilot_ecsdev_entrust_com_tls_443_statusdemo_pmilot_ecsdev_entrust_com_policy istio-system [forwarding] true [http] [0xc000b7c120] /Common/first-match}
2023/03/13 14:08:46 [DEBUG] Processing BIGIP referenced profiles for 'VirtualServer' 'istio-system'/'istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls'
2023/03/13 14:08:46 [DEBUG] Updated BIGIP referenced profiles for 'VirtualServer' 'istio-system'/'istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls'
2023/03/13 14:08:46 [DEBUG] Updated Virtual istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls with TLSProfile statusdemo-tls
2023/03/13 14:08:46 [DEBUG] Finished syncing virtual servers &{TypeMeta:{Kind: APIVersion:} ObjectMeta:{Name:istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls GenerateName: Namespace:istio-system SelfLink: UID:d14cc57a-4eb8-4fcb-a93d-b4dedc9c69dd ResourceVersion:9709708 Generation:1 CreationTimestamp:2023-03-13 14:08:13 +0000 UTC DeletionTimestamp:<nil> DeletionGracePeriodSeconds:<nil> Labels:map[f5cr:true] Annotations:map[kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"cis.f5.com/v1","kind":"VirtualServer","metadata":{"annotations":{},"labels":{"f5cr":"true"},"name":"istio.combined.statusdemo.pmilot.ecsdev.entrust.com.tls","namespace":"istio-system"},"spec":{"allowVlans":["/Common/internal"],"host":"statusdemo.pmilot.ecsdev.entrust.com","iRules":["/Common/Shared/k8s_ingress_sni_irule"],"pools":[{"monitor":{"interval":3,"timeout":10,"type":"tcp"},"path":"/","service":"istio-ingressgateway","servicePort":443}],"snat":"auto","tlsProfileName":"statusdemo-tls","virtualServerAddress":"10.xx.xx.170","virtualServerName":"statusdemo-pmilot-ecsdev-entrust-com-tls"}}
] OwnerReferences:[] Finalizers:[] ClusterName: ManagedFields:[{Manager:kubectl-client-side-apply Operation:Update APIVersion:cis.f5.com/v1 Time:2023-03-13 14:08:13 +0000 UTC FieldsType:FieldsV1 FieldsV1:{"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}},"f:labels":{".":{},"f:f5cr":{}}},"f:spec":{".":{},"f:allowVlans":{},"f:host":{},"f:iRules":{},"f:pools":{},"f:snat":{},"f:tlsProfileName":{},"f:virtualServerAddress":{},"f:virtualServerName":{}}}} {Manager:k8s-bigip-ctlr.real Operation:Update APIVersion:cis.f5.com/v1 Time:2023-03-13 14:08:46 +0000 UTC FieldsType:FieldsV1 FieldsV1:{"f:status":{".":{},"f:status":{},"f:vsAddress":{}}}}]} Spec:{Host:statusdemo.pmilot.ecsdev.entrust.com HostGroup: VirtualServerAddress:10.xx.xx.170 AdditionalVirtualServerAddresses:[] IPAMLabel: VirtualServerName:statusdemo-pmilot-ecsdev-entrust-com-tls VirtualServerHTTPPort:0 VirtualServerHTTPSPort:0 Pools:[{Name: Path:/ Service:istio-ingressgateway ServicePort:{Type:0 IntVal:443 StrVal:} NodeMemberLabel: Monitor:{Type:tcp Send: Recv: Interval:3 Timeout:10 TargetPort:0 Name: Reference:} Monitors:[] Rewrite: Balance: WAF: ServiceNamespace: ReselectTries:0 ServiceDownAction: HostRewrite:}] TLSProfileName:statusdemo-tls HTTPTraffic: SNAT:auto WAF: RewriteAppRoot: AllowVLANs:[/Common/internal] IRules:[/Common/Shared/k8s_ingress_sni_irule] ServiceIPAddress:[] PolicyName: PersistenceProfile: ProfileMultiplex: DOS: BotDefense: Profiles:{TCP:{Client: Server:} UDP: HTTP: HTTP2: RewriteProfile: PersistenceProfile: LogProfiles:[] ProfileL4: ProfileMultiplex:} AllowSourceRange:[] HttpMrfRoutingEnabled:false Partition:} Status:{VSAddress:None StatusOk:Ok}} (180.528µs)
2023/03/13 14:09:05 [DEBUG] [2023-03-13 14:09:05,020 __main__ DEBUG] config handler woken for reset
2023/03/13 14:09:05 [DEBUG] [2023-03-13 14:09:05,021 __main__ DEBUG] loaded configuration file successfully
2023/03/13 14:09:05 [DEBUG] [2023-03-13 14:09:05,021 __main__ DEBUG] NET Config: {}
2023/03/13 14:09:05 [DEBUG] [2023-03-13 14:09:05,022 __main__ DEBUG] loaded configuration file successfully
2023/03/13 14:09:05 [DEBUG] [2023-03-13 14:09:05,022 __main__ DEBUG] updating tasks finished, took 0.0010216236114501953 seconds
2023/03/13 14:09:35 [DEBUG] [2023-03-13 14:09:35,021 __main__ DEBUG] config handler woken for reset
ltm pool istio_ingressgateway_443_istio_system_statusdemo_pmilot_ecsdev_entrust_com { members { /Common/10.xx.xx.64:31390 { address 10.xx.xx.64 metadata { source { value declaration } } } /Common/10.xx.xx.77:31390 { address 10.xx.xx.77 metadata { source { value declaration } } } /Common/10.xx.xx.78:31390 { address 10.xx.xx.78 metadata { source { value declaration } } } /Common/10.xx.xx.79:31390 { address 10.xx.xx.79 metadata { source { value declaration } } } /Common/10.xx.xx.80:31390 { address 10.xx.xx.80 metadata { source { value declaration } } } } min-active-members 1 partition pmrke2v1-cluster }
root@(Lab-BIP-BIGIP-16)(cfg-sync Standalone)(Active)(/pmrke2v1-cluster/Shared)(tmos)# list ltm monitor
root@(Lab-BIP-BIGIP-16)(cfg-sync Standalone)(Active)(/pmrke2v1-cluster/Shared)(tmos)#
@trinaths Are you waiting for anything else from me ? I provided all the logs I can above I think.
I also tried installing CIS using helm in a fresh cluster and the tcp monitor is not being created.
monitors:
items:
properties:
interval:
type: integer
name:
pattern: ^\/[a-zA-Z]+([A-z0-9-_+]+\/)+([-A-z0-9_.:]+\/?)*$
type: string
recv:
type: string
reference:
enum:
- bigip
type: string
send:
type: string
targetPort:
type: integer
timeout:
type: integer
type:
enum:
- http
- https
- tcp
type: string
type: object
type: array
Created [CONTCNTR-3858] for internal tracking.
@trinaths Is their an ETA for this one ? Thank you
@trinaths I was speaking to customer today and he said you could close this. He never tested because he moved to AS3 ConfigMaps after CRD's did not meet his needs. But this is an old issue, and it looks like PR 2910 was intended to fix it.
Setup Details
CIS Version : 2.12.0 Build: f5networks/k8s-bigip-ctlr:latest BIGIP Version: Big IP 16.1.3.2 AS3 Version: 3.42.0 Agent Mode: AS3 Orchestration: K8S Orchestration Version: 1.24.9 Pool Mode: Nodeport
Description
VirtualServer crd does not create the associated monitor when set is set to tcp
Steps To Reproduce
1) Create vs using a tcp monitor type 2) Monitor is not created 3) If I also modify the parameters of the tcp profile the "syncing" log entry records the change but the AS3 declaration sent to the bigip does not include the monitor.
Expected Result
Monitor should be created or modified on change
The CIS logs shoes the monitor but nothing gets created on the bigip and the AS3 status code returned is 200OK