search

Kong / kubernetes-ingress-controller

:gorilla: Kong for Kubernetes: The official Ingress Controller for Kubernetes.
https://docs.konghq.com/kubernetes-ingress-controller/
Apache License 2.0
2.22k stars 592 forks source link

KIC + Konnect with sanitizer enabled continues to update consumer resources #6088

Open tao12345666333 opened 5 months ago

tao12345666333 commented 5 months ago

Is there an existing issue for this?

Current Behavior

When using KIC + Konnect with sanitizer enabled, the consumer will be continuously updated, which will result in continuous requests to Konnect.

The following is the data of the consumer that I obtained by simply refreshing the Konnect page without performing any operations.

{"consumer":{"id":"7fb03465-9587-5b74-af07-260b30881b27"},"created_at":1716705646,"id":"a79bac28-e7b7-411a-b6ff-3d1d2e969908","key":"{vault://dcccdf17-1cc8-4fc8-88bd-df7a8469b239}","tags":["k8s-name:alex-key-auth","k8s-namespace:default","k8s-kind:Secret","k8s-uid:e44ded76-dabd-4237-b67e-b17afe876522","k8s-version:v1"],"updated_at":1716705646}
{"consumer":{"id":"7fb03465-9587-5b74-af07-260b30881b27"},"created_at":1716705910,"id":"7c1377d4-7436-4344-a0a2-6eaf547af3ca","key":"{vault://187fa825-55c0-4339-a057-74fa03b44cc3}","tags":["k8s-name:alex-key-auth","k8s-namespace:default","k8s-kind:Secret","k8s-uid:e44ded76-dabd-4237-b67e-b17afe876522","k8s-version:v1"],"updated_at":1716705910}
{"consumer":{"id":"8fb03465-9587-5b74-af07-260b30881b27"},"created_at":1716707089,"id":"e605b1bb-1fdd-4eec-bf76-a61864378506","key":"{vault://8fff1c8b-c124-4508-ba72-6c25119da520}","tags":["k8s-name:alex-key-auth","k8s-namespace:default","k8s-kind:Secret","k8s-uid:e44ded76-dabd-4237-b67e-b17afe876522","k8s-version:v1"],"updated_at":1716707089}

In addition, I also saw the following content in KIC's debug log.

2024-05-26T06:36:12Z    debug   Successfully built data-plane configuration     {"v": 1}                                                                    14:36:18 [93/3477]
2024-05-26T06:36:12Z    debug   Sending configuration to gateway clients        {"v": 1, "urls": ["https://10.244.1.18:8444"]}                                                
2024-05-26T06:36:12Z    debug   No configuration change, skipping sync to Kong  {"url": "https://10.244.1.18:8444", "v": 1}                                                   
2024-05-26T06:36:12Z    debug   events  successfully applied Kong configuration to https://10.244.1.18:8444     {"v": 1, "type": "Normal", "object": {"kind":"Pod","namespace"
:"kong","name":"kong-controller-699df47f5b-6fvrp","apiVersion":"v1"}, "reason": "KongConfigurationSucceeded"}                                                                 
creating key-auth d2fa} for consumer alex                                                                                                                                     
deleting key-auth a5e8} for consumer alex                                                                                                                                     
2024-05-26T06:36:13Z    info    Successfully synced configuration to Konnect    {"url": "https://KIC-CP-API", "update_strategy": "WithBackoff(DBMode)", "v": 0}                                                                                                                      
2024-05-26T06:36:13Z    debug   No change in config status, not notifying       {"v": 1}                                                                                      
2024-05-26T06:36:13Z    debug   No configuration change; resource status update not necessary, skipping {"v": 1}                                                              
2024-05-26T06:36:15Z    debug   Parsing kubernetes objects into data-plane configuration        {"v": 1}                                                                      
2024-05-26T06:36:15Z    debug   Fetching EndpointSlices {"service_name": "echo", "service_namespace": "default", "service_port": "&ServicePort{Name:http,Protocol:TCP,Port:102
7,TargetPort:{0 1027 },NodePort:0,AppProtocol:nil,}", "service_name": "echo", "service_namespace": "default", "service_port": "&ServicePort{Name:http,Protocol:TCP,Port:1027,T
argetPort:{0 1027 },NodePort:0,AppProtocol:nil,}", "v": 1}                                                                                                                    
2024-05-26T06:36:15Z    debug   Fetched EndpointSlices  {"service_name": "echo", "service_namespace": "default", "service_port": "&ServicePort{Name:http,Protocol:TCP,Port:1027,TargetPort:{0 1027 },NodePort:0,AppProtocol:nil,}", "service_name": "echo", "service_namespace": "default", "service_port": "&ServicePort{Name:http,Protocol:TCP,Port:1027,T
argetPort:{0 1027 },NodePort:0,AppProtocol:nil,}", "v": 1, "count": 1}                                                                                                        
2024-05-26T06:36:15Z    debug   Found endpoints {"service_name": "echo", "service_namespace": "default", "service_port": "&ServicePort{Name:http,Protocol:TCP,Port:1027,Target
Port:{0 1027 },NodePort:0,AppProtocol:nil,}", "service_name": "echo", "service_namespace": "default", "service_port": "&ServicePort{Name:http,Protocol:TCP,Port:1027,TargetPor
t:{0 1027 },NodePort:0,AppProtocol:nil,}", "v": 1, "endpoints": [{"address":"10.244.1.19","port":"1027"}]}                                                                                                                                                                     
2024-05-26T06:36:15Z    debug   license-agent   Retrieving license from cache   {"v": 1}                                                                                      
2024-05-26T06:36:15Z    debug   Successfully built data-plane configuration     {"v": 1}                                                                                      
2024-05-26T06:36:15Z    debug   Sending configuration to gateway clients        {"v": 1, "urls": ["https://10.244.1.18:8444"]}                                                
2024-05-26T06:36:15Z    debug   No configuration change, skipping sync to Kong  {"url": "https://10.244.1.18:8444", "v": 1}                                                   
2024-05-26T06:36:15Z    debug   events  successfully applied Kong configuration to https://10.244.1.18:8444     {"v": 1, "type": "Normal", "object": {"kind":"Pod","namespace"
:"kong","name":"kong-controller-699df47f5b-6fvrp","apiVersion":"v1"}, "reason": "KongConfigurationSucceeded"}                                                                 
creating key-auth fda4} for consumer alex                                                                                                                                     
deleting key-auth d2fa} for consumer alex                                                                                                                                     
2024-05-26T06:36:16Z    info    Successfully synced configuration to Konnect    {"url": "https://KIC-CP-API", "update_strategy": "WithBackoff(DBMode)", "v": 0}                                                                                                                      
2024-05-26T06:36:16Z    debug   No change in config status, not notifying       {"v": 1}                                                                                      
2024-05-26T06:36:16Z    debug   No configuration change; resource status update not necessary, skipping {"v": 1}                                                              
2024-05-26T06:36:18Z    debug   Parsing kubernetes objects into data-plane configuration        {"v": 1}

Expected Behavior

KIC should not continue to perform update operations without modifying the consumer.

Steps To Reproduce

  1. Install KIC with Konnect
  2. Deploy echo deployment & service
kubectl apply -f https://docs.konghq.com/assets/kubernetes-ingress-controller/examples/echo-service.yaml
  1. Create Ingress + KongPlugin(key-auth) + KongConsumer
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    konghq.com/strip-path: "true"
  name: echo
  namespace: default
spec:
  ingressClassName: kong
  rules:
  - http:
      paths:
      - backend:
          service:
            name: echo
            port:
              number: 1027
        path: /echo
        pathType: ImplementationSpecific
---
apiVersion: configuration.konghq.com/v1
config:
  key_names:
  - apikey
kind: KongPlugin
metadata:
  name: key-auth
  namespace: default
plugin: key-auth
---
apiVersion: configuration.konghq.com/v1
credentials:
- alex-key-auth
kind: KongConsumer
metadata:
  annotations:
    kubernetes.io/ingress.class: kong
  name: alex
  namespace: default
username: alex
---
apiVersion: v1
data:
  key: aGVsbG9fd29ybGQ=
kind: Secret
metadata:
  labels:
    konghq.com/credential: key-auth
  name: alex-key-auth
  namespace: default
type: Opaque

Kong Ingress Controller version

v3.1

Kubernetes version

v1.29

Anything else?

No response

lahabana commented 5 months ago

@tao12345666333 Isn't this the result of #5692 ?

tao12345666333 commented 5 months ago

@lahabana I believe they have a certain connection, but it is not the result of it.

This issue is caused by KIC, which discovered inconsistent states after sanitizing the data and resulted in an update.