search

kumahq / kuma

🐻 The multi-zone service mesh for containers, Kubernetes and VMs. Built with Envoy. CNCF Sandbox Project.
https://kuma.io/install
Apache License 2.0
3.64k stars 331 forks source link

MeshGatewayInstance reconciling on a namespace being deleted #9165

Open lahabana opened 8 months ago

lahabana commented 8 months ago

What happened?

No big issue but a bunch of logs

2024-02-07T16:02:59.137Z ERROR kube-manager Reconciler error {"controller": "meshgatewayinstance", "controllerGroup": "kuma.io", "controllerKind": "MeshGatewayInstance", "MeshGatewayInstance": {"name":"simple-gateway","namespace":"simple-gateway"}, "namespace": "simple-gateway", "name": "simple-gateway", "reconcileID": "1ed39361-4ca3-4c37-984f-e6775a6101f5", "error": "unable to reconcile Service for Gateway: couldn't create object: services \"simple-gateway\" is forbidden: unable to create new content in namespace simple-gateway because it is being terminated", "errorVerbose": "services \"simple-gateway\" is forbidden: unable to create new content in namespace simple-gateway because it is being terminated\ncouldn't create object\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers/util.ManageControlledObject\n\tgithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers/util/controlled.go:56\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.(GatewayInstanceReconciler).createOrUpdateService\n\tgithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers/gateway_instance_controller.go:118\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.(GatewayInstanceReconciler).Reconcile\n\tgithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers/gateway_instance_controller.go:74\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.17.0/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.17.0/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.17.0/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.17.0/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_arm64.s:1197\nunable to reconcile Service for Gateway\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.(GatewayInstanceReconciler).Reconcile\n\tgithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers/gateway_instance_controller.go:76\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.17.0/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.17.0/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.17.0/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.17.0/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_arm64.s:1197"}

github-actions[bot] commented 5 months ago

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.

Automaat commented 5 months ago

we didn't see error for MeshGatewayInstance, but there are lots of errors still, we should reduce the number of errors.

2024-05-17T09:27:11.411Z    INFO    xds.status-tracker    config requested    {"proxyName": "redis-54754f5b57-jf2mb.kuma-demo", "mesh": "default", "streamID": 6, "type": "Listener", "resourceVersion": "", "resourceNames": ]}
2024-05-17T09:27:11.490Z    INFO    controllers.Pod    Dataplane updated    {"pod": {"name":"demo-app-fcc8bc4cb-4zqtr","namespace":"kuma-demo"}}
2024-05-17T09:27:11.825Z    INFO    opa-server    configuring OPA agent with a new config    {"nodeID": "default.demo-app-fcc8bc4cb-4zqtr.kuma-demo"}
2024-05-17T09:27:11.826Z    INFO    opa-server    OPA accepted a new config    {"nodeID": "default.demo-app-fcc8bc4cb-4zqtr.kuma-demo"}
2024-05-17T09:27:12.102Z    INFO    controllers.Pod    Dataplane updated    {"pod": {"name":"redis-54754f5b57-jf2mb","namespace":"kuma-demo"}}
2024-05-17T09:27:12.104Z    INFO    xds-server.reconcile    config has changed    {"proxyName": "demo-app-fcc8bc4cb-4zqtr.kuma-demo", "mesh": "default", "versions": ["c19db812-716a-4fe7-b016-a55747c7d3ac", "50ba9425-6f51-4ed9-9470-e9b642f55b26", "b10b28ef-45a6-4376-8a2e-df52167311d5"]}
2024-05-17T09:27:12.216Z    INFO    controllers.Pod    Dataplane created    {"pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}}
2024-05-17T09:27:12.302Z    INFO    kds-mux-client    sending health check    {"client-id": "zone-1", "rpc": "healthcheck"}
2024-05-17T09:27:12.389Z    INFO    xds.status-tracker    config requested    {"proxyName": "demo-app-fcc8bc4cb-4zqtr.kuma-demo", "mesh": "default", "streamID": 7, "type": "ClusterLoadAssignment", "resourceVersion": "", "resourceNames": ["redis_kuma-demo_svc_6379", "demo-app_kuma-demo_svc_5000"]}
2024-05-17T09:27:12.391Z    INFO    xds.status-tracker    config requested    {"proxyName": "demo-app-fcc8bc4cb-4zqtr.kuma-demo", "mesh": "default", "streamID": 7, "type": "Listener", "resourceVersion": "", "resourceNames": ]}
2024-05-17T09:27:12.708Z    INFO    kds-zone    detected changes in the resources. Sending changes to the client.    {"streamID": 1, "nodeID": "zone-1", "resourceType": "DataplaneInsight", "client": "global"}
2024-05-17T09:27:12.708Z    INFO    kds-zone    detected changes in the resources. Sending changes to the client.    {"streamID": 1, "nodeID": "zone-1", "resourceType": "Dataplane", "client": "global"}
2024-05-17T09:27:13.031Z    INFO    xds-server.reconcile    config has changed    {"proxyName": "redis-54754f5b57-jf2mb.kuma-demo", "mesh": "default", "versions": ["4b6f7eeb-c802-4360-ba0f-1165a753cf3a", "09ca0735-f33e-40fa-86b7-d9bbccf22316"]}
2024-05-17T09:27:13.059Z    INFO    controllers.Pod    Dataplane updated    {"pod": {"name":"demo-app-fcc8bc4cb-4zqtr","namespace":"kuma-demo"}}
2024-05-17T09:27:13.879Z    INFO    xds-server.reconcile    config has changed    {"proxyName": "kong-mesh-ingress-9c8b57df6-nqdj6.kong-mesh-system", "mesh": "", "versions": ["666ac109-5823-425b-b470-cdfc9800a654", "8a4cd435-e80b-489d-8586-648a73b123eb", "9b37d582-6b8e-457a-8da3-2ddf6fd462a7"]}
2024-05-17T09:27:13.889Z    INFO    xds-server.reconcile    config has changed    {"proxyName": "demo-app-fcc8bc4cb-4zqtr.kuma-demo", "mesh": "default", "versions": ["41090bb6-7995-49d4-8636-9185c15884c8", "01bc65d6-cec6-4c5c-aa6f-1f10afc418c2"]}
2024-05-17T09:27:14.033Z    INFO    xds-server.reconcile    config has changed    {"proxyName": "redis-54754f5b57-jf2mb.kuma-demo", "mesh": "default", "versions": ["7f220e74-2fbc-4e46-9c61-a8c539aa9689", "bfe558b8-406c-4665-af48-04c0845ee1df", "4f35b4e2-5248-4143-8bee-4fe36b06ad90"]}
2024-05-17T09:27:14.706Z    INFO    kds-zone    detected changes in the resources. Sending changes to the client.    {"streamID": 1, "nodeID": "zone-1", "resourceType": "Dataplane", "client": "global"}
2024-05-17T09:27:14.706Z    INFO    kds-zone    detected changes in the resources. Sending changes to the client.    {"streamID": 1, "nodeID": "zone-1", "resourceType": "ZoneIngress", "client": "global"}
2024-05-17T09:27:14.881Z    INFO    xds-server.reconcile    config has changed    {"proxyName": "kong-mesh-ingress-9c8b57df6-nqdj6.kong-mesh-system", "mesh": "", "versions": ["5f062fb7-e206-45af-8fa7-a917a10777a2", "01839323-071e-4d28-9fd8-02883215bf29", "32313a69-32d8-4308-8bb2-b73e2ff50817"]}
2024-05-17T09:27:14.890Z    INFO    xds-server.reconcile    config has changed    {"proxyName": "demo-app-fcc8bc4cb-4zqtr.kuma-demo", "mesh": "default", "versions": ["21ba7308-001d-4ae5-a329-a30dc848ce83", "66fd995f-9ba4-4d00-94d3-f9330f80750d", "6f02f890-037a-493a-98fb-101c5817795d"]}
2024-05-17T09:27:15.234Z    INFO    xds.status-tracker    proxy connected    {"proxyName": "demo-app-gateway-778b587868-tg6x7.kuma-demo", "mesh": "default", "streamID": 8, "type": "dataplane", "dpVersion": "2.7.2", "subscriptionID": "053ec635-4c5b-460d-b74f-5db936ac242f"}
2024-05-17T09:27:15.234Z    INFO    xds.status-tracker    config requested    {"proxyName": "demo-app-gateway-778b587868-tg6x7.kuma-demo", "mesh": "default", "streamID": 8, "type": "Runtime", "resourceVersion": "", "resourceNames": ["gateway.listeners"]}
2024-05-17T09:27:15.706Z    INFO    kds-zone    detected changes in the resources. Sending changes to the client.    {"streamID": 1, "nodeID": "zone-1", "resourceType": "ZoneIngress", "client": "global"}
2024-05-17T09:27:16.120Z    INFO    opa-server    configuring OPA agent with a new config    {"nodeID": "default.demo-app-gateway-778b587868-tg6x7.kuma-demo"}
2024-05-17T09:27:16.121Z    INFO    opa-server    OPA accepted a new config    {"nodeID": "default.demo-app-gateway-778b587868-tg6x7.kuma-demo"}
2024-05-17T09:27:16.345Z    INFO    xds-server.reconcile    config has changed    {"proxyName": "demo-app-gateway-778b587868-tg6x7.kuma-demo", "mesh": "default", "versions": ["2677c88d-a763-47ab-80b1-d7d826a3e83f", "760849f2-43d6-450c-9b78-f0a74318e2ca", "8be14f7f-e234-4de3-8f97-0ffba6fe0dce", "99ed7468-91d9-4138-b1f5-0ee5451e0e02", "587c1205-4918-4c7a-b168-9c72451b94b1"]}
2024-05-17T09:27:16.348Z    INFO    xds.status-tracker    config requested    {"proxyName": "demo-app-gateway-778b587868-tg6x7.kuma-demo", "mesh": "default", "streamID": 8, "type": "Cluster", "resourceVersion": "", "resourceNames": ]}
2024-05-17T09:27:16.453Z    INFO    xds.status-tracker    config requested    {"proxyName": "demo-app-gateway-778b587868-tg6x7.kuma-demo", "mesh": "default", "streamID": 8, "type": "ClusterLoadAssignment", "resourceVersion": "", "resourceNames": ["demo-app_kuma-demo_svc_5000-73793edfc5fba939"]}
2024-05-17T09:27:16.455Z    INFO    xds.status-tracker    config requested    {"proxyName": "demo-app-gateway-778b587868-tg6x7.kuma-demo", "mesh": "default", "streamID": 8, "type": "Listener", "resourceVersion": "", "resourceNames": ]}
2024-05-17T09:27:16.599Z    INFO    xds.status-tracker    config requested    {"proxyName": "demo-app-gateway-778b587868-tg6x7.kuma-demo", "mesh": "default", "streamID": 8, "type": "RouteConfiguration", "resourceVersion": "", "resourceNames": ["demo-app:HTTP:80:*"]}
2024-05-17T09:27:16.708Z    INFO    kds-zone    detected changes in the resources. Sending changes to the client.    {"streamID": 1, "nodeID": "zone-1", "resourceType": "DataplaneInsight", "client": "global"}
2024-05-17T09:27:16.708Z    INFO    kds-zone    detected changes in the resources. Sending changes to the client.    {"streamID": 1, "nodeID": "zone-1", "resourceType": "ZoneIngressInsight", "client": "global"}
2024-05-17T09:27:20.707Z    INFO    kds-zone    detected changes in the resources. Sending changes to the client.    {"streamID": 1, "nodeID": "zone-1", "resourceType": "DataplaneInsight", "client": "global"}
2024-05-17T09:27:21.706Z    INFO    kds-zone    detected changes in the resources. Sending changes to the client.    {"streamID": 1, "nodeID": "zone-1", "resourceType": "DataplaneInsight", "client": "global"}
2024-05-17T09:27:25.707Z    INFO    kds-zone    detected changes in the resources. Sending changes to the client.    {"streamID": 1, "nodeID": "zone-1", "resourceType": "DataplaneInsight", "client": "global"}
2024-05-17T09:27:51.329Z    ERROR    discovery.k8s.pod-to-dataplane-converter    could not get K8S Service for service tag    {"error": "failed to get Service \"kuma-demo/demo-app\": Service \"demo-app\" not found", "errorVerbose": "Service \"demo-app\" not found\nfailed to get Service \"kuma-demo/demo-app\"\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.k8sService\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/outbound_converter.go:126\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.KubeHosts\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/kube_hosts_converter.go:30\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.(*ConfigMapReconciler).Reconcile\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/configmap_controller.go:55\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024-05-17T09:27:51.329Z    ERROR    discovery.k8s.pod-to-dataplane-converter    could not get K8S Service for service tag    {"error": "failed to get Service \"kuma-demo/redis\": Service \"redis\" not found", "errorVerbose": "Service \"redis\" not found\nfailed to get Service \"kuma-demo/redis\"\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.k8sService\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/outbound_converter.go:126\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.KubeHosts\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/kube_hosts_converter.go:30\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.(*ConfigMapReconciler).Reconcile\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/configmap_controller.go:55\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024-05-17T09:27:51.330Z    INFO    dns-vips-allocator    mesh VIPs changed    {"mesh": "default", "changes": [{"Type":"Remove","Entry":"1:10.113.68.15"},{"Type":"Remove","Entry":"1:10.113.74.94"}]}
2024-05-17T09:27:51.360Z    ERROR    discovery.k8s.pod-to-dataplane-converter    could not get K8S Service for service tag    {"error": "failed to get Service \"kuma-demo/demo-app\": Service \"demo-app\" not found", "errorVerbose": "Service \"demo-app\" not found\nfailed to get Service \"kuma-demo/demo-app\"\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.k8sService\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/outbound_converter.go:126\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.KubeHosts\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/kube_hosts_converter.go:30\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.(*ConfigMapReconciler).Reconcile\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/configmap_controller.go:55\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024-05-17T09:27:51.360Z    ERROR    discovery.k8s.pod-to-dataplane-converter    could not get K8S Service for service tag    {"error": "failed to get Service \"kuma-demo/redis\": Service \"redis\" not found", "errorVerbose": "Service \"redis\" not found\nfailed to get Service \"kuma-demo/redis\"\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.k8sService\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/outbound_converter.go:126\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.KubeHosts\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/kube_hosts_converter.go:30\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.(*ConfigMapReconciler).Reconcile\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/configmap_controller.go:55\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024-05-17T09:27:51.362Z    ERROR    discovery.k8s.pod-to-dataplane-converter    could not get K8S Service for service tag    {"error": "failed to get Service \"kuma-demo/demo-app\": Service \"demo-app\" not found", "errorVerbose": "Service \"demo-app\" not found\nfailed to get Service \"kuma-demo/demo-app\"\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.k8sService\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/outbound_converter.go:126\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.KubeHosts\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/kube_hosts_converter.go:30\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.(*ConfigMapReconciler).Reconcile\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/configmap_controller.go:55\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024-05-17T09:27:51.362Z    ERROR    discovery.k8s.pod-to-dataplane-converter    could not get K8S Service for service tag    {"error": "failed to get Service \"kuma-demo/redis\": Service \"redis\" not found", "errorVerbose": "Service \"redis\" not found\nfailed to get Service \"kuma-demo/redis\"\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.k8sService\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/outbound_converter.go:126\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.KubeHosts\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/kube_hosts_converter.go:30\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.(*ConfigMapReconciler).Reconcile\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/configmap_controller.go:55\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024-05-17T09:27:51.891Z    INFO    xds-server.reconcile    config has changed    {"proxyName": "demo-app-fcc8bc4cb-4zqtr.kuma-demo", "mesh": "default", "versions": ["39125791-133a-47aa-9f75-c7799aacb339"]}
2024/05/17 09:27:51 http: TLS handshake error from 10.64.2.3:36176: EOF
2024-05-17T09:27:52.030Z    INFO    xds-server.reconcile    config has changed    {"proxyName": "redis-54754f5b57-jf2mb.kuma-demo", "mesh": "default", "versions": ["72cbd821-c423-4473-aa10-d97cbacf53e4"]}
2024-05-17T09:27:52.707Z    INFO    kds-zone    detected changes in the resources. Sending changes to the client.    {"streamID": 1, "nodeID": "zone-1", "resourceType": "DataplaneInsight", "client": "global"}
2024/05/17 09:27:52 http: TLS handshake error from 10.64.1.2:46248: EOF
2024/05/17 09:27:53 http: TLS handshake error from 10.64.1.2:46260: EOF
2024/05/17 09:27:54 http: TLS handshake error from 10.64.2.3:36184: EOF
2024/05/17 09:27:55 http: TLS handshake error from 10.64.1.2:50490: EOF
2024/05/17 09:27:56 http: TLS handshake error from 10.64.2.3:36200: EOF
2024/05/17 09:27:58 http: TLS handshake error from 10.64.1.2:50492: EOF
2024/05/17 09:27:59 http: TLS handshake error from 10.64.0.9:35046: EOF
2024-05-17T09:28:00.032Z    ERROR    xds.sink    failed to flush DataplaneInsight    {"dataplaneid": {"Mesh":"default","Name":"redis-54754f5b57-jf2mb.kuma-demo"}, "error": "failed to create k8s resource: dataplaneinsights.kuma.io \"redis-54754f5b57-jf2mb\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated", "errorVerbose": "dataplaneinsights.kuma.io \"redis-54754f5b57-jf2mb\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated\nfailed to create k8s resource\ngithub.com/kumahq/kuma/pkg/plugins/resources/k8s.(*KubernetesStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/resources/k8s/store.go:82\ngithub.com/kumahq/kuma/pkg/core/resources/store.(*paginationStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/pagination_store.go:30\ngithub.com/kumahq/kuma/pkg/metrics/store.(*MeteredStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/metrics/store/store.go:44\ngithub.com/Kong/kong-mesh/pkg/plugins/resources/limits.(*LimitsStore).Create\n\tgithub.com/Kong/kong-mesh/pkg/plugins/resources/limits/limits_store.go:91\ngithub.com/kumahq/kuma/pkg/core/resources/store.(*customizableResourceStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/customizable_store.go:44\ngithub.com/kumahq/kuma/pkg/core/managers/apis/dataplaneinsight.(*dataplaneInsightManager).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/managers/apis/dataplaneinsight/dataplane_insight_manager.go:41\ngithub.com/kumahq/kuma/pkg/core/resources/manager.(*customizableResourceManager).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/customizable_manager.go:51\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert.Upsert.func1.func3\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:161\ngithub.com/kumahq/kuma/pkg/core/resources/store.InTx\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/transactions.go:46\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert.func1\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:144\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:169\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightStore).Upsert\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:163\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightSink).Start.func1\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:93\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightSink).Start\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:127\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024-05-17T09:28:00.891Z    ERROR    xds.sink    failed to flush DataplaneInsight    {"dataplaneid": {"Mesh":"default","Name":"demo-app-fcc8bc4cb-4zqtr.kuma-demo"}, "error": "failed to create k8s resource: dataplaneinsights.kuma.io \"demo-app-fcc8bc4cb-4zqtr\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated", "errorVerbose": "dataplaneinsights.kuma.io \"demo-app-fcc8bc4cb-4zqtr\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated\nfailed to create k8s resource\ngithub.com/kumahq/kuma/pkg/plugins/resources/k8s.(*KubernetesStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/resources/k8s/store.go:82\ngithub.com/kumahq/kuma/pkg/core/resources/store.(*paginationStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/pagination_store.go:30\ngithub.com/kumahq/kuma/pkg/metrics/store.(*MeteredStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/metrics/store/store.go:44\ngithub.com/Kong/kong-mesh/pkg/plugins/resources/limits.(*LimitsStore).Create\n\tgithub.com/Kong/kong-mesh/pkg/plugins/resources/limits/limits_store.go:91\ngithub.com/kumahq/kuma/pkg/core/resources/store.(*customizableResourceStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/customizable_store.go:44\ngithub.com/kumahq/kuma/pkg/core/managers/apis/dataplaneinsight.(*dataplaneInsightManager).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/managers/apis/dataplaneinsight/dataplane_insight_manager.go:41\ngithub.com/kumahq/kuma/pkg/core/resources/manager.(*customizableResourceManager).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/customizable_manager.go:51\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert.Upsert.func1.func3\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:161\ngithub.com/kumahq/kuma/pkg/core/resources/store.InTx\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/transactions.go:46\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert.func1\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:144\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:169\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightStore).Upsert\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:163\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightSink).Start.func1\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:93\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightSink).Start\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:127\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024/05/17 09:28:01 http: TLS handshake error from 10.64.1.2:50500: EOF
2024/05/17 09:28:04 http: TLS handshake error from 10.64.2.3:40144: EOF
2024-05-17T09:28:10.034Z    ERROR    xds.sink    failed to flush DataplaneInsight    {"dataplaneid": {"Mesh":"default","Name":"redis-54754f5b57-jf2mb.kuma-demo"}, "error": "failed to create k8s resource: dataplaneinsights.kuma.io \"redis-54754f5b57-jf2mb\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated", "errorVerbose": "dataplaneinsights.kuma.io \"redis-54754f5b57-jf2mb\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated\nfailed to create k8s resource\ngithub.com/kumahq/kuma/pkg/plugins/resources/k8s.(*KubernetesStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/resources/k8s/store.go:82\ngithub.com/kumahq/kuma/pkg/core/resources/store.(*paginationStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/pagination_store.go:30\ngithub.com/kumahq/kuma/pkg/metrics/store.(*MeteredStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/metrics/store/store.go:44\ngithub.com/Kong/kong-mesh/pkg/plugins/resources/limits.(*LimitsStore).Create\n\tgithub.com/Kong/kong-mesh/pkg/plugins/resources/limits/limits_store.go:91\ngithub.com/kumahq/kuma/pkg/core/resources/store.(*customizableResourceStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/customizable_store.go:44\ngithub.com/kumahq/kuma/pkg/core/managers/apis/dataplaneinsight.(*dataplaneInsightManager).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/managers/apis/dataplaneinsight/dataplane_insight_manager.go:41\ngithub.com/kumahq/kuma/pkg/core/resources/manager.(*customizableResourceManager).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/customizable_manager.go:51\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert.Upsert.func1.func3\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:161\ngithub.com/kumahq/kuma/pkg/core/resources/store.InTx\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/transactions.go:46\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert.func1\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:144\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:169\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightStore).Upsert\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:163\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightSink).Start.func1\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:93\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightSink).Start\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:127\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024/05/17 09:28:10 http: TLS handshake error from 10.64.2.3:41868: EOF
2024-05-17T09:28:10.891Z    ERROR    xds.sink    failed to flush DataplaneInsight    {"dataplaneid": {"Mesh":"default","Name":"demo-app-fcc8bc4cb-4zqtr.kuma-demo"}, "error": "failed to create k8s resource: dataplaneinsights.kuma.io \"demo-app-fcc8bc4cb-4zqtr\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated", "errorVerbose": "dataplaneinsights.kuma.io \"demo-app-fcc8bc4cb-4zqtr\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated\nfailed to create k8s resource\ngithub.com/kumahq/kuma/pkg/plugins/resources/k8s.(*KubernetesStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/resources/k8s/store.go:82\ngithub.com/kumahq/kuma/pkg/core/resources/store.(*paginationStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/pagination_store.go:30\ngithub.com/kumahq/kuma/pkg/metrics/store.(*MeteredStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/metrics/store/store.go:44\ngithub.com/Kong/kong-mesh/pkg/plugins/resources/limits.(*LimitsStore).Create\n\tgithub.com/Kong/kong-mesh/pkg/plugins/resources/limits/limits_store.go:91\ngithub.com/kumahq/kuma/pkg/core/resources/store.(*customizableResourceStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/customizable_store.go:44\ngithub.com/kumahq/kuma/pkg/core/managers/apis/dataplaneinsight.(*dataplaneInsightManager).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/managers/apis/dataplaneinsight/dataplane_insight_manager.go:41\ngithub.com/kumahq/kuma/pkg/core/resources/manager.(*customizableResourceManager).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/customizable_manager.go:51\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert.Upsert.func1.func3\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:161\ngithub.com/kumahq/kuma/pkg/core/resources/store.InTx\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/transactions.go:46\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert.func1\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:144\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:169\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightStore).Upsert\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:163\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightSink).Start.func1\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:93\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightSink).Start\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:127\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024-05-17T09:28:12.355Z    INFO    kds-mux-client    sending health check    {"client-id": "zone-1", "rpc": "healthcheck"}
2024-05-17T09:28:13.171Z    ERROR    discovery.k8s.pod-to-dataplane-converter    could not get K8S Service for service tag    {"error": "failed to get Service \"kuma-demo/demo-app\": Service \"demo-app\" not found", "errorVerbose": "Service \"demo-app\" not found\nfailed to get Service \"kuma-demo/demo-app\"\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.k8sService\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/outbound_converter.go:126\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.KubeHosts\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/kube_hosts_converter.go:30\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.(*ConfigMapReconciler).Reconcile\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/configmap_controller.go:55\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024-05-17T09:28:13.171Z    ERROR    discovery.k8s.pod-to-dataplane-converter    could not get K8S Service for service tag    {"error": "failed to get Service \"kuma-demo/redis\": Service \"redis\" not found", "errorVerbose": "Service \"redis\" not found\nfailed to get Service \"kuma-demo/redis\"\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.k8sService\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/outbound_converter.go:126\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.KubeHosts\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/kube_hosts_converter.go:30\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.(*ConfigMapReconciler).Reconcile\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/configmap_controller.go:55\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024-05-17T09:28:20.030Z    ERROR    xds.sink    failed to flush DataplaneInsight    {"dataplaneid": {"Mesh":"default","Name":"redis-54754f5b57-jf2mb.kuma-demo"}, "error": "failed to create k8s resource: dataplaneinsights.kuma.io \"redis-54754f5b57-jf2mb\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated", "errorVerbose": "dataplaneinsights.kuma.io \"redis-54754f5b57-jf2mb\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated\nfailed to create k8s resource\ngithub.com/kumahq/kuma/pkg/plugins/resources/k8s.(*KubernetesStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/resources/k8s/store.go:82\ngithub.com/kumahq/kuma/pkg/core/resources/store.(*paginationStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/pagination_store.go:30\ngithub.com/kumahq/kuma/pkg/metrics/store.(*MeteredStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/metrics/store/store.go:44\ngithub.com/Kong/kong-mesh/pkg/plugins/resources/limits.(*LimitsStore).Create\n\tgithub.com/Kong/kong-mesh/pkg/plugins/resources/limits/limits_store.go:91\ngithub.com/kumahq/kuma/pkg/core/resources/store.(*customizableResourceStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/customizable_store.go:44\ngithub.com/kumahq/kuma/pkg/core/managers/apis/dataplaneinsight.(*dataplaneInsightManager).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/managers/apis/dataplaneinsight/dataplane_insight_manager.go:41\ngithub.com/kumahq/kuma/pkg/core/resources/manager.(*customizableResourceManager).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/customizable_manager.go:51\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert.Upsert.func1.func3\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:161\ngithub.com/kumahq/kuma/pkg/core/resources/store.InTx\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/transactions.go:46\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert.func1\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:144\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:169\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightStore).Upsert\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:163\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightSink).Start.func1\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:93\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightSink).Start\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:127\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024-05-17T09:28:20.891Z    ERROR    xds.sink    failed to flush DataplaneInsight    {"dataplaneid": {"Mesh":"default","Name":"demo-app-fcc8bc4cb-4zqtr.kuma-demo"}, "error": "failed to create k8s resource: dataplaneinsights.kuma.io \"demo-app-fcc8bc4cb-4zqtr\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated", "errorVerbose": "dataplaneinsights.kuma.io \"demo-app-fcc8bc4cb-4zqtr\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated\nfailed to create k8s resource\ngithub.com/kumahq/kuma/pkg/plugins/resources/k8s.(*KubernetesStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/resources/k8s/store.go:82\ngithub.com/kumahq/kuma/pkg/core/resources/store.(*paginationStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/pagination_store.go:30\ngithub.com/kumahq/kuma/pkg/metrics/store.(*MeteredStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/metrics/store/store.go:44\ngithub.com/Kong/kong-mesh/pkg/plugins/resources/limits.(*LimitsStore).Create\n\tgithub.com/Kong/kong-mesh/pkg/plugins/resources/limits/limits_store.go:91\ngithub.com/kumahq/kuma/pkg/core/resources/store.(*customizableResourceStore).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/customizable_store.go:44\ngithub.com/kumahq/kuma/pkg/core/managers/apis/dataplaneinsight.(*dataplaneInsightManager).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/managers/apis/dataplaneinsight/dataplane_insight_manager.go:41\ngithub.com/kumahq/kuma/pkg/core/resources/manager.(*customizableResourceManager).Create\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/customizable_manager.go:51\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert.Upsert.func1.func3\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:161\ngithub.com/kumahq/kuma/pkg/core/resources/store.InTx\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/store/transactions.go:46\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert.func1\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:144\ngithub.com/kumahq/kuma/pkg/core/resources/manager.Upsert\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/core/resources/manager/manager.go:169\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightStore).Upsert\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:163\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightSink).Start.func1\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:93\ngithub.com/kumahq/kuma/pkg/xds/server/callbacks.(*dataplaneInsightSink).Start\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/xds/server/callbacks/dataplane_status_sink.go:127\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024-05-17T09:28:21.067Z    INFO    xds.status-tracker    proxy disconnected    {"streamID": 8, "proxyName": "demo-app-gateway-778b587868-tg6x7.kuma-demo", "mesh": "default", "subscriptionID": "053ec635-4c5b-460d-b74f-5db936ac242f"}
2024-05-17T09:28:21.077Z    INFO    xds.status-tracker    proxy disconnected    {"streamID": 6, "proxyName": "redis-54754f5b57-jf2mb.kuma-demo", "mesh": "default", "subscriptionID": "c79c64c8-cbeb-46db-aa47-8673714fedee"}
2024-05-17T09:28:21.094Z    INFO    xds.status-tracker    proxy disconnected    {"streamID": 7, "proxyName": "demo-app-fcc8bc4cb-4zqtr.kuma-demo", "mesh": "default", "subscriptionID": "47a3a58c-8f11-4b3a-8502-fcb1d2067b62"}
2024-05-17T09:28:21.366Z    INFO    controllers.Pod    Object deleted    {"pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "cause": "pod IP is empty", "kind": "&TypeMeta{Kind:,APIVersion:,}", "name": "demo-app-gateway-778b587868-tg6x7", "namespace": "kuma-demo"}
2024-05-17T09:28:21.368Z    ERROR    discovery.k8s.pod-to-dataplane-converter    could not get K8S Service for service tag    {"error": "failed to get Service \"kuma-demo/demo-app\": Service \"demo-app\" not found", "errorVerbose": "Service \"demo-app\" not found\nfailed to get Service \"kuma-demo/demo-app\"\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.k8sService\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/outbound_converter.go:126\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.KubeHosts\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/kube_hosts_converter.go:30\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.(*ConfigMapReconciler).Reconcile\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/configmap_controller.go:55\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024-05-17T09:28:21.368Z    ERROR    discovery.k8s.pod-to-dataplane-converter    could not get K8S Service for service tag    {"error": "failed to get Service \"kuma-demo/redis\": Service \"redis\" not found", "errorVerbose": "Service \"redis\" not found\nfailed to get Service \"kuma-demo/redis\"\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.k8sService\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/outbound_converter.go:126\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.KubeHosts\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/kube_hosts_converter.go:30\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.(*ConfigMapReconciler).Reconcile\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/configmap_controller.go:55\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024-05-17T09:28:21.433Z    ERROR    controllers.Pod    unable to create/update Dataplane    {"pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "operationResult": "unchanged", "error": "dataplanes.kuma.io \"demo-app-gateway-778b587868-tg6x7\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated"}
2024-05-17T09:28:21.434Z    ERROR    kube-manager    Reconciler error    {"controller": "pod", "controllerGroup": "", "controllerKind": "Pod", "Pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "namespace": "kuma-demo", "name": "demo-app-gateway-778b587868-tg6x7", "reconcileID": "c92a1788-1fcf-4316-8d3f-ddad8321a445", "error": "dataplanes.kuma.io \"demo-app-gateway-778b587868-tg6x7\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated"}
2024-05-17T09:28:21.444Z    ERROR    controllers.Pod    unable to create/update Dataplane    {"pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "operationResult": "unchanged", "error": "dataplanes.kuma.io \"demo-app-gateway-778b587868-tg6x7\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated"}
2024-05-17T09:28:21.444Z    ERROR    kube-manager    Reconciler error    {"controller": "pod", "controllerGroup": "", "controllerKind": "Pod", "Pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "namespace": "kuma-demo", "name": "demo-app-gateway-778b587868-tg6x7", "reconcileID": "713c155a-dd06-448c-92bc-55963476a580", "error": "dataplanes.kuma.io \"demo-app-gateway-778b587868-tg6x7\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated"}
2024-05-17T09:28:21.459Z    ERROR    controllers.Pod    unable to create/update Dataplane    {"pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "operationResult": "unchanged", "error": "dataplanes.kuma.io \"demo-app-gateway-778b587868-tg6x7\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated"}
2024-05-17T09:28:21.459Z    ERROR    kube-manager    Reconciler error    {"controller": "pod", "controllerGroup": "", "controllerKind": "Pod", "Pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "namespace": "kuma-demo", "name": "demo-app-gateway-778b587868-tg6x7", "reconcileID": "08eb24a5-09cb-424b-82dc-0afe0ea5cb6f", "error": "dataplanes.kuma.io \"demo-app-gateway-778b587868-tg6x7\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated"}
2024-05-17T09:28:21.484Z    ERROR    controllers.Pod    unable to create/update Dataplane    {"pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "operationResult": "unchanged", "error": "dataplanes.kuma.io \"demo-app-gateway-778b587868-tg6x7\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated"}
2024-05-17T09:28:21.484Z    ERROR    kube-manager    Reconciler error    {"controller": "pod", "controllerGroup": "", "controllerKind": "Pod", "Pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "namespace": "kuma-demo", "name": "demo-app-gateway-778b587868-tg6x7", "reconcileID": "15bbef6b-dc5b-43e2-bac1-659657acae4e", "error": "dataplanes.kuma.io \"demo-app-gateway-778b587868-tg6x7\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated"}
2024-05-17T09:28:21.529Z    ERROR    controllers.Pod    unable to create/update Dataplane    {"pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "operationResult": "unchanged", "error": "dataplanes.kuma.io \"demo-app-gateway-778b587868-tg6x7\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated"}
2024-05-17T09:28:21.529Z    ERROR    kube-manager    Reconciler error    {"controller": "pod", "controllerGroup": "", "controllerKind": "Pod", "Pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "namespace": "kuma-demo", "name": "demo-app-gateway-778b587868-tg6x7", "reconcileID": "93426958-5d1a-45e3-b99f-5254839a94da", "error": "dataplanes.kuma.io \"demo-app-gateway-778b587868-tg6x7\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated"}
2024-05-17T09:28:21.614Z    ERROR    controllers.Pod    unable to create/update Dataplane    {"pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "operationResult": "unchanged", "error": "dataplanes.kuma.io \"demo-app-gateway-778b587868-tg6x7\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated"}
2024-05-17T09:28:21.614Z    ERROR    kube-manager    Reconciler error    {"controller": "pod", "controllerGroup": "", "controllerKind": "Pod", "Pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "namespace": "kuma-demo", "name": "demo-app-gateway-778b587868-tg6x7", "reconcileID": "fea09e20-cb54-4a8a-82aa-c71e7e4e7d0e", "error": "dataplanes.kuma.io \"demo-app-gateway-778b587868-tg6x7\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated"}
2024-05-17T09:28:21.707Z    INFO    kds-zone    detected changes in the resources. Sending changes to the client.    {"streamID": 1, "nodeID": "zone-1", "resourceType": "Dataplane", "client": "global"}
2024-05-17T09:28:21.771Z    ERROR    controllers.Pod    unable to create/update Dataplane    {"pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "operationResult": "unchanged", "error": "dataplanes.kuma.io \"demo-app-gateway-778b587868-tg6x7\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated"}
2024-05-17T09:28:21.771Z    ERROR    kube-manager    Reconciler error    {"controller": "pod", "controllerGroup": "", "controllerKind": "Pod", "Pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "namespace": "kuma-demo", "name": "demo-app-gateway-778b587868-tg6x7", "reconcileID": "a6840e55-6a81-4bad-a0c6-ffe55e16b958", "error": "dataplanes.kuma.io \"demo-app-gateway-778b587868-tg6x7\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated"}
2024-05-17T09:28:21.787Z    ERROR    controllers.Pod    unable to create/update Dataplane    {"pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "operationResult": "unchanged", "error": "dataplanes.kuma.io \"demo-app-gateway-778b587868-tg6x7\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated"}
2024-05-17T09:28:21.787Z    ERROR    kube-manager    Reconciler error    {"controller": "pod", "controllerGroup": "", "controllerKind": "Pod", "Pod": {"name":"demo-app-gateway-778b587868-tg6x7","namespace":"kuma-demo"}, "namespace": "kuma-demo", "name": "demo-app-gateway-778b587868-tg6x7", "reconcileID": "74a564aa-d540-4e69-a0ec-12d53dbe9197", "error": "dataplanes.kuma.io \"demo-app-gateway-778b587868-tg6x7\" is forbidden: unable to create new content in namespace kuma-demo because it is being terminated"}
2024-05-17T09:28:21.863Z    INFO    rbac    access granted    {"access": "DELETE", "key": {"Mesh":"default","Name":"redis-54754f5b57-jf2mb.kuma-demo"}, "resourceType": "Dataplane", "user": "system:serviceaccount:kube-system:generic-garbage-collector/system:serviceaccounts,system:serviceaccounts:kube-system,system:authenticated"}
2024-05-17T09:28:21.884Z    INFO    rbac    access granted    {"access": "DELETE", "key": {"Mesh":"default","Name":"demo-app-fcc8bc4cb-4zqtr.kuma-demo"}, "resourceType": "Dataplane", "user": "system:serviceaccount:kube-system:generic-garbage-collector/system:serviceaccounts,system:serviceaccounts:kube-system,system:authenticated"}
2024-05-17T09:28:21.886Z    ERROR    discovery.k8s.pod-to-dataplane-converter    could not get K8S Service for service tag    {"error": "failed to get Service \"kuma-demo/demo-app\": Service \"demo-app\" not found", "errorVerbose": "Service \"demo-app\" not found\nfailed to get Service \"kuma-demo/demo-app\"\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.k8sService\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/outbound_converter.go:126\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.KubeHosts\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/kube_hosts_converter.go:30\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.(*ConfigMapReconciler).Reconcile\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/configmap_controller.go:55\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024-05-17T09:28:21.887Z    INFO    dns-vips-allocator    mesh VIPs changed    {"mesh": "default", "changes": [{"Type":"Remove","Entry":"0:redis_kuma-demo_svc_6379"}]}
2024-05-17T09:28:21.906Z    ERROR    discovery.k8s.pod-to-dataplane-converter    could not get K8S Service for service tag    {"error": "failed to get Service \"kuma-demo/demo-app\": Service \"demo-app\" not found", "errorVerbose": "Service \"demo-app\" not found\nfailed to get Service \"kuma-demo/demo-app\"\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.k8sService\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/outbound_converter.go:126\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.KubeHosts\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/kube_hosts_converter.go:30\ngithub.com/kumahq/kuma/pkg/plugins/runtime/k8s/controllers.(*ConfigMapReconciler).Reconcile\n\tgithub.com/kumahq/kuma@v0.0.0-20240425141305-358de6f3e590/pkg/plugins/runtime/k8s/controllers/configmap_controller.go:55\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_amd64.s:1650"}
2024-05-17T09:28:21.911Z    INFO    dns-vips-allocator    mesh VIPs changed    {"mesh": "default", "changes": [{"Type":"Remove","Entry":"0:demo-app_kuma-demo_svc_5000"}]}
2024-05-17T09:28:22.878Z    INFO    xds-server.reconcile    config has changed    {"proxyName": "kong-mesh-ingress-9c8b57df6-nqdj6.kong-mesh-system", "mesh": "", "versions": ["54fca1bc-5870-4713-bb89-bbb66f773ee4", "9bae0ded-c06b-4d4b-955d-317043025094", "802c4c0d-e4d9-4baf-90df-44067207b21a"]}
2024-05-17T09:28:23.706Z    INFO    kds-zone    detected changes in the resources. Sending changes to the client.    {"streamID": 1, "nodeID": "zone-1", "resourceType": "ZoneIngress", "client": "global"}
2024-05-17T09:28:23.706Z    INFO    kds-zone    detected changes in the resources. Sending changes to the client.    {"streamID": 1, "nodeID": "zone-1", "resourceType": "Dataplane", "client": "global"}
2024-05-17T09:28:26.706Z    INFO    kds-zone    detected changes in the resources. Sending changes to the client.    {"streamID": 1, "nodeID": "zone-1", "resourceType": "ZoneIngressInsight", "client": "global"}
2024-05-17T09:29:12.410Z    INFO    kds-mux-client    sending health check    {"client-id": "zone-1", "rpc": "healthcheck"}
2024-05-17T09:30:12.504Z    INFO    kds-mux-client    sending health check    {"client-id": "zone-1", "rpc": "healthcheck"}
2024-05-17T09:31:12.557Z    INFO    kds-mux-client    sending health check    {"client-id": "zone-1", "rpc": "healthcheck"}
2024-05-17T09:32:12.612Z    INFO    kds-mux-client    sending health check    {"client-id": "zone-1", "rpc": "healthcheck"}
github-actions[bot] commented 2 months ago

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.