Kong / kubernetes-ingress-controller

:gorilla: Kong for Kubernetes: The official Ingress Controller for Kubernetes.
https://docs.konghq.com/kubernetes-ingress-controller/
Apache License 2.0
2.22k stars 594 forks source link

KIC 2.12.6 not filling opentelemetry propagation settings when upgrading Gateway from 3.6 to 3.7 #6703

Closed tao12345666333 closed 2 hours ago

tao12345666333 commented 1 day ago

Is there an existing issue for this?

Does this enhancement require public documentation?

Problem Statement

KIC 2.12.6 in db-less. Upgrade Gateway from 3.6 to 3.7 where propagation module has been reworked: https://docs.konghq.com/hub/kong-inc/opentelemetry/3.7.x/changelog/#kong-gateway-37x

time="2024-11-21T03:09:19Z" level=debug msg="no configuration change, skipping sync to Kong" url="https://10.244.0.13:8444"
time="2024-11-21T03:09:19Z" level=debug msg="successfully applied Kong configuration to https://10.244.0.13:8444" logger=events object="{\"kind\":\"Pod\",\"namespace\":\"kong\",\"name\":\"kong-controller-598fcb7998-l6rh9\",\"apiVersion\":\"v1\"}" reason=KongConfigurationSucceeded type=Normal
time="2024-11-21T03:09:19Z" level=error msg="entity tags missing fields" error="no namespace" name=opentelemetry
time="2024-11-21T03:09:19Z" level=debug msg="no change in config status, not notifying"
time="2024-11-21T03:09:19Z" level=debug msg="sending configuration to 2 gateway clients"
time="2024-11-21T03:09:19Z" level=debug msg="failed to apply Kong configuration to https://10.244.0.15:8444: failed posting new config to /config: got status code 400" logger=events object="{\"kind\":\"Pod\",\"namespace\":\"kong\",\"name\":\"kong-controller-598fcb7998-l6rh9\",\"apiVersion\":\"v1\"}" reason=KongConfigurationApplyFailed type=Warning
time="2024-11-21T03:09:19Z" level=debug msg="no configuration change, skipping sync to Kong" url="https://10.244.0.13:8444"
time="2024-11-21T03:09:19Z" level=debug msg="successfully applied Kong configuration to https://10.244.0.13:8444" logger=events object="{\"kind\":\"Pod\",\"namespace\":\"kong\",\"name\":\"kong-controller-598fcb7998-l6rh9\",\"apiVersion\":\"v1\"}" reason=KongConfigurationSucceeded type=Normal
time="2024-11-21T03:09:19Z" level=error msg="entity tags missing fields" error="no namespace" name=opentelemetry
time="2024-11-21T03:09:19Z" level=error msg="could not update kong admin" error="performing update for https://10.244.0.15:8444 failed: failed posting new config to /config: got status code 400\nperforming update for https://10.244.0.15:8444 failed: failed posting new config to /config: got status code 400" subsystem=dataplane-synchronizer
time="2024-11-21T03:09:19Z" level=debug msg="failed to apply Kong configuration to https://10.244.0.15:8444: failed posting new config to /config: got status code 400" logger=events object="{\"kind\":\"Pod\",\"namespace\":\"kong\",\"name\":\"kong-controller-598fcb7998-l6rh9\",\"apiVersion\":\"v1\"}" reason=KongConfigurationApplyFailed type=Warning

Proposed Solution

https://github.com/Kong/kubernetes-ingress-controller/blob/2e43fd50c1475ee031babc4e288346f01eaad5a1/internal/dataplane/deckgen/generate.go#L256

KIC uses go-kong to fill the plugin's default fields.

The go-kong fixed this issue. https://github.com/Kong/go-kong/pull/431

Additional information

The essence of this issue is that Kong v3.7 has added new fields, and these fields are required. Before the Gateway upgrade, KIC obtained the user's KongClusterPlugin configuration and filled in some default fields (these are for version 3.6). After upgrading to Gateway 3.7, KIC continued to send these configurations to the Gateway, but some required fields from version 3.7 were missing, resulting in a configuration failure and a 400 error code.

Acceptance Criteria

tao12345666333 commented 14 hours ago

xref: https://github.com/Kong/go-kong/pull/431

tao12345666333 commented 2 hours ago

6657 has been merged, I will close this one.