Add feature flag to skip consumer sync to Konnect

lahabana commented 3 weeks ago

Is there an existing issue for this?

[X] I have searched the existing issues

Does this enhancement require public documentation?

[X] I have added an Acceptance Criteria item for adding and/or adjusting public documentation (if applicable)

Problem Statement

Some users have a lot of consumers. This can make Konnect sync very slow and cause dataplane sync issues (xref: https://github.com/Kong/kubernetes-ingress-controller/issues/6188 ).

Users are ok to not sync consumers as a workaround until Konnect provide a better solution for this.

Proposed Solution

add --SkipConsumerKonnectSync feature flag which bypasses consumers when syncing in Konnect

Additional information

No response

Acceptance Criteria

[ ] Add feature flag and skip sync
[ ] Validate that it works in Konnect
[ ] Document this new flag

mheap commented 2 weeks ago

add --SkipConsumerKonnectSync feature flag which bypasses consumers when syncing in Konnect

To be specific, the idea is to skip consumer credentials, not the consumers themselves.

We should also use the FeatureGates functionality rather than a new flag as we have pre-wired usage metrics for feature gates

lahabana commented 5 days ago

There was an offline discussion around only using FeatureGates because they have metrics. Maybe we should just rework the telemetry lib to add telemetry for this.

pmalek commented 4 days ago

No need to extend the telemetry package. It's already capable of sending "fixed payloads", which is done in KIC via: https://github.com/Kong/kubernetes-ingress-controller/blob/513db87cbf94ce66207f74365b502e8cde841357/internal/manager/telemetry/reports.go#L86-L92.

It's just a matter of serializing config or CLI flags/envs.

lahabana commented 4 days ago

@mheap agrees that we don't need a FeatureGate for this.

Kong / kubernetes-ingress-controller