Increase speed sync with k8s

v-kamerdinerov commented 1 week ago

Changes proposed in this PR

Increased speed sync between k8s and consul

In addition to registering all services periodically, now sync new services as they appear, allowing to increase the full sync interval and reduce the overall number of catalog register call to Consul servers. This makes a difference for kube clusters with hundreds of instances without delaying updates.

How I've tested this PR

How I expect reviewers to test this PR

Checklist

[x] Tests added
[ ] CHANGELOG entry added

hashicorp-cla-app[bot] commented 1 week ago

Thank you for your submission! We require that all contributors sign our Contributor License Agreement ("CLA") before we can accept the contribution. Read and sign the agreement

Learn more about why HashiCorp requires a CLA and what the CLA includes

_{Have you signed the CLA already but the status is still pending? Recheck it.}

hashicorp-cla-app[bot] commented 1 week ago

Thank you for your submission! We require that all contributors sign our Contributor License Agreement ("CLA") before we can accept the contribution. Read and sign the agreement

Learn more about why HashiCorp requires a CLA and what the CLA includes

_{Have you signed the CLA already but the status is still pending? Recheck it.}

blake commented 1 week ago

Hi @v-kamerdinerov, the sync speed can be adjusted using the syncCatalog.consulWriteInterval parameter in the Helm chart, or by passing the -consul-write-interval flag to consul-k8s-control-plane sync-catalog, if you happen to be manually starting the control plane process.

Is this suitable for your needs?

v-kamerdinerov commented 1 week ago

Hi @v-kamerdinerov, the sync speed can be adjusted using the syncCatalog.consulWriteInterval parameter in the Helm chart, or by passing the -consul-write-interval flag to consul-k8s-control-plane sync-catalog, if you happen to be manually starting the control plane process.

Is this suitable for your needs?

Hello @blake

That's something - thank you!

However, my fix also allows synchronization when - registration is new or changed

ConsulSyncPeriod I can reset to default, but my functional seems to be useful too.

jukie commented 4 days ago

However, my fix also allows synchronization when - registration is new or changed

I may be mistaken but I thought thtat syncer already reacts as soon as endpoints are updated by use of the Informer? ConsulSyncPeriod is useful to reconcile drift for example and ensure that periodic syncs happen but what are you looking to achieve with this?

v-kamerdinerov commented 4 days ago

However, my fix also allows synchronization when - registration is new or changed

I may be mistaken but I though thtat syncer already reacts as soon as endpoints are updated by use of the Informer? ConsulSyncPeriod is useful to reconcile drift for example and ensure that periodic syncs happen but what are you looking to achieve with this?

In addition to registering all services periodically, now sync new services as they appear, allowing to increase the full sync interval and reduce the overall number of catalog register call to Consul servers. This makes a difference for kube clusters with hundreds of instances without delaying updates.

v-kamerdinerov commented 1 hour ago

Checked latest version - build was successful.

❯ ./control-plane version
consul-k8s-control-plane v1.6.0-dev

hashicorp / consul-k8s