search

kubernetes / enhancements

Enhancements tracking repo for Kubernetes
Apache License 2.0
3.42k stars 1.48k forks source link

[Beta: 1.31] Consistent Reads from Cache #2340

Open shekhar-rajak opened 3 years ago

shekhar-rajak commented 3 years ago

Enhancement Description

Kubernetes Get and List requests are guaranteed to be "consistent reads" if the resourceVersion parameter is not provided. Consistent reads are served from etcd using a "quorum read".

But often the watch cache contains sufficiently up-to-date data to serve the read request, and could serve it far more efficiently.

This KEP proposes a mechanism to serve most reads from the watch cache while still providing the same consistency guarantees as serving the read from etcd.

All tasks done for beta

shekhar-rajak commented 3 years ago

2331 created for migrating the old keps into new template.

LappleApple commented 3 years ago

/sig api-machinery

fejta-bot commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

k8s-triage-robot commented 3 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten

k8s-triage-robot commented 3 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-ci-robot commented 3 years ago

@k8s-triage-robot: Closing this issue.

In response to [this](https://github.com/kubernetes/enhancements/issues/2340#issuecomment-913153528): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues and PRs according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue or PR with `/reopen` >- Mark this issue or PR as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
serathius commented 1 year ago

Reviving the KEP as agreed on SIG-Apimachinery meeting May 31st 2023. https://youtu.be/sueYXW89ZHg /assign

PR https://github.com/kubernetes/enhancements/pull/4047 to move KEP to implementable

serathius commented 1 year ago

/stage alpha

serathius commented 1 year ago

/milestone v1.28

jpbetz commented 1 year ago

/lead opt-in

Atharva-Shinde commented 1 year ago

Hello @shekhar-rajak @serathius 👋, Enhancements team here.

Just checking in as we approach Enhancements freeze on Thursday, 16th June 2023.

Looks like this enhancement is targeting for stage alpha for v1.28

Here's where this enhancement currently stands:

There are some nits remaining to be addressed:

I won't decline this KEP for enhancements freeze because of these minor changes but would be great to get a PR ready addressing them.

The status of this enhancement is marked as tracked. Please keep the issue description up-to-date with appropriate stages as well. Thank you :)

serathius commented 1 year ago

Don't understand the unmarked points.

KEP readme using the latest template has been merged into the k/enhancements repo. KEP readme has an updated detailed test plan section filled out

Tests follows the latest template (maybe roughly) and uses a detailed test plan. Please provide criteria that you used to decide that they are not.

Atharva-Shinde commented 1 year ago

Hey @serathius , the second unmarked point is a subset of the first unmarked point. As mentioned in my comment above, there is an acknowledgement at the start of the Test Plan section quoting:

[ ] I/we understand the owners of the involved components may require updates to existing tests to make this code solid enough prior to committing the changes necessary to implement this enhancement.

which this KEP currently doesn't have and we need that to be included so that the KEP is in compliance with the latest KEP readme template.

serathius commented 1 year ago

Makes sense, suggestion for future. Separate questionnaire to KEP authors (also applies to PRR) from KEPs design document into separate file. Having agreements in the middle of the design makes it less readable, doesn't contribute to overall design and also cannot be automatically validated via ./hack/verify-kep.sh.

serathius commented 1 year ago

Sent https://github.com/kubernetes/enhancements/pull/4094

Atharva-Shinde commented 1 year ago

Thanks for the PR :) We appreciate your suggestion, I'll bring this up in our retrospection meeting.

Atharva-Shinde commented 1 year ago

Also @serathius, it would be great if you update this field in the kep.yaml alongside.

serathius commented 1 year ago

Done

taniaduggal commented 1 year ago

Hello @shekhar-rajak 1.28 Docs Shadow here.

Does this enhancement work planned for 1.28 require any new docs or modification to existing docs?

If so, please follows the steps here to open a PR against dev-1.28 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday 20th July 2023.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.

Thank you!

Rishit-dagli commented 1 year ago

Hey @serathius , could you please create a docs PR even if it is a draft PR with no content yet against dev-1.28 branch in the k/website repo. The deadline to create this draft PR is Thursday 20th July 2023.

serathius commented 1 year ago

Not sure what needs to be documented as this is just a pure feature gate with no user facing effects aside of performance. cc @jpbetz @deads2k any suggestions?

Atharva-Shinde commented 1 year ago

Hey again @serathius :wave:

Just checking in as we approach Code freeze at 01:00 UTC Friday, 19th July 2023 .

Here’s the enhancement’s state for the upcoming code freeze:

For this enhancement, it looks like the following code related PR is open and it needs to be merged or should be in merge-ready state before the code freeze commences :

These are the code freeze related PR's that I found on this KEP issue

Please keep the issue description up-to-date with all the PR/s that are associated with this KEP and let me know if there are other PR/s in k/k we should be tracking for this KEP.

As always, we are here to help if any questions come up. Thanks!

serathius commented 1 year ago

Results of benchmarking consistent reads from cache against reads from etcd. Added performance tests to https://testgrid.k8s.io/sig-scalability-experiments, following 2 scenarios:

In both cases we put load of 1 LIST per second with selector selecting no objects.

Comparing resource usage and latency with and without consistent list from watch cache enabled.

Handled List requests [qps] kube-apiserver CPU [cores] etcd CPU [cores] LIST latency [ms]
50%ile 90%ile 99%ile 50%ile 90%ile 99%ile 50%ile 90%ile 99%ile
Baseline 0 0.10 0.11 0.12 0.18 0.19 0.19 25.00 45.00 49.50
Enabled Large Objects 1 0.09 0.11 0.11 0.18 0.19 0.19 25.00 45.00 49.49
Disabled Large Objects 1 3.13 3.14 3.16 1.73 16.16 16.37 1438.49 1856.13 1985.61
Enabled Small Objects 1 0.63 0.64 0.68 0.23 2.11 2.16 499.32 582.04 648.00
Disabled Small Objects 0.86 6.92 70.85 71.41 3.57 3.72 3.75 10493.83 17910.71 21800.00
taniaduggal commented 1 year ago

Hey @serathius , Docs Shadow here! could you please create a docs PR even if it is a draft PR with no content yet against dev-1.28 branch in the k/website repo. The deadline to create this draft PR is Thursday 20th July 2023.

Atharva-Shinde commented 1 year ago

Hey @serathius 👋 Enhancements Lead here, With https://github.com/kubernetes/kubernetes/pull/118508 and https://github.com/kubernetes/test-infra/pull/30094 merged as per the issue description, this enhancement is now tracked for v1.28 Code Freeze. Thanks!

serathius commented 1 year ago

Done https://github.com/kubernetes/website/pull/42083

Rishit-dagli commented 1 year ago

Hello @shekhar-rajak @serathius wave: please take a look at Documenting for a release - PR Ready for Review to get your docs PR ready for review before Tuesday 25th July 2023. Thank you!

Ref: https://github.com/kubernetes/website/pull/42083

npolshakova commented 1 year ago

Hello 👋, 1.29 Enhancements Lead here.

If you wish to progress this enhancement in v1.29, please have the SIG lead opt-in your enhancement by adding the lead-opt-in and milestone v1.29 labels before the Production Readiness Review Freeze.

/remove-label lead-opted-in

salehsedghpour commented 9 months ago

Hello 👋 1.30 Enhancements Lead here,

I'm closing milestone 1.28 now, If you wish to progress this enhancement in v1.30, please follow the instructions here to opt in the enhancement and make sure the lead-opted-in label is set so it can get added to the tracking board and finally add /milestone v1.30. Thanks!

/milestone clear

wojtek-t commented 8 months ago

/milestone v1.30 /label lead-opted-in /label stage/beta /remove-label stage/alpha

k8s-ci-robot commented 8 months ago

@wojtek-t: The label(s) /label stage/beta , /remove-label stage/alpha cannot be applied. These labels are supported: api-review, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, team/katacoda, refactor, lead-opted-in, tracked/no, tracked/out-of-tree, tracked/yes. Is this label configured under labels -> additional_labels or labels -> restricted_labels in plugin.yaml?

In response to [this](https://github.com/kubernetes/enhancements/issues/2340#issuecomment-1931501681): >/milestone v1.30 >/label lead-opted-in >/label stage/beta >/remove-label stage/alpha Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
salehsedghpour commented 8 months ago

Hello @serathius 👋, Enhancements team here.

Just checking in as we approach enhancements freeze on 02:00 UTC Friday 9th February 2024.

This enhancement is targeting for stage beta for v1.30 (correct me, if otherwise)

Here's where this enhancement currently stands:

The status of this enhancement is marked as tracked for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

kcmartin commented 8 months ago

Hi @serathius,

👋 from the v1.30 Communications Team! We'd love for you to opt in to write a feature blog about your enhancement!

We encourage blogs for features including, but not limited to: breaking changes, features and changes important to our users, and features that have been in progress for a long time and are graduating.

To opt in, you need to open a Feature Blog placeholder PR against the website repository. The placeholder PR deadline is 27th February, 2024. Here's the 1.30 Release Calendar

drewhagen commented 8 months ago

Hello @serathius 👋, 1.30 Docs Lead here.

Does this enhancement work planned for 1.30 require any new docs or modification to existing docs? If so, please follows the steps here to open a PR against dev-1.30 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday February 22nd 2024 18:00 PDT.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release. Thank you!

serathius commented 8 months ago

@drewhagen Done https://github.com/kubernetes/website/pull/45239

salehsedghpour commented 8 months ago

Hey again @serathius 👋 Enhancements team here,

Just checking in as we approach code freeze at 02:00 UTC Wednesday 6th March 2024 .

Here's where this enhancement currently stands:

For this enhancement, it looks like the following PRs are open and need to be merged before code freeze (and we need to update the Issue description to include all the related PRs of this KEP):

With this, it is now marked as at risk for code freeze for the v1.30 Code Freeze!

Also, please let me know if there are other PRs in k/k we should be tracking for this KEP. As always, we are here to help if any questions come up. Thanks!

serathius commented 8 months ago

@salehsedghpour Did you mean 5th March and not 9th February which is the enhancement freeze?

salehsedghpour commented 8 months ago

Oh You're right, I copied the wrong date. Thanks for noticing! I updated my comment.

serathius commented 8 months ago

@kcmartin done

salehsedghpour commented 7 months ago

Hello @serathius 👋 Enhancements team here,

Unfortunately, the implementation (code related) PR(s) associated with this enhancement is not in the merge-ready state by code-freeze and hence this enhancement is now removed from the v1.30 milestone.

If you still wish to progress this enhancement in v1.30, please file an exception request. Thanks!

/milestone clear

serathius commented 6 months ago

/milestone v1.31

serathius commented 6 months ago

We slipped from v1.30 due to https://github.com/etcd-io/etcd/issues/17507.

Will attempt to land it in v1.31, preliminary validation of v3.5.13 shows that K8s test pass https://github.com/kubernetes/kubernetes/pull/124469.

Next step is to merge https://github.com/kubernetes/kubernetes/pull/124469, https://github.com/kubernetes/kubernetes/pull/124027 and https://github.com/kubernetes/kubernetes/pull/123264

sreeram-venkitesh commented 5 months ago

@serathius Since this KEP is planned for v1.31, I'm keeping the lead-opted-in label. It has already been added to the v1.31 enhancements tracking board. Please let me know if this works.

serathius commented 5 months ago

Great, thanks @sreeram-venkitesh!

serathius commented 4 months ago

The performance comparison of scalability 5k performance tests with consistent list from cache enabled:

Consistent List From Cache kube-apiserver CPU [cores] kube-apiserver Memory [MB] etcd CPU [cores] etcd Memory [GB] LIST latency [ms] slowest resource (pods) LIST latency [ms] second slowest resource LIST latency [ms] second third resource
Enabled 50ile 6.305 42.18 1.1 4.68 200.00 55.00 38.00
90ile 11.99 47.78 1.426 5.40 1460 180 180
99ile 26.466 49.59 2.1847 6.26 1490.00 300.00 198.00
Disabled 50ile 9.085 43.90 1.5 4.84 3500 38 38
90ile 17.72 48.94 1.964 5.29 4850.00 92.70 92.70
99ile 27.132 50.39 2.5547 5.93 4990.00 274.00 272.00

Improvements summary

Some degradations that might require parameter tuning/investigation.

cc @deads2k @MadhavJivrajani @p0lyn0mial

dipesh-rawat commented 4 months ago

Hello @serathius 👋, 1.31 Enhancements team here.

Just checking in as we approach enhancements freeze on on 02:00 UTC Friday 14th June 2024 / 19:00 PDT Thursday 13th June 2024.

This enhancement is targeting for stage beta for 1.31 (correct me, if otherwise)

Here's where this enhancement currently stands:

Nit: Update issue description to highlight Beta: 1.31

With all the KEP requirements in place and merged into k/enhancements, this enhancement is all good for the upcoming enhancements freeze. 🚀

The status of this enhancement is marked as tracked for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

chanieljdan commented 4 months ago

Hi @serathius :wave:, 1.31 Docs Shadow here.

Does this enhancement work planned for 1.31 require any new docs or modification to existing docs?

If so, please follows the steps here to open a PR against dev-1.31 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday June 27, 2024 18:00 PDT.

Also, take a look at Documenting for a release to get yourself familiarized with the docs requirement for the release.

Thank you!

Daniel

serathius commented 4 months ago

Does this enhancement work planned for 1.31 require any new docs or modification to existing docs?

Yes, will create PR when I'm able.

a-mccarthy commented 4 months ago

Hi @serathius!

👋 from the v1.31 Communications Team! We'd love for you to opt in to write a feature blog about your enhancement! Some reasons why you might want to write a blog for this feature include (but are not limited to) if this introduces breaking changes, is important to our users, or has been in progress for a long time and is graduating.

To opt in, let us know and open a Feature Blog placeholder PR against the website repository by 3rd July, 2024. For more information about writing a blog see the blog contribution guidelines.

Note: In your placeholder PR, use XX characters for the blog date in the front matter and file name. We will work with you on updating the PR with the publication date once we have a final number of feature blogs for this release.

chanieljdan commented 4 months ago

Hi @serathius 👋, 1.31 Docs Shadow here.

Does this enhancement work planned for 1.31 require any new docs or modification to existing docs?

If so, please follows the steps here to open a PR against dev-1.31 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday June 27, 2024 18:00 PDT.

Also, take a look at Documenting for a release to get yourself familiarized with the docs requirement for the release.

Thank you!

Daniel

Hi @serathius 👋,

Just a reminder to open a placeholder PR against dev-1.31 branch in the k/website repo for this (steps available here). The deadline for this is a week away at Thursday June 27, 2024 18:00 PDT.

Thanks,

Daniel