search

scylladb / scylla-operator

The Kubernetes Operator for ScyllaDB
https://operator.docs.scylladb.com/
Apache License 2.0
323 stars 159 forks source link

Split ScyllaDB API status probes into a dedicated container #1926

Closed tnozicka closed 3 weeks ago

tnozicka commented 1 month ago

Description of your changes: At this point, ScyllaDB doesn't provide standardized /readyz and /healthz probes, so we serve a wrapper around CQL status (with some tweak) from an operator binary. This is unfortunately running in the same container as ScyllaDB and shares the cpu quota with it and has a negative impact on latency. There are others but this is a first step towards fixing and probes are really the really problematic ones as they serve continuously.

This PR move ScyllaDB probes to run in a separate container so it run in a separate cpu slice and doesn't affect ScyllaDB latency. This is also more architecturally clean and is a step forward to stop coping the operator binary into ScyllaDB container. It also abstracts generic probe patterns and related objects, so we can add other probes in the future, like for the Alternator or other APIs.

This also puts the probe logs at a correct place and no longer mix them with ScyllaDB logs.

Which issue is resolved by this Pull Request: Resolves #620

scylla-operator-bot[bot] commented 1 month ago

@tnozicka: GitHub didn't allow me to request PR reviews from the following users: tnozicka.

Note that only scylladb members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to [this](https://github.com/scylladb/scylla-operator/pull/1926): >**Description of your changes:** >TODO > >**Which issue is resolved by this Pull Request:** >Resolves #620 > >/cc > Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
tnozicka commented 1 month ago

/cc @zimnx @rzetelskik

scylla-operator-bot[bot] commented 3 weeks ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rzetelskik, tnozicka, zimnx

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/scylladb/scylla-operator/blob/master/OWNERS)~~ [tnozicka,zimnx] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
tnozicka commented 3 weeks ago

https://github.com/scylladb/scylla-operator-release/issues/212 /retest