search

kubernetes / kubernetes

Production-Grade Container Scheduling and Management
https://kubernetes.io
Apache License 2.0
110.91k stars 39.61k forks source link

StatefulSet without Service #69608

Closed pohly closed 2 years ago

pohly commented 6 years ago

Is this a BUG REPORT or FEATURE REQUEST?:

/kind feature

What happened:

Developers have started to use StatefulSet without a Service definition as a way to deploy a pod on a cluster exactly once, for example here: https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver/blob/master/deploy/kubernetes/stable/controller.yaml

Conceptually that makes sense when the app running in the pod doesn't accept any connections from outside. In this example, it's the CSI sidecar containers which react to changes in the apiserver.

But is this a supported mode of operation for a StatefulSet? https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.12/#statefulset-v1-apps says that

serviceName is the name of the service that governs this StatefulSet. This service must exist before the StatefulSet, and is responsible for the network identity of the set. Pods get DNS/hostnames that follow the pattern: pod-specific-string.serviceName.default.svc.cluster.local where "pod-specific-string" is managed by the StatefulSet controller.

serviceName is mandatory. The example above gets around this by setting serviceName without defining a corresponding Service.

The concern (raised by @rootfs in https://github.com/ceph/ceph-csi/issues/81) is that while it currently works, future revisions of the StatefulSet controller might not support this anymore.

What you expected to happen:

Clarify in the docs that the Service definition is not needed or (better?) make serviceName optional.

Environment:

pohly commented 6 years ago

/sig apps

fejta-bot commented 5 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot commented 5 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten

pohly commented 5 years ago

/remove-lifecycle stale

Still a valid question...

kfox1111 commented 5 years ago

+1. I'd like to know this too. Having statefulsets without a service makes sense to me if they don't care about a stable identity, just that there is no more then 1.

pohly commented 5 years ago

/remove-lifecycle rotten

fejta-bot commented 5 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

pohly commented 5 years ago

/remove-lifecycle stale

Noah-Huppert commented 5 years ago

Would a headless service be the solution to this problem?

Kubernetes Docs:

Sometimes you don’t need load-balancing and a single Service IP. In this case, you can create what are termed “headless” Services, by explicitly specifying "None" for the cluster IP (.spec.clusterIP).

pohly commented 5 years ago

Noah Huppert notifications@github.com writes:

Would a headless service be the solution to this problem?

It's better, but still a useless object that could be avoided if we could get clarified that a StatefulSet is okay to use without a Service.

kfox1111 commented 5 years ago

What is the status of this?

marcelloromani commented 5 years ago

@Noah-Huppert Yes, a headless service was the solution for me. From what I understand it's the only workaround at the moment.

kfox1111 commented 5 years ago

I've launched statefulsets without services and it seems to work ok. We're just looking for a guarantee that this will continue to be safe going forward. The docs say its required, but the implementation does not enforce it last I looked.

kow3ns commented 5 years ago

We will always support this. You can close this issue. Please feel free to open an issue against the docs repository, or (if you are feeling generous) a PR against the same.

kfox1111 commented 5 years ago

From sig-apps channel: @kowens @kfox1111 if you don’t need DNS for the Pod it is supported

@kowens the reason the docs are written as is is for conceptually simplicity. You do need a Service if you want to generate a CNAME and SRV records for the Pods it is the most common use case

So, they won't break serviceless statefulsets.

yashbhutwala commented 4 years ago

So, we had fun facing the issue today, learned a bunch of networking, passing it on, hopefully someone else will find this useful.

If you create a StatefulSet with a headless service, the ports exposed in the headless service don't matter. The communication in this case happens via DNS directly to the pod. kube-proxy is not involved at all. Found out the hard way because the exposed port on the service was configured wrongly but the communication was still happening, but when istio was installed, it did not work 😄

fyi, follow-up PR here: https://github.com/EventStore/EventStore.Charts/pull/52

fejta-bot commented 4 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

kfox1111 commented 4 years ago

/remove-lifecycle stale

It looks like when going to apps/v1, this actually did become a required feature: error: error validating "test.yaml": error validating data: ValidationError(StatefulSet.spec): missing required field "serviceName" in io.k8s.api.apps.v1.StatefulSetSpec; if you choose to ignore these errors, turn validation off with --validate=false

This should be made less restrictive if possible.

Nhat002 commented 4 years ago

@kfox1111 if you turn validation on in apps/v1beta2, you will still face this issue

Deploy failed: error: error validating "STDIN": error validating data: ValidationError(StatefulSet.spec): missing required field "serviceName" in io.k8s.api.apps.v1beta2.StatefulSetSpec; if you choose to ignore these errors, turn validation off with --validate=false
kfox1111 commented 4 years ago

I was told that it was marked as required in the docs but it actually wasn't required. But looking at it now, it is actually required to pass validation. I would request that this be made optional in validation so that it is actually optional in practice. Turning off all validation on the statefulset to make it the service optional is not a good way of doing this. For things not needing DNS, the service should be optional. There are a lot of things that could benefit from being statefulsets but don't need DNS.

fejta-bot commented 4 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

pohly commented 4 years ago

/remove-lifecycle stale

fejta-bot commented 4 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

kfox1111 commented 4 years ago

/remove-lifecycle stale

fejta-bot commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot commented 3 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten

fejta-bot commented 3 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /close

k8s-ci-robot commented 3 years ago

@fejta-bot: Closing this issue.

In response to [this](https://github.com/kubernetes/kubernetes/issues/69608#issuecomment-769967589): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >Send feedback to sig-testing, kubernetes/test-infra and/or [fejta](https://github.com/fejta). >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
kfox1111 commented 3 years ago

/reopen /remove-lifecycle rotten

k8s-ci-robot commented 3 years ago

@kfox1111: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to [this](https://github.com/kubernetes/kubernetes/issues/69608#issuecomment-771016762): >/reopen >/remove-lifecycle rotten Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
pohly commented 3 years ago

@kfox1111 should be a collaborator, right?

/reopen

k8s-ci-robot commented 3 years ago

@pohly: Reopened this issue.

In response to [this](https://github.com/kubernetes/kubernetes/issues/69608#issuecomment-771074664): >@kfox1111 should be a collaborator, right? > >/reopen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
k8s-ci-robot commented 3 years ago

@pohly: This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
fejta-bot commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

fejta-bot commented 3 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten

TBBle commented 3 years ago

/remove-lifecycle rotten

https://github.com/kubernetes/kubernetes/issues/69608#issuecomment-594677833 still seems to be the state of this issue.

k8s-triage-robot commented 3 years ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

TBBle commented 3 years ago

At this point the two CSI examples from the original ticket have gone away (see https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver/pull/521 and https://github.com/ceph/ceph-csi/pull/414 which replaced the StatefulSet singletons with Deployments using leader-election).

Are there any practical/current use-cases for StatefulSet without a backing Service? The ticket's approaching three years old, and has not yet been triaged, suggesting there's not much appetite/motivation to advanced the proposed change in https://github.com/kubernetes/kubernetes/issues/69608#issuecomment-594677833.

pohly commented 3 years ago

This is still relevant, for example here: https://github.com/kubernetes-csi/csi-driver-host-path/blob/c480b671f63c142defd2180a6ca68f85327c331f/deploy/kubernetes-1.21/hostpath/csi-hostpath-plugin.yaml#L189-L199

The Service referenced there never gets created because this issue clarified that this is okay.

TBBle commented 3 years ago

/remove-lifecycle stale

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

kfox1111 commented 2 years ago

/remove-lifecycle stale

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-ci-robot commented 2 years ago

@k8s-triage-robot: Closing this issue.

In response to [this](https://github.com/kubernetes/kubernetes/issues/69608#issuecomment-1024426810): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues and PRs according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue or PR with `/reopen` >- Mark this issue or PR as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
TBBle commented 2 years ago

/reopen

https://github.com/kubernetes/kubernetes/issues/69608#issuecomment-1002681415 intended to keep this ticket active, but named the wrong lifecycle, I assume by accident.

k8s-ci-robot commented 2 years ago

@TBBle: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to [this](https://github.com/kubernetes/kubernetes/issues/69608#issuecomment-1024809963): >/reopen > >https://github.com/kubernetes/kubernetes/issues/69608#issuecomment-1002681415 intended to keep this ticket active, but named the wrong lifecycle, I assume by accident. Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
djschny commented 2 years ago

/reopen

k8s-ci-robot commented 2 years ago

@djschny: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to [this](https://github.com/kubernetes/kubernetes/issues/69608#issuecomment-1024918399): >/reopen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.