Statefulset ingress not as expected

amey-matroid commented 1 year ago

I am using the ingress-nginx chart with kind:Statefulset.

I noticed there is a PR out to support it (https://github.com/kubernetes/ingress-nginx/pull/9442/files#diff-b7eb4cfcc41db87fb13fba887399b550ce60060eb0eb5cbe09cba580390954f3) and I used the values recommended here to patch the chart

As it is currently not supported, I made some jsonPatches as follows:

- name: ingress-nginx
  <<: *defaultChartParams
  version: 4.0.18
  namespace: default
  chart: ingress-nginx/ingress-nginx
  jsonPatches:
  - target:
      version: v1
      kind: Deployment
      name: ingress-nginx-controller
      namespace: default
    patch:
      - op: replace
        path: /kind
        value: StatefulSet
      - op: remove
        path: /spec/strategy
      - op: add
        path: /spec/serviceName
        value: "ingress-nginx-headless"
     - op: replace
        path: /spec/podManagementPolicy
        value: Parallel

However. on using replicas=2, the ingress (kubectl get ingress) only shows address for the first pod ingress-controller-0. Note that with kind: ReplicaSet, it shows addresses corresponding to both the nodes running the ingress resource. I expect it to show both the nodes running the ingress resource when using Statefulset too.

longwuyuan commented 1 year ago

/remove-kind bug /kind feature

strongjz commented 1 year ago

/triage accepted /help-wanted /priority backlog /kind feature

Keeping this as a feature request, it will need to be tested as we don't support this deployment method currently. We rely on the lease API for getting the leader to manage the nginx.conf.

github-actions[bot] commented 1 year ago

This is stale, but we won't close it automatically, just bare in mind the maintainers may be busy with other tasks and will reach your issue ASAP. If you have any question or request to prioritize this, please reach #ingress-nginx-dev on Kubernetes Slack.

k8s-triage-robot commented 1 month ago

This issue has not been updated in over 1 year, and should be re-triaged.

You can:

Confirm that this issue is still relevant with /triage accepted (org members only)
Close this issue with /close

For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/

/remove-triage accepted

longwuyuan commented 3 days ago

There are no resources to create, maintain and support statefulset of ingress-nginx controller. There is no sensible connection or reasoning involving persistent-volumes for pods that have uniqe identifier etc etc, in the context of this controller or any ingress-controller for that matter.

Since there is no action item here, I will close the issue as it is adding to the tally of open issues witout any action item. In future if a developer reasons why STS is even required etc, then this can be reopened

/close

k8s-ci-robot commented 3 days ago

@longwuyuan: Closing this issue.

In response to [this](https://github.com/kubernetes/ingress-nginx/issues/10152#issuecomment-2348303086): >There are no resources to create, maintain and support statefulset of ingress-nginx controller. There is no sensible connection or reasoning involving persistent-volumes for pods that have uniqe identifier etc etc, in the context of this controller or any ingress-controller for that matter. > >Since there is no action item here, I will close the issue as it is adding to the tally of open issues witout any action item. In future if a developer reasons why STS is even required etc, then this can be reopened > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.

rvos-73 commented 2 days ago

We actually also need the ingress controller as a statefulset. A statefulset is not only usefull for persistent-volumes.

We have an external LB in front of the k8s clusters, This external LB has a backend pool which we 'feed' with the ip adresses of the nodes where an nginx pod is running. Currently, it is not easy to create 3 seperate services for each one of the nginx pods.

If nginx was a statefulset, we would have named pods and could create 3 headless services, 1 to each of the pods.

So, basically our usecase is: We have nginx pods on a random node in the k8s cluster. We create services of the type loadbalancer (metallb) and configure fixed external IP adresses. We need to point these services to the nginx pods (1 svc for each pod, traffic is local). This is difficult/impossible without a statefulset (if you have more then 1 pod).

longwuyuan commented 2 days ago

Thank you for the updaed info. It clears the need for the use case.

Practical implications for that is ;

controller just creates one single service of --type Loadbalancer. That is the end of the controller code
it is Kubernetes specs and objects like cloud-controller-manager, or another CRD like Metallb, that responds to that
So there is no code in the controller related to assigning ipaddresses, or creating headless services
and there are no resources like developer time available to extend the specs of the Ingress-API to maintain statefulset

Hence look for the Gateway-API features to bring new features that may fit but please understand that there is no resources available to maintain a statefulset

kubernetes / ingress-nginx

Statefulset ingress not as expected #10152