search

kubernetes / ingress-nginx

Ingress-NGINX Controller for Kubernetes
https://kubernetes.github.io/ingress-nginx/
Apache License 2.0
17.28k stars 8.21k forks source link

Allow custom routing to the Stateful Set pods #7858

Closed ilia-dot-karelia closed 2 years ago

ilia-dot-karelia commented 2 years ago

What do you want to happen? Make it possible to route traffic to specific pod. Having an Ingress Service my-svc.ing.company.com: Route web-0.my-svc.ing.company.com to pod web-0, web-N..my-svc.ing.company.com to pod web-N;

Use case example. We have a process that prepares tasks for Grid valuation (legacy product). It uses the Grid's client library. It sends tasks to Grid broker using HTTP 1.1. At the same time this lib runs an imbedded HTTP server. When Grid tasks reaches a Grid broker it assigns them to an Engine which tries to connect to the agent`s HTTP server and /GET them from it. So this is the reverse traffic. This is a legacy solution with almost zero flexibility.

k8s-ci-robot commented 2 years ago

@ilia-dot-karelia: This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
longwuyuan commented 2 years ago

Have you already tried putting the pod's dns name in the backend.service.name field.

Thanks, ; Long

On Fri, 29 Oct, 2021, 4:38 PM Kubernetes Prow Robot, < @.***> wrote:

@ilia-dot-karelia https://github.com/ilia-dot-karelia: This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here https://git.k8s.io/community/contributors/guide/pull-requests.md. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue: repository.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kubernetes/ingress-nginx/issues/7858#issuecomment-954655499, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGZVWV3NJRC6A3MN5AWVKDUJJ6BRANCNFSM5G7CCPEA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

ilia-dot-karelia commented 2 years ago

Have you already tried putting the pod's dns name in the backend.service.name field. Thanks, ; Long

Hi. Yes, I have. I tried to put smth like web-0.nginx.my-ns.svc.cluster.local but kuber is warning me that the service name is invalid or web-0, but then Ingress returns 503.

longwuyuan commented 2 years ago

Show ouput of kubectl commands that explains all major and minor details related to this. Also show your curl exactly as executed.

Thanks, ; Long

On Fri, 29 Oct, 2021, 6:56 PM ilia-dot-karelia, @.***> wrote:

Have you already tried putting the pod's dns name in the backend.service.name field. Thanks, ; Long Hi. Yes, I have. I tried to put smth like web-0.nginx.my-ns.svc.cluster.local but kuber is warning me that the service name is invalid or web-0, but then Ingress returns 503.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kubernetes/ingress-nginx/issues/7858#issuecomment-954743476, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGZVWQDNPXDW4RW476LK6TUJKOJBANCNFSM5G7CCPEA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

ilia-dot-karelia commented 2 years ago

@longwuyuan Hi. I cant paste the data from my work, so I have tried to reproduce it with GCP and Nginx . Looks like that ingress controller they use is the same that we need.

image

`kubectl get statefulset web -o yaml` ```yml apiVersion: v1 kind: Service metadata: name: nginx labels: app: nginx spec: ports: - port: 80 name: web clusterIP: None selector: app: nginx --- apiVersion: apps/v1 kind: StatefulSet metadata: name: web spec: serviceName: "nginx" replicas: 2 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: k8s.gcr.io/nginx-slim:0.8 ports: - containerPort: 80 name: web ```
`kubectl run -i --tty --image busybox:1.28 dns-test` / # nslookup web-1.nginx Server: 10.108.0.10 Address 1: 10.108.0.10 kube-dns.kube-system.svc.cluster.local Name: web-1.nginx Address 1: 10.104.3.10 web-1.nginx.default.svc.cluster.local / # nslookup web-0.nginx Server: 10.108.0.10 Address 1: 10.108.0.10 kube-dns.kube-system.svc.cluster.local Name: web-0.nginx Address 1: 10.104.0.9 web-0.nginx.default.svc.cluster.local / # wget -qO- web-0.nginx.default.svc.cluster.local web-0 / # wget -qO- web-0.nginx.default.svc.cluster.local web-0 / # wget -qO- web-0.nginx.default.svc.cluster.local web-0 / # wget -qO- web-0.nginx.default.svc.cluster.local web-0 / # wget -qO- web-1.nginx.default.svc.cluster.local web-1 / # wget -qO- web-1.nginx.default.svc.cluster.local web-1 / # wget -qO- web-1.nginx.default.svc.cluster.local web-1 / # wget -qO- web-1.nginx.default.svc.cluster.local web-1 / # wget -qO- web-1.nginx.default.svc.cluster.local web-1 ilia_karelia@cloudshell:~ (gcp-telegram-project)$ curl 34.66.184.234 web-0 ilia_karelia@cloudshell:~ (gcp-telegram-project)$ curl 34.66.184.234 web-1 ilia_karelia@cloudshell:~ (gcp-telegram-project)$ curl 34.66.184.234 web-0 ilia_karelia@cloudshell:~ (gcp-telegram-project)$ curl 34.66.184.234 web-1 ilia_karelia@cloudshell:~ (gcp-telegram-project)$ curl 34.66.184.234
`Ingress` ```yml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: my-ingress annotations: # If the class annotation is not specified it defaults to "gce". kubernetes.io/ingress.class: "nginx" spec: rules: - http: paths: - path: / pathType: Prefix backend: service: name: nginx port: number: 80 ```

When I tried to set backend.service.name to FQDN of web-0.nginx. I got warning that it is not possible to use dots here. (looks like this is the Service Name constraint). The Ingress "my-ingress" is invalid: spec.rules[0].http.paths[0].backend.service.name: Invalid value: "web-1.nginx.default.svc.cluster.local": a DNS-1035 label must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character (e.g. 'my-name', or 'abc-123', regex used for validation is 'a-z?'

Part of GK Ingress meta ```json metadata: annotations: cloud.google.com/neg: '{"ingress":true}' kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app.kubernetes.io/component":"controller","app.kubernetes.io/instance":"ingress-nginx","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"ingress-nginx","app.kubernetes.io/version":"1.0.4","helm.sh/chart":"ingress-nginx-4.0.6"},"name":"ingress-nginx-controller","namespace":"ingress-nginx"},"spec":{"externalTrafficPolicy":"Local","ipFamilies":["IPv4"],"ipFamilyPolicy":"SingleStack","ports":[{"appProtocol":"http","name":"http","port":80,"protocol":"TCP","targetPort":"http"},{"appProtocol":"https","name":"https","port":443,"protocol":"TCP","targetPort":"https"}],"selector":{"app.kubernetes.io/component":"controller","app.kubernetes.io/instance":"ingress-nginx","app.kubernetes.io/name":"ingress-nginx"},"type":"LoadBalancer"}} creationTimestamp: "2021-11-01T07:20:51Z" finalizers: - service.kubernetes.io/load-balancer-cleanup labels: app.kubernetes.io/component: controller app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/managed-by: Helm app.kubernetes.io/name: ingress-nginx app.kubernetes.io/version: 1.0.4 helm.sh/chart: ingress-nginx-4.0.6 ```

Thanks!

longwuyuan commented 2 years ago

ah ok. that is reasonable. can you check if there is a unique distinct label on that 1st pod. I think the most popular generic software, deployed as sts workloads, ship with a service that meets internal traffic needs. So in this case, if there was a unique label, then you can maybe create a service using that label as selector.

look at the sts charts of postgres or mysql that offer master-slave deployments, and check if there is a technique to have unique labels for pods of a sts.

You need architecture manhours on this rather than plain old individual pod addressing problem for external-to-internal traffic. Maybe you will see other problems (even/if/when) you solve this specific individual pod addressing problem.

longwuyuan commented 2 years ago

I think I saw the solution in kubectl expose --help . You can expose just the pod. Meaning if you use a command like kubectl expose pod web-0, you can get a service that points only to web-0 and then you can use that service's name in the field ing.spec.rules.http.paths.backend.service.name

then you can kubectl get svc <svcname> -o yaml and get a manifest

ilia-dot-karelia commented 2 years ago

@longwuyuan Hey. Yeah, my pods don`t have any unique label. If I expose a pod It is exposed to the cluster IP, so I will need to reconfigure ingress (add a rule) every time to pass the traffic from /new-pod to the new-pod-svc. This is similar to what they have implemented IMHO (https://voyagermesh.com/docs/7.1.1/guides/ingress/http/statefulset-pod/). The only difference is that the service is the same only the backend changes.

What I want to achieve is to whenever sts changes it is possible to access a new pod routing by subdomain, like web-0.some.ingress.entrypoint. Suppose I extract a subdomain with lua code snippet but how to pass incoming request up to subdomain.some.ingress.entrypoint?

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-ci-robot commented 2 years ago

@k8s-triage-robot: Closing this issue.

In response to [this](https://github.com/kubernetes/ingress-nginx/issues/7858#issuecomment-1085094529): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues and PRs according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue or PR with `/reopen` >- Mark this issue or PR as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.