namespaced ingress doesn't work as expected

[yong-jie-gong]

What happened: From kubernetes 1.18, kubernetes deprecate ingress annotation "kubernetes.io/ingress.class", instead, it is replaced with ingress.Spec.IngressClass. for cluster Ingress, it is ok. but for namespaced ingress. cluster don't want to grant any cluster resource permission to ingress-controller. it means nginx-ingrss-controller have no permissions to access the IngressClass object. in current nginx-ingress-controller, it mandate the IngressClass existence referred as ingress.Spec.IngressClassName.

As a result, for Namespaced ingress scenario, ingress annnotation "kubernetes.io/ingress.class" is the only choice. it works at this time, but from kubernets 1.28, kubernetes server keep printing warning if ingress has annotation "kubernetes.io/ingress.class". it is not ideal.

What you expected to happen:

so it is better support namespaced ingressClass without accessing the IngresClass object and using the annotation. suggestions: 1) IngressController needn't cluster level permission to access the IngressClass for namespaced Ingress 2) consumer drop annotation "kubernetes.io/ingress.class" from ingress 3) Consumer set the ingressClassName by ingress.Spec.IngressClassName 4) IngressController accept the incoming ingress object when 1) IngressController has permission to IngressClass, keep the current implementation. 2) IngressController dont' have permission to access the IngressClass but ingress.Spec.IngressClassName is equals to the ingress class name specified by CLI parameter "--ingress-class"

NGINX Ingress controller version (exec into the pod and run nginx-ingress-controller --version.):

Kubernetes version (use kubectl version): v1.29.2

Environment:

• Cloud provider or hardware configuration:

• OS (e.g. from /etc/os-release):

• Kernel (e.g. uname -a):

• Install tools:

  • Please mention how/where was the cluster created like kubeadm/kops/minikube/kind etc.

• Basic cluster related info:

  • kubectl version
  • kubectl get nodes -o wide

• How was the ingress-nginx-controller installed:

  • If helm was used then please show output of helm ls -A | grep -i ingress
  • If helm was used then please show output of helm -n <ingresscontrollernamespace> get values <helmreleasename>
  • If helm was not used, then copy/paste the complete precise command used to install the controller, along with the flags and options used
  • if you have more than one instance of the ingress-nginx-controller installed in the same cluster, please provide details for all the instances /nginx-ingress-controller --kubeconfig=/root/.kube/config --default-ssl-certificate=core/demo1-nginx-secret --v=0 --configmap=core/demo1-ingress-controller-conf --watch-namespace=core --annotations-prefix=ingress.kubernetes.io --enable-ssl-chain-completion=false --http-port=8080 --https-port=8443 --enable-annotation-validation=true --update-status=false --ingress-class=demo1-nginx --metrics-per-host=false --enable-metrics=false "

• Current State of the controller:

  • kubectl describe ingressclasses
  • kubectl -n <ingresscontrollernamespace> get all -A -o wide
  • kubectl -n <ingresscontrollernamespace> describe po <ingresscontrollerpodname>
  • kubectl -n <ingresscontrollernamespace> describe svc <ingresscontrollerservicename>

• Current state of ingress object, if applicable:

  • kubectl -n <appnamespace> get all,ing -o wide
  • kubectl -n <appnamespace> describe ing <ingressname>
  • If applicable, then, your complete and exact curl/grpcurl command (redacted if required) and the reponse to the curl/grpcurl command with the -v flag

• Others:

  • Any other related information like ;
  • copy/paste of the snippet (if applicable)
  • kubectl describe ... of any custom configmap(s) created and in use
  • Any other related information that may help

How to reproduce this issue:

Anything else we need to know:

[k8s-ci-robot]

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

[yong-jie-gong]

raise PR #11223

[longwuyuan]

• I can't find documentation that describes installing namespace scoped instance

• Before Kubernetes v1.24, there used to be attempts and use-cases of namespace scoped installation

• Even though you mentioned that some cluster-admins do not allow users to access cluster-wide resources, the upstream K8S Ingress API based design of this controller involves to access to cluster-wide resources. So I don't think its a improvement to change the the behavior of the controller to restrict access to a namespace

• There are far too important and huge number of users who use the annotation for the ingressClassName, particularly cert-manager. So this proiect has to continue support for the annotation

• I don't see the practical benefit of the change you suggest here because I don't see how a ingress-controller is appropriate to be running in a cluster without the cluster-admin's approval/consent & co-operation. The PR you submitted does not have a description of the solution. At least describe the entire solution in small details so that a valid case is presented to the reader

• I see a hard change just to 2 go files, without any consideration to how it will impact a user's experience and tests to show the working of the changes.

• Does your changes impact the rest of the controller's features like --default-ssl-certificate etc,

[longwuyuan]

/remove-kind bug

[yong-jie-gong]

@longwuyuan thanks for your quick response. as requested in PR https://github.com/kubernetes/ingress-nginx/pull/11223

Assume that the sample application to be deployed is --image nginx:alpine
kubectl create deployment test0 --image nginx:alpine --port 80
Assume that the service for this is kubectl expose deployment test0 --port 80
Now write a ingress resource yaml file for it and keep it ready for use after the clusrter is ready
Create a minikube cluster
Fork the project on github
create a branch and clone
Make your changes to the code
Run make dev-env
Now there will be a cluster ready with your changes to the controller code
Deploy your app and service and ingress
Copy/paste all the test and logs and state related info as outputs of commands here on in the issue
Then I will have more practical ways to copy your fork's branch and do the same and test your changed controller locally
I can then put the default-ssl-certificate in a different namespace and see how I can configure ingress with TLS but without a cert
I can then see first hand what you mean by not-using-cluster-ingress-class

please check information from my env as below 1) Setup dev test with "make dev-env". Kubernetes cluster ready and ingress-nginx listening in localhost using ports 80 and 443 To delete the dev cluster execute: 'kind delete cluster --name ingress-nginx-dev'

# kind get clusters
ingress-nginx-dev

# kubectl get po
NAME                                        READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-2vf6g        0/1     Completed   0          16m
ingress-nginx-admission-patch-cxtx8         0/1     Completed   2          16m
ingress-nginx-controller-659c6c4948-pr8jm   1/1     Running     0          16m
test0-574c47cb97-fzhjf                      1/1     Running     0          8m11s

2) by default, one "ingress-nginx-controller" is deployed in my env whose service account is bound to cluster role "" below

# kubectl get clusterrolebinding |grep ingres
ingress-nginx                                          ClusterRole/ingress-nginx                                                          18m
ingress-nginx-admission                                ClusterRole/ingress-nginx-admission                                                18m

# kubectl get clusterrolebinding ingress-nginx  -ojsonpath='{.roleRef} {"\n"} {.subjects}'
 {"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"ingress-nginx"}
 [{"kind":"ServiceAccount","name":"ingress-nginx","namespace":"ingress-nginx"}]

3) for namespaced deployment, nginx-ingress-controller is not supposed to have cluster level permission. so remove cluster rolebinding "ingress-nginx"

# kubectl delete clusterrolebinding  ingress-nginx
clusterrolebinding.rbac.authorization.k8s.io "ingress-nginx" deleted

4) default nginx-ingress-controller pod watches the whole cluster, so update nginx-ingress-controller deployment add the CLI parameter "- --watch-namespace=$(POD_NAMESPACE)"

5) deployment my service/app/ingress as requested from your more test0.svc.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app: test0
  name: test0
  namespace: ingress-nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: test0
  type: ClusterIP

more test0.deploy.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: test0
  name: test0
  namespace: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: test0
  template:
    metadata:
      labels:
        app: test0
    spec:
      containers:
      - image: nginx:alpine
        imagePullPolicy: IfNotPresent
        name: nginx
        ports:
        - containerPort: 80
          protocol: TCP
      restartPolicy: Always

more test0.ing.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-class-name-no-perm
  namespace: ingress-nginx
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - backend:
          service:
            name: test0
            port:
              number: 80
        path: /demo/http1
        pathType: Prefix
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-invalid-ingress-class-name-no-perm
  namespace: ingress-nginx
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx-not-match
  rules:
  - http:
      paths:
      - backend:
          service:
            name: test0
            port:
              number: 80
        path: /demo/http2
        pathType: Prefix
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx-annotation
    nginx.ingress.kubernetes.io/rewrite-target: /
  name: ingress-from-annotation
  namespace: ingress-nginx
spec:
  rules:
  - http:
      paths:
      - backend:
          service:
            name: test0
            port:
              number: 80
        path: /demo/http3
        pathType: Prefix

6) check nginx-ingress-controller pod output which is not trying to watch cluter level resource IngressClass "No permissions to list and get Ingress Classes:"

# kubectl logs ingress-nginx-controller-7fd476c957-6t5cp

-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       1.0.0-dev
  Build:         git-7c2b047c5
  Repository:    git@github.com:yong-jie-gong/ingress-nginx.git
  nginx version: nginx/1.25.3

-------------------------------------------------------------------------------

W0415 08:06:53.880250      12 client_config.go:618] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0415 08:06:53.880720      12 main.go:205] "Creating API client" host="https://10.96.0.1:443"
I0415 08:06:53.934795      12 main.go:248] "Running in Kubernetes cluster" major="1" minor="26" git="v1.26.3" state="clean" commit="9e644106593f3f4aa98f8a84b23db5fa378900bd" platform="linux/amd64"
I0415 08:06:55.130756      12 main.go:101] "SSL fake certificate created" file="/etc/ingress-controller/ssl/default-fake-certificate.pem"
W0415 08:06:55.154262      12 main.go:111] No permissions to list and get Ingress Classes: ingressclasses.networking.k8s.io is forbidden: User "system:serviceaccount:ingress-nginx:ingress-nginx" cannot list resource "ingressclasses" in API group "networking.k8s.io" at the cluster scope, IngressClass feature will be disabled

7) check generated nginx.conf file

# kubectl get po|grep ingress-nginx-controller-7fd476c957-6t5cp
ingress-nginx-controller-7fd476c957-6t5cp   1/1     Running     0          14m

# kubectl exec -it ingress-nginx-controller-7fd476c957-6t5cp -- bash -c 'more /etc/nginx/nginx.conf|grep /demo'
                location ~* "^/demo/http1" {
                        set $location_path  "/demo/http1";
                        rewrite "(?i)/demo/http1" / break;

8) Check deployed demo application.  "/demo/http1" is picked up when "nginx-ingress-controller" pod don't have permission to access the cluster level resource "IngressClass"
# kubectl  get svc -n$ns|grep ingress
ingress-nginx-controller             NodePort    10.96.31.140   <none>        80:32691/TCP,443:31098/TCP   67m

# ip=10.96.31.140  
root@ingress-nginx-dev-control-plane:/# curl -s -o /dev/null --head --write-out '%{http_code}' --noproxy $ip http://$ip:80/demo/http1 ; echo $http_code
200
root@ingress-nginx-dev-control-plane:/# curl -s -o /dev/null --head --write-out '%{http_code}' --noproxy $ip http://$ip:80/demo/http2 ; echo $http_code
404
root@ingress-nginx-dev-control-plane:/# curl -s -o /dev/null --head --write-out '%{http_code}' --noproxy $ip http://$ip:80/demo/http3 ; echo $http_code
404

root@ingress-nginx-dev-control-plane:/# curl -s -o /dev/null --head --write-out '%{http_code}' -k --noproxy $ip https://$ip:443/demo/http1 ; echo $http_code
200
root@ingress-nginx-dev-control-plane:/# curl -s -o /dev/null --head --write-out '%{http_code}' -k --noproxy $ip https://$ip:443/demo/http2 ; echo $http_code
404
root@ingress-nginx-dev-control-plane:/# curl -s -o /dev/null --head --write-out '%{http_code}' -k --noproxy $ip https://$ip:443/demo/http3 ; echo $http_code
404

[longwuyuan]

@yong-jie-gong I request some detailed information which helps reduce the work to be done by others. Is it possible for you ti kindly edit the above message and post information as per hints below ;

• When you run make make dev-env, please do it from a shell, where you have your fork+clone+branch (in which you made your changes to the controller code)

• First show git diff so that all the changes you made are visible

• Show output of following commands instead of yaml files

  • helm ls -A
  • kubectl - ingress-nginx get all
  • kubectl describe clusterrole ingress-nginx
  • kubectl describe clusterrolebindings.rbac.authorization.k8s.io ingress-nginx
  • kubectl describe sa
  • kubectl get all,ing
  • kubectl describe ing
  • kubectl get events
  • curl test0.local -v

And other such information. This is to see the live state of the resources like clusterrole and others from your changes as well the curl command and the other commands that explains how ingress is working after your changes

[longwuyuan]

/kind feature

[longwuyuan]

/triage needs-information

[yong-jie-gong]

@yong-jie-gong I request some detailed information which helps reduce the work to be done by others. Is it possible for you ti kindly edit the above message and post information as per hints below ;

• When you run make make dev-env, please do it from a shell, where you have your fork+clone+branch (in which you made your changes to the controller code)
• First show git diff so that all the changes you made are visible

• Show output of following commands instead of yaml files

  • helm ls -A
  • kubectl - ingress-nginx get all
  • kubectl describe clusterrole ingress-nginx
  • kubectl describe clusterrolebindings.rbac.authorization.k8s.io ingress-nginx
  • kubectl describe sa
  • kubectl get all,ing
  • kubectl describe ing
  • kubectl get events
  • curl test0.local -v

And other such information. This is to see the live state of the resources like clusterrole and others from your changes as well the curl command and the other commands that explains how ingress is working after your changes

Add more information as requested

1. helm ls -A

/root$ helm ls -A
NAME    NAMESPACE       REVISION        UPDATED STATUS  CHART   APP VERSION

2. kubectl - ingress-nginx get all

NAME                                            READY   STATUS      RESTARTS   AGE
pod/ingress-nginx-admission-create-2vf6g        0/1     Completed   0          47h
pod/ingress-nginx-admission-patch-cxtx8         0/1     Completed   2          47h
pod/ingress-nginx-controller-6484977b56-tp7tr   1/1     Running     0          46h
pod/test0-574c47cb97-fzhjf                      1/1     Running     0          47h

NAME                                         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
service/ingress-nginx-controller             NodePort    10.96.31.140   <none>        80:32691/TCP,443:31098/TCP   47h
service/ingress-nginx-controller-admission   ClusterIP   10.96.33.172   <none>        443/TCP                      47h
service/test0                                ClusterIP   10.96.41.149   <none>        80/TCP                       47h

NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/ingress-nginx-controller   1/1     1            1           47h
deployment.apps/test0                      1/1     1            1           47h

NAME                                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/ingress-nginx-controller-6484977b56   1         1         1       46h
replicaset.apps/ingress-nginx-controller-659c6c4948   0         0         0       47h
replicaset.apps/ingress-nginx-controller-77d966f98c   0         0         0       47h
replicaset.apps/ingress-nginx-controller-7fd476c957   0         0         0       47h
replicaset.apps/test0-574c47cb97                      1         1         1       47h

NAME                                       COMPLETIONS   DURATION   AGE
job.batch/ingress-nginx-admission-create   1/1           73s        47h
job.batch/ingress-nginx-admission-patch    1/1           86s        47h

3. kubectl describe clusterrole ingress-nginx

   
   $ kubectl describe clusterrole ingress-nginx
   Name:         ingress-nginx
   Labels:       app.kubernetes.io/instance=ingress-nginx
             app.kubernetes.io/managed-by=Helm
             app.kubernetes.io/name=ingress-nginx
             app.kubernetes.io/part-of=ingress-nginx
             app.kubernetes.io/version=1.10.0
             helm.sh/chart=ingress-nginx-4.10.0
   Annotations:  <none>
   PolicyRule:
   Resources                           Non-Resource URLs  Resource Names  Verbs
   ---------                           -----------------  --------------  -----
   events                              []                 []              [create patch]
   services                            []                 []              [get list watch]
   ingressclasses.networking.k8s.io    []                 []              [get list watch]
   ingresses.networking.k8s.io         []                 []              [get list watch]
   nodes                               []                 []              [list watch get]
   endpointslices.discovery.k8s.io     []                 []              [list watch get]
   configmaps                          []                 []              [list watch]
   endpoints                           []                 []              [list watch]
   namespaces                          []                 []              [list watch]
   pods                                []                 []              [list watch]
   secrets                             []                 []              [list watch]
   leases.coordination.k8s.io          []                 []              [list watch]
   ingresses.networking.k8s.io/status  []                 []              [update]

$ kubectl describe role ingress-nginx Name: ingress-nginx Labels: app.kubernetes.io/component=controller app.kubernetes.io/instance=ingress-nginx app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=ingress-nginx app.kubernetes.io/part-of=ingress-nginx app.kubernetes.io/version=1.10.0 helm.sh/chart=ingress-nginx-4.10.0 Annotations: PolicyRule: Resources Non-Resource URLs Resource Names Verbs

---

events [] [] [create patch] leases.coordination.k8s.io [] [] [create] configmaps [] [] [get list watch] endpoints [] [] [get list watch] pods [] [] [get list watch] secrets [] [] [get list watch] services [] [] [get list watch] ingressclasses.networking.k8s.io [] [] [get list watch] ingresses.networking.k8s.io [] [] [get list watch] leases.coordination.k8s.io [] [ingress-nginx-leader] [get update] namespaces [] [] [get] endpointslices.discovery.k8s.io [] [] [list watch get] ingresses.networking.k8s.io/status [] [] [update]

4. kubectl describe clusterrolebindings.rbac.authorization.k8s.io ingress-nginx
```sh
# kubectl describe clusterrolebindings.rbac.authorization.k8s.io ingress-nginx

Name:         ingress-nginx-admission
Labels:       app.kubernetes.io/component=admission-webhook
              app.kubernetes.io/instance=ingress-nginx
              app.kubernetes.io/managed-by=Helm
              app.kubernetes.io/name=ingress-nginx
              app.kubernetes.io/part-of=ingress-nginx
              app.kubernetes.io/version=1.10.0
              helm.sh/chart=ingress-nginx-4.10.0
Annotations:  helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
              helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
Role:
  Kind:  ClusterRole
  Name:  ingress-nginx-admission
Subjects:
  Kind            Name                     Namespace
  ----            ----                     ---------
  ServiceAccount  ingress-nginx-admission  ingress-nginx

# kubectl describe rolebindings.rbac.authorization.k8s.io ingress-nginx
Name:         ingress-nginx
Labels:       app.kubernetes.io/component=controller
              app.kubernetes.io/instance=ingress-nginx
              app.kubernetes.io/managed-by=Helm
              app.kubernetes.io/name=ingress-nginx
              app.kubernetes.io/part-of=ingress-nginx
              app.kubernetes.io/version=1.10.0
              helm.sh/chart=ingress-nginx-4.10.0
Annotations:  <none>
Role:
  Kind:  Role
  Name:  ingress-nginx
Subjects:
  Kind            Name           Namespace
  ----            ----           ---------
  ServiceAccount  ingress-nginx  ingress-nginx

5. kubectl describe sa

   
   $ kubectl describe sa
   Name:                default
   Namespace:           ingress-nginx
   Labels:              <none>
   Annotations:         <none>
   Image pull secrets:  <none>
   Mountable secrets:   <none>
   Tokens:              <none>
   Events:              <none>

Name: ingress-nginx Namespace: ingress-nginx Labels: app.kubernetes.io/component=controller app.kubernetes.io/instance=ingress-nginx app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=ingress-nginx app.kubernetes.io/part-of=ingress-nginx app.kubernetes.io/version=1.10.0 helm.sh/chart=ingress-nginx-4.10.0 Annotations: Image pull secrets: Mountable secrets: Tokens: Events:

Name: ingress-nginx-admission Namespace: ingress-nginx Labels: app.kubernetes.io/component=admission-webhook app.kubernetes.io/instance=ingress-nginx app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=ingress-nginx app.kubernetes.io/part-of=ingress-nginx app.kubernetes.io/version=1.10.0 helm.sh/chart=ingress-nginx-4.10.0 Annotations: helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded Image pull secrets: Mountable secrets: Tokens: Events:

6. kubectl get all,ing
```sh
$ kubectl get all,ing
NAME                                            READY   STATUS      RESTARTS   AGE
pod/ingress-nginx-admission-create-2vf6g        0/1     Completed   0          3d
pod/ingress-nginx-admission-patch-cxtx8         0/1     Completed   2          3d
pod/ingress-nginx-controller-6484977b56-tp7tr   1/1     Running     0          2d23h
pod/test0-574c47cb97-fzhjf                      1/1     Running     0          3d

NAME                                         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
service/ingress-nginx-controller             NodePort    10.96.31.140   <none>        80:32691/TCP,443:31098/TCP   3d
service/ingress-nginx-controller-admission   ClusterIP   10.96.33.172   <none>        443/TCP                      3d
service/test0                                ClusterIP   10.96.41.149   <none>        80/TCP                       3d

NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/ingress-nginx-controller   1/1     1            1           3d
deployment.apps/test0                      1/1     1            1           3d

NAME                                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/ingress-nginx-controller-6484977b56   1         1         1       2d23h
replicaset.apps/ingress-nginx-controller-659c6c4948   0         0         0       3d
replicaset.apps/ingress-nginx-controller-77d966f98c   0         0         0       3d
replicaset.apps/ingress-nginx-controller-7fd476c957   0         0         0       3d
replicaset.apps/test0-574c47cb97                      1         1         1       3d

NAME                                       COMPLETIONS   DURATION   AGE
job.batch/ingress-nginx-admission-create   1/1           73s        3d
job.batch/ingress-nginx-admission-patch    1/1           86s        3d

NAME                                                                   CLASS             HOSTS   ADDRESS        PORTS   AGE
ingress.networking.k8s.io/ingress-class-name-no-perm                   nginx             *       10.96.31.140   80      2d23h
ingress.networking.k8s.io/ingress-from-annotation                      <none>            *                      80      2d23h
ingress.networking.k8s.io/ingress-invalid-ingress-class-name-no-perm   nginx-not-match   *                      80      2d23h

7. kubectl describe ing

   
   $ kubectl describe ing
   Name:             ingress-class-name-no-perm
   Labels:           <none>
   Namespace:        ingress-nginx
   Address:          10.96.31.140
   Ingress Class:    nginx
   Default backend:  <default>
   Rules:
   Host        Path  Backends
   ----        ----  --------
   *
             /demo/http1   test0:80 (10.244.0.8:80)
   Annotations:  nginx.ingress.kubernetes.io/rewrite-target: /
   Events:       <none>

Name: ingress-from-annotation Labels: Namespace: ingress-nginx Address: Ingress Class: Default backend: Rules: Host Path Backends

---

* /demo/http3 test0:80 (10.244.0.8:80) Annotations: kubernetes.io/ingress.class: nginx-annotation nginx.ingress.kubernetes.io/rewrite-target: / Events:

Name: ingress-invalid-ingress-class-name-no-perm Labels: Namespace: ingress-nginx Address: Ingress Class: nginx-not-match Default backend: Rules: Host Path Backends

---

* /demo/http2 test0:80 (10.244.0.8:80) Annotations: nginx.ingress.kubernetes.io/rewrite-target: / Events:

8. kubectl get events
```sh
$ kubectl get events
No resources found in ingress-nginx namespace.

9. curl test0.local -v

curl test0.local -v

GET http://test0.local/ HTTP/1.1 Host: test0.local User-Agent: curl/7.76.1 Accept: / Proxy-Connection: Keep-Alive

• Mark bundle as not supporting multiuse
• HTTP 1.0, assume close after body < HTTP/1.0 503 Service Unavailable < Connection: close <
• Closing connection 0 DNS lookup failed

$ kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-nginx-controller NodePort 10.96.31.140 80:32691/TCP,443:31098/TCP 3d ingress-nginx-controller-admission ClusterIP 10.96.33.172 443/TCP 3d test0 ClusterIP 10.96.41.149 80/TCP 3d

kubectl exec -it ingress-nginx-controller-6484977b56-tp7tr sh

/etc/nginx $ curl http://test0.local -v

• Could not resolve host: test0.local
• Closing connection curl: (6) Could not resolve host: test0.local

/etc/nginx $ curl http://test0 -v

• Host test0:80 was resolved.
• IPv6: (none)
• IPv4: 10.96.41.149
• Trying 10.96.41.149:80...
• Connected to test0 (10.96.41.149) port 80

  GET / HTTP/1.1 Host: test0 User-Agent: curl/8.5.0 Accept: /

  < HTTP/1.1 200 OK < Server: nginx/1.25.4 < Date: Thu, 18 Apr 2024 08:23:19 GMT < Content-Type: text/html < Content-Length: 615 < Last-Modified: Wed, 14 Feb 2024 16:20:36 GMT < Connection: keep-alive < ETag: "65cce854-267" < Accept-Ranges: bytes < <!DOCTYPE html>

  Welcome to nginx!

  If you see this page, the nginx web server is successfully installed and working. Further configuration is required.

For online documentation and support please refer to nginx.org.
Commercial support is available at nginx.com.

Thank you for using nginx.

• Connection #0 to host test0 left intact

[longwuyuan]

• I think I am totally lost. Maybe someone else can make sense out of the data you provided
• You want namespaced controller but your data above show clusterrole. So your controller is not namespaced
• You want namespaced ingressClassName. I don't know what that is as defined in K8S upstream KEP. And your data above does not show any namespaced ingressClassName

[yong-jie-gong]

• I think I am totally lost. Maybe someone else can make sense out of the data you provided
• You want namespaced controller but your data above show clusterrole. So your controller is not namespaced
• You want namespaced ingressClassName. I don't know what that is as defined in K8S upstream KEP. And your data above does not show any namespaced ingressClassName

@longwuyuan
1) This is namespaced rolebinding createing by "make dev-env"

 kubectl get  rolebinding  ingress-nginx -oyaml

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:   
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
    helm.sh/chart: ingress-nginx-4.10.0
  name: ingress-nginx
  namespace: ingress-nginx
  resourceVersion: "646"
  uid: 1a2703cc-a10e-46be-94b5-b2eedfe8d4ea
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: ingress-nginx

2) Regarding namespaced ingressClassName, there is no such thing in k8s. in IngressClass object is cluster level resources. Cluster level permission is mandatory to access it. k8s defined namespaced IngressClass mandate cluster level permisison as below

It break back-award compatiblity. to use ingressClassName in ingress.spec, cluter permission is mandatory for nginx-ingress-controller. that is why i raise this enhancement. with this enhancement, nginx-ingress-controller can manage ingresses with ingress.spec.ingressClassName in specific namespaces without cluster level permission.

[longwuyuan]

Apologies. I am 100% lost. Wait for other comments.

[github-actions[bot]]

This is stale, but we won't close it automatically, just bare in mind the maintainers may be busy with other tasks and will reach your issue ASAP. If you have any question or request to prioritize this, please reach #ingress-nginx-dev on Kubernetes Slack.

[yong-jie-gong]

still in discussion, should not be closed