kubernetes / ingress-nginx

Ingress-NGINX Controller for Kubernetes
https://kubernetes.github.io/ingress-nginx/
Apache License 2.0
16.93k stars 8.14k forks source link

Ingress controller with scoped ingress class validates ingress of other class #11505

Open yene opened 1 week ago

yene commented 1 week ago

What happened:

We have two ingress controllers running, each with its own ingressclass. Problem is, they read and try to validate each others ingressclass, which causes multiple different error logs, like

"Ignoring ingress because of error while validating ingress class"
I0625 12:42:50.077264       7 store.go:436] "Ignoring ingress because of error while validating ingress class" ingress="default/ingress-dummy" error="no object matching key \"dummyclass\" in local store"

This is dangerous since it shadows real errors and causes alarm fatique.

What you expected to happen:

Each ingress controller only fetches the ingress selected by --ingress-class=.

NGINX Ingress controller version (exec into the pod and run nginx-ingress-controller --version.): v1.10.0

Kubernetes version (use kubectl version): v1.29.2

Environment:

fullnameOverride: ingress-test-internal
controller:
  ingressClass: ingress-test-internal
  scope:
    enabled: true # only checks the $(POD_NAMESPACE) for resources
  ingressClassResource:
    name: ingress-test-internal
    enabled: true
    default: false
    controllerValue: 'k8s.io/ingress-test-internal'

ingress

error │ W0625 12:40:54.609709 7 controller.go:331] ignoring ingress robotstxt-dummy in default based on annotation : no object matching key "dummy" in local store │ │ I0625 12:40:54.609748 7 main.go:107] "successfully validated configuration, accepting" ingress="default/ingress-dummy"

How to reproduce this issue:

  1. create an ingress-nginx A that listen to a dedicated ingressClass A
  2. deploy ingress with different ingressclass B

Problem: A will complain about ingress that belong to class B

k8s-ci-robot commented 1 week ago

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
longwuyuan commented 1 week ago

/remove-kind bug /kind support

I am almost certain that you need to check if both the controllers are different as per this example https://kubernetes.github.io/ingress-nginx/faq/#how-can-i-easily-install-multiple-instances-of-the-ingress-nginx-controller-in-the-same-cluster

yene commented 6 days ago

@longwuyuan election and controllers are different, everything is already deployed and works.

Just these annoying error messages about validation of unrelated Ingress. Well it is probably helpful for some people, but can it be turned off?

longwuyuan commented 6 days ago

any chance you can show the cluster details in a screenshare ?

longwuyuan commented 6 days ago

In any case I will mark it as a feature-request for this issue.

I tested and I see the message logged, once, at the time of creating a ingress resource, in the logs of the controller that does not own the ingressClass, of the newly created ingress resource.

The controllers watch all ingresses but only process the ingresses that are spec'd out with the ingressClass they own. This message to me is a info level log (need to check the code for exact spot) and there is no feature to optionally enable/disable such info log.

I guess if you are reporting then it must be impacting you but I am not inclined to expect that disabling this log message can be implemented in the near future, due to multiple reasons. Having it tracked here as a feature request seem the appropriate state for now.