Open lambchr opened 3 years ago
/ assign
Please upgrade to 0.46.0 and see if this is still an issue.
/triage needs-information
Hi @strongjz sorry for the delayed response. I upgrade to 0.47.0 and repeated my above test (created new debug ingress, waited for host and address to appear, checked if metric appeared) and could still not see any nginx_ingress_controller_requests
metrics from the /metrics
endpoint of any NGINX pod in the cluster. When I curled my new debug ingress and checked the NGINX pods /metrics
endpoint again it showed me an nginx_ingress_controller_requests
metric for that ingress. Please let me know if you need any more info from me :)
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
lifecycle/stale
is appliedlifecycle/stale
was applied, lifecycle/rotten
is appliedlifecycle/rotten
was applied, the issue is closedYou can:
/remove-lifecycle stale
/lifecycle rotten
/close
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
/remove-lifecycle stale
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
lifecycle/stale
is appliedlifecycle/stale
was applied, lifecycle/rotten
is appliedlifecycle/rotten
was applied, the issue is closedYou can:
/remove-lifecycle stale
/lifecycle rotten
/close
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
/remove-lifecycle stale
Hey @strongjz do you need any more information?
i can confirm this behavior on version v1.0.4 / 4.0.6 where the metric nginx_ingress_controller_requests
is missing until you do any request (for example curl) to an ingress resource.
/triage accepted /priority important-longterm
We are happy for any contributions /help /good-first-issue
@iamNoah1: This request has been marked as suitable for new contributors.
Please ensure that the issue body includes answers to the following questions:
For more details on the requirements of such an issue, please see here and ensure that they are met.
If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-good-first-issue
command.
/remove needs-information
/remove triage/needs-information
/remove triage-needs-information
The issue seems to be a bit old, but I'm presenting my thoughts here to provide some closure to others who might see this.
I feel the problem at hand is that the metric itself could have multiple labels, and those labels could have multiple values. This makes it harder to initialize nginx_ingress_controller_requests
for all combination of labels and values (maybe, some values might not even be known until the request arrives). Something like https://github.com/prometheus/client_golang/issues/190 might be helpful but even that might have problems with dynamic values of labels.
@iamNoah1 I am new to open source and Kubernetes, and I would like to work on this issue. Extra help would be great. šš
@johurul000 can you write your own technical description of the problem to be solved here
@longwuyuan no, since I am new, guidance is very helpful
I mean is the metric missing in your cluster also ?
Hope it helps a fellow user, I was using wildcard domain and I have to add --metrics-per-host=false
for this metric to work.
Hey, is anyone still working on this? I think I can take it up if someone can guide me on how to achieve this.
Hey @strongjz and @lambchr, the problem appointed in this issue still need to be fixed? If so I would like to work on this issue.
I too am running into this issue when trying to implement Flagger which refers to this metric in the documents. Would love an update on this.
I'm running into this issue while using NGINX Ingress Prometheus Overview in GCP Monitoring with the latest version.
imo this is a prometheus issue, they could introduce a function that returns a 0 if null. This isn't an issue specific to nginx_ingress_controller_requests.
imo this is a prometheus issue, they could introduce a function that returns a 0 if null. This isn't an issue specific to nginx_ingress_controller_requests.
Have you tried to report this "issue" to Prometheus? If yes, please share the link to your report.
imo this is a prometheus issue, they could introduce a function that returns a 0 if null. This isn't an issue specific to nginx_ingress_controller_requests.
Have you tried to report this "issue" to Prometheus? If yes, please share the link to your report.
I haven't, I have read read somewhere it's the responsibility of the metric reporter to pre populate the 0's. I can't find the github issue page now.
I can easily reproduce this and can see that the metric doesn't appear until requests come through. However, I'm not sure there's an obvious fix. We could just set the metric to 0 when an ingress object is found, but the metric includes labels like: method, path and status (as in http status code). What should these be set too when we set the value to 0? I'm not sure there's a nice way to do this so that the metric appears right away and we don't end up with metrics that will always be 0 because the default labels we choose are never matched with a real request.
@rikatz I know that I have a followup for the tests, but if you don't mind I would like to work on this issue :D
/assign @StuxxNet Go for it
Any update on this?
NGINX Ingress controller version: 0.30.0
Kubernetes version (use
kubectl version
): 1.15Environment:
What happened:
The
nginx_ingress_controller_requests
metric was missing for Ingresses that have had 0 requests. I queriednginx_ingress_controller_requests == 0
for our prometheus metrics and found no time series.What you expected to happen:
I expected ingresses that have had no requests sent to them to have a
nginx_ingress_controller_requests
metric with a count of 0 rather than not being present.How to reproduce it:
nginx_ingress_controller_requests{ingress="test"}
exec
'd to a pod on the cluster and hit the NGINX controller's/metrics
endpoint to check if our prometheus stack was filtering out the metric, but it didn't appear there eitherAnything else we need to know:
Please let me know if you need any other info from me, thanks for your time :)
/kind bug