search

istio / old_issues_repo

Deprecated issue-tracking repo, please post new issues or feature requests to istio/istio instead.
37 stars 9 forks source link

Exclude health checks from telemetry #252

Open jsw opened 6 years ago

jsw commented 6 years ago

Feature request

Describe the feature:

Add ability to exclude GLB and kubernetes health checks from telemetry. The paths could either be inferred from the environment and/or allow users to specify a set of paths or path expressions.

jsw commented 6 years ago

On further thought, this is probably better done later in the flow, such as in the Grafana charts.

sakshigoel12 commented 6 years ago

@douglas-reid can you take a look at this request

douglas-reid commented 6 years ago

It should be possible to use Mixer config and various attributes to handle (at least some of ) this via config today (depending on deployment).

match: request.path != "/healthz"
...

It would be nice to have a way to annotate, in service config, paths not meant for monitoring and have that pass down into Envoy filter configuration.

catalinpan commented 5 years ago

I'm trying to exclude the health checks from tracing using the example from the documentation but it doesn't really work for me https://istio.io/help/faq/telemetry/#controlling-what-the-sidecar-reports Can I have some help please on where this flag should be set?

douglas-reid commented 5 years ago

@catalinpan is there a given path (or pattern of paths) that you use for health checking? the example on the FAQ uses the source.name attribute, but that could easily be switched to something like request.path (as in my example above).

catalinpan commented 5 years ago

@douglas-reid thanks for the reply. The path is FQDN/_health but I'm not sure where I should add that flag. Is part of a rule similar to this example https://raw.githubusercontent.com/istio/istio/release-1.0/samples/bookinfo/policy/mixer-rule-deny-label.yaml or should be set somewhere else? When I use source.name as a part of a rule (like on the above mixer example) it blocks everything from accessing the health check endpoint.

douglas-reid commented 5 years ago

@catalinpan are you using Mixer for trace span generation, or do you have the proxies (envoy) configured for tracing in your install?

catalinpan commented 5 years ago

@douglas-reid if you are referring on how the http headers are set on each request, the apps are setting the tracing http headers based on the istio docs from here https://istio.io/docs/tasks/telemetry/distributed-tracing/#understanding-what-happened I have tested today also this without any luck https://github.com/istio/old_issues_repo/issues/335#issuecomment-395114771

Is there any sample configuration I can adapt to use the match from the istio documentation?

douglas-reid commented 5 years ago

@catalinpan i'm wondering if you are using Mixer for tracing (with explicit config for tracespans) or if you are using Envoy. If you are using Envoy (the default case), then there are no match clauses etc that you can use, as Mixer isn't in the path.

catalinpan commented 5 years ago

@douglas-reid it looks like we are using the default Envoy. I will check what options there are for this. Thanks for all your help!