search

jaegertracing / jaeger-client-csharp

🛑 This library is DEPRECATED!
https://jaegertracing.io/
Apache License 2.0
302 stars 67 forks source link

Infinite span creation for Jaeger internal requests when OpenTracing.Contrib.NetCore and HttpSender are used #154

Closed bocharovf closed 5 years ago

bocharovf commented 5 years ago

Requirement - what kind of business use case are you trying to solve?

Hide Jaeger internal request traces in Jaeger UI to simplify trace searching for customers.

Problem - what in Jaeger blocks you from solving the requirement?

If JAEGER_ENDPOINT property is specified Jaeger uses HttpSender to send spans via Http directly to Jaeger Collector. OpenTracing.Contrib.NetCore package uses DiagnosticSource to intercept each Http request and build a span. It leads to infinite span creation loop because each POST request to collector with span data creates another span and cause another POST request. Huge amount of such traces make it difficult to search real user traces.

изображение изображение

Proposal - what do you suggest to solve the problem or improve the existing situation?

Make OpenTracing.Contrib.NetCore ignore request to Jaeger Collector by specifing "ot-ignore" request property. It's already done in DefaultHttpClient to hide receiving of sampling settings. The same could be done in THttpClientTransport

See also HttpHandlerDiagnosticOptions in OpenTracing.Contrib.NetCore

Falco20019 commented 5 years ago

I‘m currently not sure in which case this loop appears that you mention. I‘m using the endpoint directly myself and have not stumbled over this behavior. Can you explain more the circumstances?

I think you are most probably missing this here: https://github.com/opentracing-contrib/csharp-netcore/blob/5d9c34090d7b4741bbf1bee7bfc7b2aa2b1ebda1/samples/Shared/JaegerServiceCollectionExtensions.cs#L40-L44

Since this is something that arises on the OpenTracing level, I‘m not sure that fixing it in Jaeger as an OpenTracing implementation would be the best solution. When changing to another implementation, the problem would reoccur. So using the ignore pattern in the Lib should be the cleanest solution as shown in the sample.

bocharovf commented 5 years ago

Well, yes, we currently use workaround you provided and it works.

But in our case we have 240+ products and approximately the same number of teams. Some of them use .net core, some of them not. Using workaround provided above, .net core teams have to deal with the details of Jaeger / OpenTracing internal implementation for .net core. While other teams (e.g. Java teams) don't need that. So we get additional ".net core only" step to configure Jaeger. if we could avoid it it would be great.

And the main question is - why do we solve that issue for DefaultHttpClient in Jaeger csharp client (using ot-ignore parameter) and do not want to solve it for THttpClientTransport?

Falco20019 commented 5 years ago

Thanks for the input. I will give it another look later.