How to combine Nginx trace spans with backend service call trace spans into a single trace.

harshalschaudhari commented 6 months ago

Describe the bug

I am trying to make a call to user service from Nginx -> UserService -> Calculator service to perform an addition operation. I am using the docker image: nginx:otel

There are two separate traces coming one for unknown_service:nginx: and another UserRequestService: HTTP GET

unknown_service:nginx: screenshot as below

UserRequestService screenshot as below

A clear and concise description of what the bug is. nginx and user service trace span are coming separately, they should comes together.

To reproduce

Steps to reproduce the behavior:

Run below docker-compose Docker compose file otel-config.yaml file nginx.conf file
docker-compose up
call URL from browser "http://localhost:8082/api/UserRequest/add?num1=5&num2=24"

Expected behavior

Nginx trace span should come with User Service span. E.g. nginx -> proxy to the backend (User Service) -> Calculator Service

Your environment

Version/release of this project or specific commit
Target deployment platform

Additional context

Add any other context about the problem here.

I have tried Parent-based Tracing, but it give only User Service -> Calculator Service trace

p-pautov commented 6 months ago

It's not clear what is here to fix in Nginx OTel module. Can you restate the problem in terms of Nginx generated trace attributes/headers, i.e. do we need to add/change something?

I'd guess, that your "UserService" just ignores Nginx generated "traceparent" header.

harshalschaudhari commented 6 months ago

Thanks for the pointer, I have added the below the code for propagation

services.AddOpenTelemetryTracing( (builder) => builder .AddAspNetCoreInstrumentation() .AddHttpClientInstrumentation() .AddJaegerExporter(j => { j.AgentHost = serviceName; j.AgentPort = 6831; }) .SetResourceBuilder(ResourceBuilder.CreateDefault().AddService("UserRequest")) );

`

    <PackageReference Include="OpenTelemetry.Extensions.Hosting" Version="1.0.0-rc1.1" />
    <PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.0.0-rc1.1" />
    <PackageReference Include="OpenTelemetry.Instrumentation.Http" Version="1.0.0-rc1.1" />`

nginxinc / nginx-otel