open-telemetry / opentelemetry-go-instrumentation

OpenTelemetry Auto Instrumentation using eBPF
https://opentelemetry.io
Apache License 2.0
493 stars 77 forks source link

Otel Auto Config - Tracing name is showing only http method but url is missing #896

Open arasuse opened 3 months ago

arasuse commented 3 months ago

Describe the bug

I'm using Otel Auto Config with agent-extension for tracing. In GKE under tracing screen i see only http method but url is missing. Can someone help me to resolve the issue?

Environment

image

damemi commented 3 months ago

@arasuse sorry but I don't quite understand your question, could you please explain more where the url is missing? It would also help to see what config you are running and an example YAML of your deployment with the auto-instrumentation agent if possible

NhatVuDuy commented 2 months ago

@arasuse sorry but I don't quite understand your question, could you please explain more where the url is missing? It would also help to see what config you are running and an example YAML of your deployment with the auto-instrumentation agent if possible

I think that I have the same problem. When I use go-otel manually, I set the trace name to request endpoint. It helps to trace easier. But when I use go-otel auto instrumentation, the trace name is set to http method request. I have been trying to find the config for getting this feature, but have not found yet.

This is what I get from Kibana transaction list when using coding integrated instrumentation (What we need): image

And this is what I get from Kibana transaction list when using auto instrumentation: image

Thank you!

damemi commented 2 months ago

@NhatVuDuy ah, I understand now. We do support {method} {target} where there is a low-cardinality {target} value available. Starting in https://github.com/open-telemetry/opentelemetry-go-instrumentation/pull/740, this is automatically parsed from templated path names in Go 1.22+.

For versions below that, we don't support {target} values because we don't know the cardinality of the path. So just using {method} is compliant with Otel semantic conventions.

We don't have any configuration to enable this. But I think we could add that as a feature. I also think it might be possible for us to use {method} {target} as the span name when the path doesn't have any arguments. @RonFed wdyt?

RonFed commented 2 months ago

@damemi That sounds good to me. We can add this as an Opt-in, similar to what we have to include db query.

damemi commented 2 months ago

@RonFed cool, we can look into it. @NhatVuDuy unfortunately right now we're prioritizing work for our beta release which is more focused on stability than features, so this won't be a top priority at the moment. But we will keep this open to track the feature request, thanks!

NhatVuDuy commented 2 months ago

@RonFed cool, we can look into it. @NhatVuDuy unfortunately right now we're prioritizing work for our beta release which is more focused on stability than features, so this won't be a top priority at the moment. But we will keep this open to track the feature request, thanks!

Thank you so much. I will check my go version to ensure it's supported for this feature. About adding this as an Opt-in, it's great. No rush to do. Just keep your team's plan. In case needed, I will try to modify the base source and build my personal docker image for this temporarily. I did it once before, when the otel service could not read my kafka version then crash. So I disabled kafka tracing from base source. Maybe it's my service fault.

Thanks again!

NhatVuDuy commented 1 month ago

@NhatVuDuy ah, I understand now. We do support {method} {target} where there is a low-cardinality {target} value available. Starting in #740, this is automatically parsed from templated path names in Go 1.22+.

For versions below that, we don't support {target} values because we don't know the cardinality of the path. So just using {method} is compliant with Otel semantic conventions.

We don't have any configuration to enable this. But I think we could add that as a feature. I also think it might be possible for us to use {method} {target} as the span name when the path doesn't have any arguments. @RonFed wdyt?

Hi @damemi,

I upgraded my go version to 1.22.0, but the tracing data still show {method} only (attached photo below). Go version: 1.22.0 Transaction name: POST (not included path) Request path: /intake/v2/events Monitoring system: Kibana

image Do you have any suggestion for this?

BTW, Do we have any option to disable a service (e.g kafka) from auto tracing? Thank you.

damemi commented 1 month ago

@NhatVuDuy are you using routing patterns in your code? That is what auto-instrumentation relies on to know it's a low-cardinality route. That feature became available in go 1.22

RonFed commented 1 month ago

@damemi I think those are client spans, hence the path templates are not relevant in that case.

NhatVuDuy commented 1 month ago

Thank @dameni @RonFed . Let me check my code. Maybe I didn't understanding something.

NhatVuDuy commented 1 month ago

@NhatVuDuy are you using routing patterns in your code? That is what auto-instrumentation relies on to know it's a low-cardinality route. That feature became available in go 1.22

Sorry for my late response. What a pity, I am using Bone mux (https://github.com/go-zoo/bone), a wrapper lib of net/http, for routing. Maybe it makes this otel feature not work.