search

DataDog / dd-trace-java

Datadog APM client for Java
https://docs.datadoghq.com/tracing/languages/java
Apache License 2.0
583 stars 289 forks source link

undertow_http.request span does not always fully enclose it's child spans #3551

Open shiyanhui opened 2 years ago

shiyanhui commented 2 years ago

Hi, recently I tested the latest agent version, i.e. dd-java-agent-0.100.0.jar and found in the APM that the undertow-2.0 instrumentation seems have some problem.

Please see this screenshot below, the undertow_http.request span is created by undertow-2.0 instrumentation, and the StatsController.ready span is created by spring-webflux instrumentation. I understand that the correct behavior is that the undertow_http.request span should be longer than the StatsController.ready span, right?

Screen Shot 2022-05-12 at 22 51 48

Another problem is that we didn't find the servlet.request span which is created in undertow-2.0 instrumentation, why?

Thanks!

devinsba commented 2 years ago

1) We are aware of this timing discrepancy, we haven't been able to address this yet but it is on our radar. As for suitable for production, if those ~200microseconds are important to you and your metrics then not at this time, otherwise it should be fine.

2) You don't see a servlet.request span because you are using webflux which does not work on top of a servlet engine so would be added into undertow as a handler

shiyanhui commented 2 years ago

We are using undertow and webflux in our tech stack, so how can we get the servlet.request span or trace in the APM? I mean, is there any workaround way to get it?

devinsba commented 2 years ago

Assuming this is using spring-boot you would probably have to configure it differently. I can't find an example of that configuration though. See: https://docs.spring.io/spring-framework/docs/current/reference/html/web-reactive.html#webflux-server-choice

For Undertow, Spring WebFlux uses Undertow APIs directly without the Servlet API.

In this case we won't create a servlet.request span because the servlet handler will not be used. From what I can tell you would have to use embedded tomcat or jetty to get the servlet.request span. Is having a span with that operation name required for you, or can you instead use undertow_http.request?

shiyanhui commented 2 years ago

Most our services use spring-mvc+undertow and several services use webflux+undertow, and we configure the monitoring metrics in one place and we want the query syntax to follow the same pattern. So it's better to emit the servlet.request trace.