Broken e2e transaction when using Spring Cloud Gateway (spring-webflux) with agent 3.0.3-BETA.*

paveljakov commented 3 years ago

Expected behavior

Single e2e transaction should be built for single operation incoming into reactive server and going out to downstream services.

Actual behavior

While using AI agent version 3.0.2 traces from reactive gateway to downstream services are correctly correlated and mapped to initial operation transaction (tx = [request -> reactive-gateway -> downstream-service -> ...]), but when using AI agent version 3.0.3-BETA or 3.0.3-BETA.2 e2e transaction is broke, request to reactive gateway is logged separately (different operation IDs, different transactions) from downstream service (tx1 = [request -> reactive-gateway], tx2 = [request -> downstream-service -> ...]). I have not tested reactive clients like org.springframework.web.reactive.function.client.WebClient directly.

To Reproduce

Steps to reproduce the behavior: simple Spring MVC application and Spring Cloud Gateway application, gateway routes to spring mvc application, both applications uses AI javaagent.

Sample Application

If applicable, provide a sample application which reproduces the issue.

System information

Please provide the following information:

SDK Version: 3.0.3-BETA.*
OS type and version: Docker openjdk:11.0.9-jre-slim
Application Server type and version (if applicable): Netty 0.9.10.RELEASE
Using spring-boot? Yes
Additional relevant libraries (with version, if applicable): spring-boot-starter-webflux:2.3.2.RELEASE

Logs

Turn on SDK logs and attach/paste them to the issue. If using an application server, also attach any relevant server logs.

Be sure to remove any private information from the logs before posting!

Screenshots

If applicable, add screenshots to help explain your problem.

Expected behavior: Single transaction

Actual behavior: Two separate transactions

trask commented 3 years ago

hi @paveljakov, thanks for reporting this! If you are able to provide a sample github repo with instructions for us how to run and see the issue occurring, that will help us to get to it faster.

paveljakov commented 3 years ago

Hello @trask, I have created sample git repo that reproduces the bug: https://github.com/paveljakov/ai-e2e-bug-sample Hope it helps.

paveljakov commented 3 years ago

Hi @trask, I have found a workaround. It seems problem is related to the fact that we are defining our own reactor.netty.http.client.HttpClient bean.

@Bean
public HttpClient httpClient() {
  final TcpClient tcpClient = buildTcpClient();
  final HttpClient httpClient = HttpClient.from(tcpClient);
  return httpClient.compress(true);
}

When I use HttpClientCustomizer, it starts to work correctly and e2e transactions are correlated correctly.

@Bean
public HttpClientCustomizer httpClientCustomizer() {
  return httpClient -> httpClient
      .compress(true)
      .tcpConfiguration(this::configureTcpClient);
}

With 3.0.3 AI agent version our custom bean is not getting instrumented. I still think that this is a bug, because providing custom reactor.netty.http.client.HttpClient ben is a valid usecase.

trask commented 3 years ago

Thanks @paveljakov!

This should be fixed now in the latest SNAPSHOT release:

applicationinsights-agent-3.0.3-BETA.4.jar.zip

microsoft / ApplicationInsights-Java