Using the `@ExecuteOn=ThreadExecutors.IO` on endpoints breaks new relic instrumentation

[saihegde]

I have a REST Controller which collects data from a bunch of rest endpoints... New Relic instrumentation for the external calls works when I let micronaut determine the thread pool automagically.. but this does not work great always as 1/10 calls will show some latency in the responses being returned. Explicitly setting it to the ThreadExecutors.IO pool fixed the latency issue but broke the new relic instrumentation.

Thoughts?

[graemerocher]

Provide an example with steps to reproduce, hard to say if the bug is in the New Relic instrumentation code (which I am unaware of) or here

[saihegde]

Maybe we can start with something simpler.

Steps to recreate -

Start with a simple micronaut project

pom.xml

<dependency>
      <groupId>com.newrelic.agent.java</groupId>
      <artifactId>newrelic-api</artifactId>
      <version>6.0.0</version>
      <scope>provided</scope>
</dependency>

City.java

public class City {

    private String code;
    private String name;

    public String getCode() {
        return code;
    }

    public void setCode(String code) {
        this.code = code;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

}

import com.newrelic.api.agent.NewRelic;
import io.micronaut.http.annotation.Controller;
import io.micronaut.http.annotation.Get;
import io.micronaut.scheduling.TaskExecutors;
import io.micronaut.scheduling.annotation.ExecuteOn;

@Controller("/city")
//@ExecuteOn(TaskExecutors.IO)
public class CityController {

    @Get("/{code}")
    City get(String code) {
        NewRelic.setTransactionName(null, "GET_CITY_BY_CODE");
        City city = new City();
        city.setCode(code);
        city.setName("Salt Lake City");
        return city;
    }

}

If I use the ExecuteOn annotation, New Relic Instrumentation breaks. All calls show as NettyDispatcher as opposed to the endpoint. If I do not use the ExecuteOn annotation, the response times are not consistent. You can run the same thing over and over again and sometimes you get 100ms, 15 seconds, or a complete timeout.

I can add a more complex example with httpclient if needed.