search

spring-cloud / spring-cloud-stream-binder-kafka

Spring Cloud Stream binders for Apache Kafka and Kafka Streams
Apache License 2.0
331 stars 301 forks source link

BindersHealthContributor throws NPE when access by actuate endpoint(health) #779

Closed fanticat closed 4 years ago

fanticat commented 4 years ago

When a reactive(webflux) project including following pom, and no define any "Input" or "Output" channel, then get health info in browser "GET http://hostname:port/actuate/health", ReactiveWebOperationAdapter throws exception, just disable binder actuate is OK.

 <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-starter-stream-kafka</artifactId>
  </dependency>

disable it is OK.

management:
  health:
    binders:
      enabled: false
java.lang.NullPointerException: null
    at reactor.core.publisher.MonoZip.subscribe(MonoZip.java:128)
    Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Assembly trace from producer [reactor.core.publisher.MonoLiftFuseable] :
    reactor.core.publisher.Mono.flatMap(Mono.java:2713)
    org.springframework.boot.actuate.endpoint.web.reactive.AbstractWebFluxEndpointHandlerMapping$ReactiveWebOperationAdapter.handle(AbstractWebFluxEndpointHandlerMapping.java:323)
Error has been observed at the following site(s):
    |_           Mono.flatMap ? at org.springframework.boot.actuate.endpoint.web.reactive.AbstractWebFluxEndpointHandlerMapping$ReactiveWebOperationAdapter.handle(AbstractWebFluxEndpointHandlerMapping.java:323)
    |_           Mono.flatMap ? at org.springframework.web.reactive.result.method.annotation.ResponseEntityResultHandler.handleResult(ResponseEntityResultHandler.java:130)
    |_             checkpoint ? Handler org.springframework.boot.actuate.endpoint.web.reactive.AbstractWebFluxEndpointHandlerMapping$ReadOperationHandler#handle(ServerWebExchange) [DispatcherHandler]
    |_             Mono.error ? at org.springframework.web.reactive.result.method.annotation.RequestMappingHandlerAdapter.handleException(RequestMappingHandlerAdapter.java:238)
sobychacko commented 4 years ago

@fanticat Can you share a small sample application (or your maven pom.xml) where we can reproduce this issue?

fanticat commented 4 years ago

@sobychacko

Springboot version 2.2.0.RELEASE, Springcloud version Hoxton.M3

<dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-webflux</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-stream-kafka</artifactId>
        </dependency>
    </dependencies>
sobychacko commented 4 years ago

@fanticat Can you upgrade to Hoxton.RC1 or the latest snapshot and try again? I am able to get the proper response from the health endpoint with and without any input/output bindings.

Here is part of the maven pom.

         <properties>
                <java.version>1.8</java.version>
        <spring-cloud.version>Hoxton.BUILD-SNAPSHOT</spring-cloud.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-webflux</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream-binder-kafka</artifactId>
        </dependency>
        ...
                 ...
    </dependencies>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

and here is a sample application:

@SpringBootApplication
public class DemoApplication {

    @Bean
    public Function<String, String> hello() {
        return s -> s;
    }

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }

}
matthewhaywardmsm commented 4 years ago

This is still an issue using Hoxton.RC1. I've added an example project to demonstrate the issue.

https://github.com/matthewh86/spring-cloud-stream-binder-kafka-issue-779

The test fails with the NPE

java.lang.NullPointerException: null
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[na:1.8.0_232]
    at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[na:1.8.0_232]
    at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) ~[na:1.8.0_232]
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[na:1.8.0_232]
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[na:1.8.0_232]
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[na:1.8.0_232]
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[na:1.8.0_232]
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566) ~[na:1.8.0_232]
    at org.springframework.boot.actuate.health.HealthEndpointSupport.getCompositeHealth(HealthEndpointSupport.java:136) ~[spring-boot-actuator-2.2.0.RELEASE.jar:2.2.0.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpointWebExtension.aggregateContributions(HealthEndpointWebExtension.java:98) ~[spring-boot-actuator-2.2.0.RELEASE.jar:2.2.0.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpointWebExtension.aggregateContributions(HealthEndpointWebExtension.java:42) ~[spring-boot-actuator-2.2.0.RELEASE.jar:2.2.0.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpointSupport.getAggregateHealth(HealthEndpointSupport.java:124) ~[spring-boot-actuator-2.2.0.RELEASE.jar:2.2.0.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpointSupport.getContribution(HealthEndpointSupport.java:103) ~[spring-boot-actuator-2.2.0.RELEASE.jar:2.2.0.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpointSupport.getHealth(HealthEndpointSupport.java:81) ~[spring-boot-actuator-2.2.0.RELEASE.jar:2.2.0.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpointSupport.getHealth(HealthEndpointSupport.java:68) ~[spring-boot-actuator-2.2.0.RELEASE.jar:2.2.0.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpointWebExtension.health(HealthEndpointWebExtension.java:80) ~[spring-boot-actuator-2.2.0.RELEASE.jar:2.2.0.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpointWebExtension.health(HealthEndpointWebExtension.java:69) ~[spring-boot-actuator-2.2.0.RELEASE.jar:2.2.0.RELEASE]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_232]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_232]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_232]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_232]
    at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:279) ~[spring-core-5.2.0.RELEASE.jar:5.2.0.RELEASE]
    at org.springframework.boot.actuate.endpoint.invoke.reflect.ReflectiveOperationInvoker.invoke(ReflectiveOperationInvoker.java:77) ~[spring-boot-actuator-2.2.0.RELEASE.jar:2.2.0.RELEASE]
    at org.springframework.boot.actuate.endpoint.annotation.AbstractDiscoveredOperation.invoke(AbstractDiscoveredOperation.java:60) ~[spring-boot-actuator-2.2.0.RELEASE.jar:2.2.0.RELEASE]
    at org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$ServletWebOperationAdapter.handle(AbstractWebMvcEndpointHandlerMapping.java:305) ~[spring-boot-actuator-2.2.0.RELEASE.jar:2.2.0.RELEASE]
    at org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(AbstractWebMvcEndpointHandlerMapping.java:388) ~[spring-boot-actuator-2.2.0.RELEASE.jar:2.2.0.RELEASE]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_232]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_232]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_232]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_232]
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190) ~[spring-web-5.2.0.RELEASE.jar:5.2.0.RELEASE]
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) ~[spring-web-5.2.0.RELEASE.jar:5.2.0.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106) ~[spring-webmvc-5.2.0.RELEASE.jar:5.2.0.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:888) ~[spring-webmvc-5.2.0.RELEASE.jar:5.2.0.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793) ~[spring-webmvc-5.2.0.RELEASE.jar:5.2.0.RELEASE]
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.2.0.RELEASE.jar:5.2.0.RELEASE]
    at org.springframework.boot.actuate.autoconfigure.web.servlet.CompositeHandlerAdapter.handle(CompositeHandlerAdapter.java:58) ~[spring-boot-actuator-autoconfigure-2.2.0.RELEASE.jar:2.2.0.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040) ~[spring-webmvc-5.2.0.RELEASE.jar:5.2.0.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943) ~[spring-webmvc-5.2.0.RELEASE.jar:5.2.0.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) ~[spring-webmvc-5.2.0.RELEASE.jar:5.2.0.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) ~[spring-webmvc-5.2.0.RELEASE.jar:5.2.0.RELEASE]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) ~[tomcat-embed-core-9.0.27.jar:9.0.27]
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) ~[spring-webmvc-5.2.0.RELEASE.jar:5.2.0.RELEASE]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) ~[tomcat-embed-core-9.0.27.jar:9.0.27]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat-embed-core-9.0.27.jar:9.0.27]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.27.jar:9.0.27]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) ~[tomcat-embed-core-9.0.27.jar:9.0.27]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-9.0.27.jar:9.0.27]
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:526) [tomcat-embed-core-9.0.27.jar:9.0.27]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) [tomcat-embed-core-9.0.27.jar:9.0.27]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat-embed-core-9.0.27.jar:9.0.27]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) [tomcat-embed-core-9.0.27.jar:9.0.27]
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) [tomcat-embed-core-9.0.27.jar:9.0.27]
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408) [tomcat-embed-core-9.0.27.jar:9.0.27]
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-9.0.27.jar:9.0.27]
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861) [tomcat-embed-core-9.0.27.jar:9.0.27]
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1579) [tomcat-embed-core-9.0.27.jar:9.0.27]
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-9.0.27.jar:9.0.27]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_232]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_232]
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-9.0.27.jar:9.0.27]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_232]
sobychacko commented 4 years ago

@matthewhaywardmsm This looks like testing related issue. When I run the actual application and hit the health endpoint, I get the proper response back. However, when running the test, I am not seeing binding infrastructure code kicks in. No EnableBinding processing happening in the test and thus no health check components for the binders. I believe it is the source of that NPE. Can you debug further to see if its anyhow related to your test fixture?

matthewhaywardmsm commented 4 years ago

@sobychacko There's nothing special about the test config, it just uses the @SpringBootTest annotation to bring up the context. The configuration should be the same for both the app running as normal, and the app running in the test.

I guess one question is why the @EnableBinding isn't working when the app is running up as part of the @SpringBootTest. And even if it isn't, the main question is why the health check endpoint is causing an NPE, which is related to the initial issue reported.

The example project gives an easy way to reproduce the healthcheck issue. From debugging the application startup, the HealthComponent created for bindings is null.

Null Binder Health

EDIT: IMO the bindings should have an UNKNOWN or DOWN status, or not even be present, rather than having a null value which causes the NPE issues.

sobychacko commented 4 years ago

@matthewhaywardmsm We will see if there are any issues from the testing angle.

sobychacko commented 4 years ago

@matthewhaywardmsm We were able to look further with @artembilan on the issue. Here is the problem. In your build dependency, you have this:

testImplementation 'org.springframework.cloud:spring-cloud-stream-test-support'

When you have this dependency, the test context will use this binder as the default binder. This binder does not provide any health indicators at all. This is the reason for the test failure. Since you are trying to use it with Kafka binder, I suggest that you remove this dependency and then rely on the production binder only - implementation 'org.springframework.cloud:spring-cloud-stream-binder-kafka'. When we tried it locally, the test passes. If you really want to use the test binder for any testing, you might want to isolate that into a separate module.

Hope this helps.

sobychacko commented 4 years ago

@matthewhaywardmsm @fanticat Closing this issue. Please re-open if you are still facing an issue with this.