openzipkin / zipkin-support

repository for support questions raised as issues
4 stars 2 forks source link

brave.mysql8 Tracing stuck on startup when running a Spring Boot + Wavefront application #34

Open odedia opened 4 years ago

odedia commented 4 years ago

Describe the Bug

When a Spring Boot application contains brave mysql8 tracing plus Spring Cloud Sleuth and Wavefront dependencies, it causes the application to freeze on startup after the HikariCP pool is being created. Specifically at line 52 of TracingQueryInterceptor.

Steps to Reproduce

Steps to reproduce the behavior: Clone this simple repo: https://github.com/odedia/todo-service

Start a local mariabdb docker container:

docker run --name mariadb -e MYSQL_ROOT_PASSWORD=unix11 -e MYSQL_DATABASE=my_database -p 3306:3306 mariadb:10.3.23

Run the application using:

SPRING_PROFILES_ACTIVE=local mvn spring-boot:run

The app will boot successfully.

Comment line 7 and uncomment line 8 in application-local.properties to add the mysql tracing interceptor:

# spring.datasource.url: jdbc:mysql://localhost:3306/my_database?user=root&password=unix11&useSSL=false
spring.datasource.url: jdbc:mysql://localhost:3306/my_database?user=root&password=unix11&useSSL=false&queryInterceptors=brave.mysql8.TracingQueryInterceptor&exceptionInterceptors=brave.mysql8.TracingExceptionInterceptor&zipkinServiceName=tac_db

Expected Behaviour

App should still boot, however in practice it is now stuck on startup after line 52 of TracingQueryInterceptor.

codefromthecrypt commented 4 years ago

It is unlikely that the solution to this will end up in this repo. as brave implements a mysql interceptor and isn't doing anything around connection management or configuration.

This sounds like a configuration lock. Did you try asking on https://gitter.im/spring-cloud/spring-cloud-sleuth maybe a thread dump for the hung process will help proceed.

odedia commented 4 years ago

Found the root cause - the startup is stuck when no default Sampler bean is defined. It occurs in the isSampled() method. Fixed by adding the following to a @Configuration class (for example):

    @Bean
    public Sampler defaultSampler() {
        return Sampler.ALWAYS_SAMPLE;
    }
codefromthecrypt commented 4 years ago

cool. did you report to sleuth? this might impact others.

On Thu, Jul 30, 2020, 6:43 PM odedia notifications@github.com wrote:

Found the root cause - the startup is stuck when no default Sampler bean is defined. It occurs in the isSampled() method. Fixed by adding the following to a @configuration https://github.com/configuration class (for example):

@Bean public Sampler defaultSampler() { return Sampler.ALWAYS_SAMPLE; }

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin-support/issues/34#issuecomment-666291038, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAPVV5TKFA2IBG2XH25UJTR6FFG3ANCNFSM4PJ4NXJQ .