search

pinpoint-apm / pinpoint

APM, (Application Performance Management) tool for large-scale distributed systems.
https://pinpoint-apm.gitbook.io/
Apache License 2.0
13.38k stars 3.75k forks source link

unable to start pinot supported pinpoint collector 3.0.0 #11491

Open snmsmanager opened 1 week ago

snmsmanager commented 1 week ago

hello,

I’m going to deploy PINPONT 3.0.0 with external PINOT, KAFKA and HBASE cluster. All clusters are running and configured properly as per the documentation. I’ve received REDISH connectivity error while starting collector by using custom application file as below. java -jar -Dspring.config.additional-location=collector-starter-application.yml pinpoint-collector-starter-3.0.0-exec.jar

pinpoint-collector.log pinpoint-collector.txt collector-starter-application.txt collector-starter-application.yml as below pinpoint: zookeeper: address: localhost metric: kafka: bootstrap: servers: 192.168.62.112:9092 spring: pinot-datasource: pinot: jdbc-url: jdbc:pinot://192.168.62.113:9001 username: password:

and return error as below

**09-11 15:51:24.024 [ main] ERROR o.s.b.SpringApplication :851 -- Application run failed org.springframework.context.ApplicationContextException: Failed to start bean 'redisMessageListenerContainer' at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:291) ~[spring-context-6.1.5.jar!/:6.1.5] at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:471) ~[spring-context-6.1.5.jar!/:6.1.5] at java.lang.Iterable.forEach(Iterable.java:75) ~[?:?] at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:260) ~[spring-context-6.1.5.jar!/:6.1.5] at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:205) ~[spring-context-6.1.5.jar!/:6.1.5] at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:981) ~[spring-context-6.1.5.jar!/:6.1.5] at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:627) ~[spring-context-6.1.5.jar!/:6.1.5] at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146) ~[spring-boot-3.2.4.jar!/:3.2.4] at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:754) [spring-boot-3.2.4.jar!/:3.2.4] at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:456) [spring-boot-3.2.4.jar!/:3.2.4] at org.springframework.boot.SpringApplication.run(SpringApplication.java:334) [spring-boot-3.2.4.jar!/:3.2.4] at com.navercorp.pinpoint.collector.starter.multi.application.PinpointCollectorStarter.main(PinpointCollectorStarter.java:77) [classes!/:3.0.0] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?] at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) [pinpoint-collector-starter-3.0.0-exec.jar:3.0.0] at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) [pinpoint-collector-starter-3.0.0-exec.jar:3.0.0] at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) [pinpoint-collector-starter-3.0.0-exec.jar:3.0.0] at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65) [pinpoint-collector-starter-3.0.0-exec.jar:3.0.0] Caused by: org.springframework.data.redis.listener.adapter.RedisListenerExecutionFailedException: org.springframework.data.redis.RedisConnectionFailureException: Unable to connect to Redis at org.springframework.data.redis.listener.RedisMessageListenerContainer.lazyListen(RedisMessageListenerContainer.java:382) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.listener.RedisMessageListenerContainer.start(RedisMessageListenerContainer.java:360) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:288) ~[spring-context-6.1.5.jar!/:6.1.5] ... 19 more Caused by: org.springframework.data.redis.RedisConnectionFailureException: Unable to connect to Redis at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$ExceptionTranslatingConnectionProvider.translateException(LettuceConnectionFactory.java:1795) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$ExceptionTranslatingConnectionProvider.getConnection(LettuceConnectionFactory.java:1726) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$SharedConnection.getNativeConnection(LettuceConnectionFactory.java:1528) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$SharedConnection.lambda$getConnection$0(LettuceConnectionFactory.java:1508) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.doInLock(LettuceConnectionFactory.java:1469) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$SharedConnection.getConnection(LettuceConnectionFactory.java:1505) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getSharedConnection(LettuceConnectionFactory.java:1191) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getConnection(LettuceConnectionFactory.java:997) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.listener.RedisMessageListenerContainer$Subscriber.lambda$initialize$0(RedisMessageListenerContainer.java:1239) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.listener.RedisMessageListenerContainer$Subscriber.doInLock(RedisMessageListenerContainer.java:1453) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.listener.RedisMessageListenerContainer$Subscriber.initialize(RedisMessageListenerContainer.java:1233) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.listener.RedisMessageListenerContainer.doSubscribe(RedisMessageListenerContainer.java:427) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.listener.RedisMessageListenerContainer.lazyListen(RedisMessageListenerContainer.java:403) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.listener.RedisMessageListenerContainer.lazyListen(RedisMessageListenerContainer.java:373) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.listener.RedisMessageListenerContainer.start(RedisMessageListenerContainer.java:360) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:288) ~[spring-context-6.1.5.jar!/:6.1.5] ... 19 more Caused by: io.lettuce.core.RedisConnectionException: Unable to connect to localhost/:6379 at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:78) ~[lettuce-core-6.3.2.RELEASE.jar!/:6.3.2.RELEASE/8941aea] at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:56) ~[lettuce-core-6.3.2.RELEASE.jar!/:6.3.2.RELEASE/8941aea] at io.lettuce.core.AbstractRedisClient.getConnection(AbstractRedisClient.java:350) ~[lettuce-core-6.3.2.RELEASE.jar!/:6.3.2.RELEASE/8941aea] at io.lettuce.core.RedisClient.connect(RedisClient.java:215) ~[lettuce-core-6.3.2.RELEASE.jar!/:6.3.2.RELEASE/8941aea] at org.springframework.data.redis.connection.lettuce.StandaloneConnectionProvider.lambda$getConnection$1(StandaloneConnectionProvider.java:112) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at java.util.Optional.orElseGet(Optional.java:364) ~[?:?] at org.springframework.data.redis.connection.lettuce.StandaloneConnectionProvider.getConnection(StandaloneConnectionProvider.java:112) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$ExceptionTranslatingConnectionProvider.getConnection(LettuceConnectionFactory.java:1724) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$SharedConnection.getNativeConnection(LettuceConnectionFactory.java:1528) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$SharedConnection.lambda$getConnection$0(LettuceConnectionFactory.java:1508) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.doInLock(LettuceConnectionFactory.java:1469) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$SharedConnection.getConnection(LettuceConnectionFactory.java:1505) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getSharedConnection(LettuceConnectionFactory.java:1191) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getConnection(LettuceConnectionFactory.java:997) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.listener.RedisMessageListenerContainer$Subscriber.lambda$initialize$0(RedisMessageListenerContainer.java:1239) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.listener.RedisMessageListenerContainer$Subscriber.doInLock(RedisMessageListenerContainer.java:1453) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.listener.RedisMessageListenerContainer$Subscriber.initialize(RedisMessageListenerContainer.java:1233) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.listener.RedisMessageListenerContainer.doSubscribe(RedisMessageListenerContainer.java:427) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.listener.RedisMessageListenerContainer.lazyListen(RedisMessageListenerContainer.java:403) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.listener.RedisMessageListenerContainer.lazyListen(RedisMessageListenerContainer.java:373) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.data.redis.listener.RedisMessageListenerContainer.start(RedisMessageListenerContainer.java:360) ~[spring-data-redis-3.2.4.jar!/:3.2.4] at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:288) ~[spring-context-6.1.5.jar!/:6.1.5] ... 19 more Caused by: io.lettuce.core.RedisCommandExecutionException: ERR wrong number of arguments for 'auth' command at io.lettuce.core.internal.ExceptionFactory.createExecutionException(ExceptionFactory.java:147) ~[lettuce-core-6.3.2.RELEASE.jar!/:6.3.2.RELEASE/8941aea] at io.lettuce.core.internal.ExceptionFactory.createExecutionException(ExceptionFactory.java:116) ~[lettuce-core-6.3.2.RELEASE.jar!/:6.3.2.RELEASE/8941aea] at io.lettuce.core.protocol.AsyncCommand.completeResult(AsyncCommand.java:120) ~[lettuce-core-6.3.2.RELEASE.jar!/:6.3.2.RELEASE/8941aea] at io.lettuce.core.protocol.AsyncCommand.complete(AsyncCommand.java:111) ~[lettuce-core-6.3.2.RELEASE.jar!/:6.3.2.RELEASE/8941aea] at io.lettuce.core.protocol.CommandHandler.complete(CommandHandler.java:745) ~[lettuce-core-6.3.2.RELEASE.jar!/:6.3.2.RELEASE/8941aea] at io.lettuce.core.protocol.CommandHandler.decode(CommandHandler.java:680) ~[lettuce-core-6.3.2.RELEASE.jar!/:6.3.2.RELEASE/8941aea] at io.lettuce.core.protocol.CommandHandler.channelRead(CommandHandler.java:597) ~[lettuce-core-6.3.2.RELEASE.jar!/:6.3.2.RELEASE/8941aea] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[netty-transport-4.1.100.Final.jar!/:4.1.100.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.100.Final.jar!/:4.1.100.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.100.Final.jar!/:4.1.100.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.100.Final.jar!/:4.1.100.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[netty-transport-4.1.100.Final.jar!/:4.1.100.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.100.Final.jar!/:4.1.100.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.100.Final.jar!/:4.1.100.Final] at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) ~[netty-transport-classes-epoll-4.1.100.Final.jar!/:4.1.100.Final] at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:509) ~[netty-transport-classes-epoll-4.1.100.Final.jar!/:4.1.100.Final] at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[netty-transport-classes-epoll-4.1.100.Final.jar!/:4.1.100.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.100.Final.jar!/:4.1.100.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.100.Final.jar!/:4.1.100.Final] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.100.Final.jar!/:4.1.100.Final] at java.lang.Thread.run(Thread.java:842) ~[?:?]**

snmsmanager commented 6 days ago

hello,

I've changed the REDIS version from 3.2 to 7.2.0 and now both collector and web are up and running and profiling application successfully. But not able to get Inspector or URL Statistics and returns Failed to connect to url : jdbc:pinot://192.168.62.113:9001. But my PINOT controllers is listening over 9001 and 9002.

image

image

minwoo-jung commented 5 days ago

Overall, there seem to be a number of issues happening, and you need to address them one by one.

case 1

pinpoint-collector.txt If you've configured redis and you're getting redis errors, it's probably worth trying to figure out why redis isn't communicating with you. Is the IP address correct, is there no network firewall, etc...

Caused by: io.lettuce.core.RedisConnectionException: Unable to connect to localhost/<unresolved>:6379

case2

failed to connect to url : jdbc:pinot:192.168.62.113:9001

Assuming pinot is working properly, it sounds like your network connection is not working for some reason. Recommend testing network connection using telnet from your web server.

In summary

You seem to have the right setup. But the components are not communicating to the desired targets. Check your network settings to make sure they're okay.