pinpoint-apm / pinpoint

APM, (Application Performance Management) tool for large-scale distributed systems.
https://pinpoint-apm.gitbook.io/
Apache License 2.0
13.42k stars 3.76k forks source link

rpc agent run error #2275

Closed xingkaihu closed 7 years ago

xingkaihu commented 7 years ago

[22/11/16 11:02:40:040 CST] Pinpoint-PinpointClientHandler-Timer(11-0) WARN sender.RetryQueue: discard retry message. retryCount:3 [22/11/16 11:02:40:040 CST] Pinpoint-PinpointClientHandler-Timer(11-0) WARN sender.TcpDataSender: request fail. clazz:TApiMetaData(agentId:12345678, agentStartTime:1479783744733, apiId:-264, apiInfo:net.spy.memcached.MemcachedClient.asyncCAS(java.lang.String null, long null, int null, java.lang.Object null), line:139, type:0) Caused:timeout com.navercorp.pinpoint.rpc.PinpointSocketException: timeout at com.navercorp.pinpoint.rpc.DefaultFuture.fireTimeout(DefaultFuture.java:120) at com.navercorp.pinpoint.rpc.DefaultFuture.run(DefaultFuture.java:235) at org.jboss.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:561) at org.jboss.netty.util.HashedWheelTimer$Worker.notifyExpiredTimeouts(HashedWheelTimer.java:465) at org.jboss.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:404) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at java.lang.Thread.run(Thread.java:722) [22/11/16 11:02:40:040 CST] Pinpoint-PinpointClientHandler-Timer(11-0) WARN sender.RetryQueue: discard retry message. retryCount:3 [22/11/16 11:02:40:040 CST] Pinpoint-PinpointClientHandler-Timer(11-0) WARN sender.TcpDataSender: request fail. clazz:TApiMetaData(agentId:12345678, agentStartTime:1479783744733, apiId:-267, apiInfo:net.spy.memcached.MemcachedClient.prepend(long null, java.lang.String null, java.lang.Object null), line:139, type:0) Caused:timeout com.navercorp.pinpoint.rpc.PinpointSocketException: timeout at com.navercorp.pinpoint.rpc.DefaultFuture.fireTimeout(DefaultFuture.java:120) at com.navercorp.pinpoint.rpc.DefaultFuture.run(DefaultFuture.java:235) at org.jboss.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:561) at org.jboss.netty.util.HashedWheelTimer$Worker.notifyExpiredTimeouts(HashedWheelTimer.java:465) at org.jboss.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:404) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at java.lang.Thread.run(Thread.java:722) [22/11/16 11:02:40:040 CST] Pinpoint-PinpointClientHandler-Timer(11-0) WARN sender.RetryQueue: discard retry message. retryCount:3 [22/11/16 11:02:40:040 CST] Pinpoint-PinpointClientHandler-Timer(11-0) WARN sender.TcpDataSender: request fail. clazz:TApiMetaData(agentId:12345678, agentStartTime:1479783744733, apiId:-268, apiInfo:net.spy.memcached.MemcachedClient.append(long null, java.lang.String null, java.lang.Object null, net.spy.memcached.transcoders.Transcoder null), line:139, type:0) Caused:timeout com.navercorp.pinpoint.rpc.PinpointSocketException: timeout at com.navercorp.pinpoint.rpc.DefaultFuture.fireTimeout(DefaultFuture.java:120) at com.navercorp.pinpoint.rpc.DefaultFuture.run(DefaultFuture.java:235) at org.jboss.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:561) at org.jboss.netty.util.HashedWheelTimer$Worker.notifyExpiredTimeouts(HashedWheelTimer.java:465) at org.jboss.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:404) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at java.lang.Thread.run(Thread.java:722) [22/11/16 11:02:40:040 CST] Pinpoint-PinpointClientHandler-Timer(11-0) WARN sender.RetryQueue: discard retry message. retryCount:3 [22/11/16 11:02:41:041 CST] Pinpoint-PinpointClientHandler-Timer(11-0) WARN sender.TcpDataSender: request fail. clazz:TApiMetaData(agentId:12345678, agentStartTime:1479783744733, apiId:269, apiInfo:com.alibaba.dubbo.rpc.proxy.AbstractProxyInvoker.invoke(com.alibaba.dubbo.rpc.Invocation invocation), line:72, type:0) Caused:timeout com.navercorp.pinpoint.rpc.PinpointSocketException: timeout

Xylus commented 7 years ago

Agent seems to be unable to connect to the collector. Please check your ports.

xingkaihu commented 7 years ago

below config is agent's pinpoint.config. agent run with dubbo. hbase ,pinpoint-collector, pinpoint-web all run OK without error logs. but agent again occur above exception log.

#

Pinpoint agent configuration

#

###########################################################

Collector server

########################################################### profiler.collector.ip=192.168.77.153

placeHolder support "${key}"

profiler.collector.span.ip=${profiler.collector.ip} profiler.collector.span.port=29996

placeHolder support "${key}"

profiler.collector.stat.ip=${profiler.collector.ip} profiler.collector.stat.port=29995

placeHolder support "${key}"

profiler.collector.tcp.ip=${profiler.collector.ip} profiler.collector.tcp.port=29994

###########################################################

Profiler Global Configuration

########################################################### profiler.enable=true

profiler.jvm.collect.interval=1000

profiler.sampling.enable=true

Set sampling rate. If you set it to 10, 1 out of 10 transaction will be sampled.

profiler.sampling.rate=1

profiler.io.buffering.enable=true profiler.io.buffering.buffersize=20

profiler.spandatasender.write.queue.size=5120

profiler.spandatasender.socket.sendbuffersize=1048576

profiler.spandatasender.socket.timeout=3000

profiler.spandatasender.chunk.size=16384 profiler.spandatasender.socket.type=OIO

profiler.statdatasender.write.queue.size=5120

profiler.statdatasender.socket.sendbuffersize=1048576

profiler.statdatasender.socket.timeout=3000

profiler.statdatasender.chunk.size=16384 profiler.statdatasender.socket.type=OIO

profiler.agentInfo.send.retry.interval=300000

Allows TCP data command

profiler.tcpdatasender.command.accept.enable=true

Trace Agent active thread info

profiler.pinpoint.activethread=true

Call Stack

Set max depth, if -1 is unlimited and min is 2.

profiler.callstack.max.depth=64

weather or not to propagate exceptions occurred at interceptor

profiler.interceptor.exception.propagate=false

Allow bytecode framework (JAVASSIST or ASM)

profiler.instrument.engine=ASM

bytecode dump option

java bytecode debug option

bytecode.dump.enable=false

bytecode.dump.classlist=com.naver.user.UserService,com.pinpoint.debug.TestClass

bytecode.dump.classlist= bytecode.dump.bytecode=false bytecode.dump.verify=false bytecode.dump.asm=false

###########################################################

application type

########################################################### profiler.applicationservertype=DUBBO

profiler.applicationservertype=TOMCAT

profiler.applicationservertype=BLOC

###########################################################

user defined classes

########################################################### profiler.include= profiler.entrypoint=

###########################################################

TOMCAT

########################################################### profiler.tomcat.enable=true

Classes for detecting application server type. Comma separated list of fully qualified class names. Wildcard not supported.

profiler.tomcat.bootstrap.main=org.apache.catalina.startup.Bootstrap

Check pre-conditions when registering class file transformers mainly due to JBoss plugin transforming the same class.

Setting this to true currently adds transformers only if the application was launched via org.apache.catalina.startup.Bootstrap,

or SpringBoot's launchers.

Set this to false to bypass this check entirely (such as when launching standalone applications running embedded Tomcat).

profiler.tomcat.conditional.transform=false

Hide pinpoint headers.

profiler.tomcat.hidepinpointheader=true

URLs to exclude from tracing

profiler.tomcat.excludeurl=/aa/test.html, /bb/exclude.html profiler.tomcat.tracerequestparam=true

original IP address header

https://en.wikipedia.org/wiki/X-Forwarded-For

profiler.tomcat.realipheader=X-Forwarded-For

nginx real ip header

profiler.tomcat.realipheader=X-Real-IP

optional parameter, If the header value is ${profiler.tomcat.realipemptyvalue}, Ignore header value.

profiler.tomcat.realipemptyvalue=unknown

###########################################################

JETTY

########################################################### profiler.jetty.enable=true

Classes for detecting application server type. Comma separated list of fully qualified class names. Wildcard not supported.

profiler.jetty.bootstrap.main=org.eclipse.jetty.start.Main

URLs to exclude from tracing

profiler.jetty.excludeUrl=

###########################################################

DUBBO

########################################################### profiler.dubbo.enable=true

Classes for detecting application server type. Comma separated list of fully qualified class names. Wildcard not supported.

profiler.dubbo.bootstrap.main=com.alibaba.dubbo.container.Main

###########################################################

JBOSS

########################################################### profiler.jboss.enable=true

Classes for detecting application server type. Comma separated list of fully qualified class names. Wildcard not supported.

profiler.jboss.bootstrap.main=org.jboss.modules.Main

Check pre-conditions when registering class file transformers mainly due to Tomcat plugin transforming the same class.

Setting this to true currently adds transformers only if the application was launched via org.jboss.modules.Main.

Set this to false to bypass this check entirely.

profiler.jboss.conditional.transform=true

Hide pinpoint headers.

profiler.jboss.hidepinpointheader=true

URLs to exclude from tracing

profiler.jboss.excludeurl=

HTTP Request methods to exclude from tracing

profiler.jboss.excludemethod=

profiler.jboss.tracerequestparam=true

original IP address header

https://en.wikipedia.org/wiki/X-Forwarded-For

profiler.jboss.realipheader=X-Forwarded-For

nginx real ip header

profiler.jboss.realipheader=X-Real-IP

optional parameter, If the header value is ${profiler.jboss.realipemptyvalue}, Ignore header value.

profiler.jboss.realipemptyvalue=unknown

###########################################################

SPRING BOOT

########################################################### profiler.springboot.enable=true

Classes for detecting application server type. Comma separated list of fully qualified class names. Wildcard not supported.

profiler.springboot.bootstrap.main=org.springframework.boot.loader.JarLauncher, org.springframework.boot.loader.WarLauncher, org.springframework.boot.loader.PropertiesLauncher

###########################################################

JDBC

########################################################### profiler.jdbc=true profiler.jdbc.sqlcachesize=1024 profiler.jdbc.maxsqlbindvaluesize=1024

#

MYSQL

# profiler.jdbc.mysql=true profiler.jdbc.mysql.setautocommit=true profiler.jdbc.mysql.commit=true profiler.jdbc.mysql.rollback=true

#

MSSQL Jtds

# profiler.jdbc.jtds=true profiler.jdbc.jtds.setautocommit=true profiler.jdbc.jtds.commit=true profiler.jdbc.jtds.rollback=true

#

Oracle

# profiler.jdbc.oracle=true profiler.jdbc.oracle.setautocommit=true profiler.jdbc.oracle.commit=true profiler.jdbc.oracle.rollback=true

#

CUBRID

# profiler.jdbc.cubrid=true profiler.jdbc.cubrid.setautocommit=true profiler.jdbc.cubrid.commit=true profiler.jdbc.cubrid.rollback=true

#

DBCP

# profiler.jdbc.dbcp=true profiler.jdbc.dbcp.connectionclose=true

###########################################################

Apache HTTP Client 4.x

########################################################### profiler.apache.httpclient4=true profiler.apache.httpclient4.cookie=true

When cookies should be dumped. It could be ALWAYS or EXCEPTION.

profiler.apache.httpclient4.cookie.dumptype=ALWAYS profiler.apache.httpclient4.cookie.sampling.rate=1

Dump entities of POST or PUT request. limited to entities which is HttpEtity.isRepeatable() == true.

profiler.apache.httpclient4.entity=true

When entities should be dumped. ALWAYS or EXCEPTION.

profiler.apache.httpclient4.entity.dumptype=ALWAYS profiler.apache.httpclient4.entity.sampling.rate=1

Can profile status code value

profiler.apache.httpclient4.entity.statuscode=true

Not supported yet

profiler.apache.nio.httpclient4=true

###########################################################

JDK HTTPURLConnection

########################################################### profiler.jdk.httpurlconnection=true

###########################################################

Ning Async HTTP Client

########################################################### profiler.ning.asynchttpclient=true profiler.ning.asynchttpclient.cookie=true profiler.ning.asynchttpclient.cookie.dumptype=ALWAYS profiler.ning.asynchttpclient.cookie.dumpsize=1024 profiler.ning.asynchttpclient.cookie.sampling.rate=1 profiler.ning.asynchttpclient.entity=true profiler.ning.asynchttpclient.entity.dumptype=ALWAYS profiler.ning.asynchttpclient.entity.dumpsize=1024 profiler.ning.asynchttpclient.entity.sampling.rate=1 profiler.ning.asynchttpclient.param=true profiler.ning.asynchttpclient.param.dumptype=ALWAYS profiler.ning.asynchttpclient.param.dumpsize=1024 profiler.ning.asynchttpclient.param.sampling.rate=1

###########################################################

LINE+ baseframework

########################################################### profiler.line.game.netty.param.dumpsize=512 profiler.line.game.netty.entity.dumpsize=512

###########################################################

Arcus

########################################################### profiler.arcus=true profiler.arcus.keytrace=true

###########################################################

Memcached

########################################################### profiler.memcached=true profiler.memcached.keytrace=true

###########################################################

ibatis

########################################################### profiler.orm.ibatis=true

###########################################################

mybatis

########################################################### profiler.orm.mybatis=true

###########################################################

spring-beans

###########################################################

Profile spring-beans

profiler.spring.beans=true

filters

filter

filter OR filters

filter

value

value AND filter

value

token

token OR token

token

profiler.spring.beans.n.scope= [component-scan | post-processor] default is component-scan.

profiler.spring.beans.n.base-packages= [package name, ...]

profiler.spring.beans.n.name.pattern= [regex pattern, regex:regex pattern, antstyle:antstyle pattern, ...]

profiler.spring.beans.n.class.pattern= [regex pattern, regex:regex pattern, antstyle:antstyle pattern, ...]

profiler.spring.beans.n.annotation= [annotation name, ...]

#

Scope:

component-scan: <context:component-scan ... /> or @ComponentScan

post-processor: BeanPostProcessor - Slow!!!

#

ANT Style pattern rules:

? - matches on character

* - matches zero or more characters

** - matches zero or more 'directories' in a path

Examples

profiler.spring.beans.1.name.scope=component-scan

profiler.spring.beans.1.base-packages=com.foo, com.bar

profiler.spring.beans.1.name.pattern=.Foo, regex:.Bar, antstyle:*Controller

profiler.spring.beans.1.class.pattern=

profiler.spring.beans.1.annotation=org.springframework.stereotype.Controller,org.springframework.stereotype.Service,org.springframework.stereotype.Repository

#

profiler.spring.beans.2.name.scope=post-processor

profiler.spring.beans.2.base-packages=com.foo

profiler.spring.beans.2.name.pattern=

profiler.spring.beans.2.class.pattern=antstyle:com.foo.repository.Repository, antstyle:com.foo.Service.Main

profiler.spring.beans.2.annotation=

profiler.spring.beans.1.name.scope=component-scan profiler.spring.beans.1.base-packages= profiler.spring.beans.1.name.pattern= profiler.spring.beans.1.class.pattern= profiler.spring.beans.1.annotation=org.springframework.stereotype.Controller,org.springframework.stereotype.Service,org.springframework.stereotype.Repository

profiler.spring.beans.mark.error=false ###########################################################

log4j

########################################################### profiler.log4j.logging.transactioninfo=false

###########################################################

logback

########################################################### profiler.logback.logging.transactioninfo=false

###########################################################

gson

########################################################### profiler.json.gson=true

###########################################################

jackson

########################################################### profiler.json.jackson=true

###########################################################

json-lib

########################################################### profiler.json.jsonlib=true

###########################################################

cxf

########################################################### profiler.cxf.client=true