qunarcorp / bistoury

Bistoury是去哪儿网的java应用生产问题诊断工具,提供了一站式的问题诊断方案
GNU General Public License v3.0
4.01k stars 825 forks source link

not find proxy for agent #118

Open zengxiaoliang opened 3 years ago

zengxiaoliang commented 3 years ago

启动成功后,使用主机信息-基础信息,发现无法正常获取主机信息;

启动进程信息: 10680 qunar.tc.bistoury.ui.container.Bootstrap 10648 qunar.tc.bistoury.indpendent.agent.Main 10537 qunar.tc.bistoury.proxy.container.Bootstrap

异常日志如下:

[15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:46] - OS Name: Linux [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:47] - OS Version: 3.10.0-957.1.3.el7.x86_64 [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:48] - Architecture: amd64 [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:49] - Java Home: /usr/java/jdk1.8.0_60/jre [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:50] - JVM Version: 1.8.0_60-b27 [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:51] - JVM Vendor: Oracle Corporation [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:52] - CATALINA_BASE: null [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:53] - CATALINA_HOME: null [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:55] - Command line argument: -Dbistoury.store.path=/home/test/bis/bistoury-2.0.7/bistoury-agent/bin/../store [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:55] - Command line argument: -Dbistoury.proxy.host=127.0.0.1:9099 [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:55] - Command line argument: -Dbistoury.local.host=10.248.224.72 [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:55] - Command line argument: -Dbistoury.user.pid=3203 [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:55] - Command line argument: -Dbistoury.app.lib.class=org.springframework.web.servlet.DispatcherServlet [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:55] - Command line argument: -Dbistoury.log.dir=/home/dmadmin/bis/bistoury-2.0.7/bistoury-agent/bin/../logs [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:55] - Command line argument: -Xmx80m [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:55] - Command line argument: -Xmn50m [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:55] - Command line argument: -XX:+UseParallelGC [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:55] - Command line argument: -XX:+UseParallelOldGC [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:55] - Command line argument: -XX:+UseCodeCacheFlushing [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:55] - Command line argument: -Xloggc:/home/test/bis/bistoury-2.0.7/bistoury-agent/bin/../logs/bistoury-gc-1612338940.log [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:55] - Command line argument: -XX:+PrintGC [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:55] - Command line argument: -XX:+PrintGCDetails [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:55] - Command line argument: -XX:+HeapDumpOnOutOfMemoryError [15:55:40 [main] INFO q.tc.bistoury.indpendent.agent.Main:55] - Command line argument: -XX:HeapDumpPath=/home/test/bis/bistoury-2.0.7/bistoury-agent/bin/../logs

[15:55:46 [cpu-jstack-task-thread-1] INFO q.t.b.agent.common.pid.PidUtils:75] - get pid by PidBySystemPropertyHandler success, pid is 3203 [15:55:46 [cpu-jstack-task-thread-1] INFO q.t.b.a.task.cpujstack.TaskRunner:101] - start cpu jstack task, pid 3203, timestamp 202102031555 [15:56:46 [cpu-jstack-task-thread-1] INFO q.t.b.agent.common.pid.PidUtils:75] - get pid by PidBySystemPropertyHandler success, pid is 3203 [15:56:46 [cpu-jstack-task-thread-1] INFO q.t.b.a.task.cpujstack.TaskRunner:101] - start cpu jstack task, pid 3203, timestamp 202102031556 [15:56:46 [cpu-jstack-task-thread-1] ERRORq.t.b.a.t.c.JStackPidExecutor:46] - run JStackPidExecutor error pid:3203 java.io.IOException: Connection refused at sun.tools.attach.LinuxVirtualMachine.connect(Native Method) ~[tools.jar:na] at sun.tools.attach.LinuxVirtualMachine.(LinuxVirtualMachine.java:124) ~[tools.jar:na] at sun.tools.attach.LinuxAttachProvider.attachVirtualMachine(LinuxAttachProvider.java:63) ~[tools.jar:na] at com.sun.tools.attach.VirtualMachine.attach(VirtualMachine.java:208) ~[tools.jar:na] at qunar.tc.bistoury.agent.task.cpujstack.JStackPidExecutor.execute(JStackPidExecutor.java:42) ~[bistoury-agent-task-2.0.7.jar:na] at qunar.tc.bistoury.agent.task.cpujstack.TaskRunner.doRun(TaskRunner.java:113) [bistoury-agent-task-2.0.7.jar:na] at qunar.tc.bistoury.agent.task.cpujstack.TaskRunner.run(TaskRunner.java:86) [bistoury-agent-task-2.0.7.jar:na] at com.google.common.util.concurrent.MoreExecutors$ScheduledListeningDecorator$NeverSuccessfulListenableFutureTask.run(MoreExecutors.java:623) [guava-20.0.jar:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_60] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_60] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_60] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_60] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_60] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_60] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60] [15:56:46 [cpu-jstack-task-thread-1] ERRORq.t.b.a.task.cpujstack.TaskRunner:88] - cpu jstack tast run error java.lang.IllegalStateException: can not open pid directory : /proc/3203 at qunar.tc.bistoury.agent.task.proc.StatParser.parseProcessInfo(StatParser.java:70) ~[bistoury-agent-task-2.0.7.jar:na] at qunar.tc.bistoury.agent.task.proc.ProcessStateCalculator.getCurrentFullState(ProcessStateCalculator.java:80) ~[bistoury-agent-task-2.0.7.jar:na] at qunar.tc.bistoury.agent.task.proc.ProcessStateCalculator.threadCpuMinuteUsage(ProcessStateCalculator.java:48) ~[bistoury-agent-task-2.0.7.jar:na] at qunar.tc.bistoury.agent.task.cpujstack.TaskRunner.doRun(TaskRunner.java:115) ~[bistoury-agent-task-2.0.7.jar:na] at qunar.tc.bistoury.agent.task.cpujstack.TaskRunner.run(TaskRunner.java:86) ~[bistoury-agent-task-2.0.7.jar:na] at com.google.common.util.concurrent.MoreExecutors$ScheduledListeningDecorator$NeverSuccessfulListenableFutureTask.run(MoreExecutors.java:623) [guava-20.0.jar:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_60] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_60]

xleiy commented 3 years ago

你这个3203 进程还在吗,是java进程吗

zengxiaoliang commented 3 years ago

你这个3203 进程还在吗,是java进程吗

没有3203进程,刚刚停掉了,是我之前的Java进程

xleiy commented 3 years ago

换个进程还能复现吗

zengxiaoliang commented 3 years ago

换个进程还能复现吗

刚刚重启了,还是出现 not find proxy for agent;
bistoury-2.0.7/bistoury-agent/logs 中未出现异常日志

xleiy commented 3 years ago

https://github.com/qunarcorp/bistoury/blob/master/docs/cn/FAQ.md#not-find-proxy-for-agent

zengxiaoliang commented 3 years ago

https://github.com/qunarcorp/bistoury/blob/master/docs/cn/FAQ.md#not-find-proxy-for-agent

启动进程: 16357 qunar.tc.bistoury.indpendent.agent.Main 16399 qunar.tc.bistoury.ui.container.Bootstrap 16255 qunar.tc.bistoury.proxy.container.Bootstrap

检查了端口,配置均正常; 现各日志文件中也未出现异常信息;但仍然是找不到agent

[13:51:07 [bistoury-agent-netty-thread-1] INFO q.t.b.commands.MetaRefreshProcessor:55] - meta refresh data receive, {heapJMapHistoOn=false, agent.push.interval.min=1, app.config.exclusion.file.equal=web.xml, agent.refresh.interval.min=10, download.kb.per.second=10240, profiler.compact.package.prefix=java.;javax.;sun.;org.springframework.;org.jboss.;org.apache.;com.sun.;org.mybatis.;com.mysql.;io.netty.;com.google.;ch.qos.;org.slf4j.;io.termd.core.;com.taobao.arthas.;com.taobao.middleware., tomcat.command=/home/java/default/bin/java, port=8080, app.config.exclusion.file.suffix=class,vm,css,js,vue,ts,jsp,sql,jar, cpuJStackOn=true, debug.json.limit.kb=10240, tomcat.user=tomcat} [13:51:12 [cpu-jstack-task-thread-1] INFO q.t.b.agent.common.pid.PidUtils:75] - get pid by PidBySystemPropertyHandler success, pid is 8607 [13:51:12 [cpu-jstack-task-thread-1] INFO q.t.b.a.task.cpujstack.TaskRunner:101] - start cpu jstack task, pid 8607, timestamp 202102051351 [13:52:12 [cpu-jstack-task-thread-1] INFO q.t.b.agent.common.pid.PidUtils:75] - get pid by PidBySystemPropertyHandler success, pid is 8607 [13:52:12 [cpu-jstack-task-thread-1] INFO q.t.b.a.task.cpujstack.TaskRunner:101] - start cpu jstack task, pid 8607, timestamp 202102051352 [13:53:12 [cpu-jstack-task-thread-1] INFO q.t.b.agent.common.pid.PidUtils:75] - get pid by PidBySystemPropertyHandler success, pid is 8607 [13:53:12 [cpu-jstack-task-thread-1] INFO q.t.b.a.task.cpujstack.TaskRunner:101] - start cpu jstack task, pid 8607, timestamp 202102051353 [13:54:12 [cpu-jstack-task-thread-1] INFO q.t.b.agent.common.pid.PidUtils:75] - get pid by PidBySystemPropertyHandler success, pid is 8607 [13:54:12 [cpu-jstack-task-thread-1] INFO q.t.b.a.task.cpujstack.TaskRunner:101] - start cpu jstack task, pid 8607, timestamp 202102051354 [13:55:12 [cpu-jstack-task-thread-1] INFO q.t.b.agent.common.pid.PidUtils:75] - get pid by PidBySystemPropertyHandler success, pid is 8607 [13:55:12 [cpu-jstack-task-thread-1] INFO q.t.b.a.task.cpujstack.TaskRunner:101] - start cpu jstack task, pid 8607, timestamp 202102051355 [13:56:12 [cpu-jstack-task-thread-1] INFO q.t.b.agent.common.pid.PidUtils:75] - get pid by PidBySystemPropertyHandler success, pid is 8607 [13:56:12 [cpu-jstack-task-thread-1] INFO q.t.b.a.task.cpujstack.TaskRunner:101] - start cpu jstack task, pid 8607, timestamp 202102051356 [13:57:12 [cpu-jstack-task-thread-1] INFO q.t.b.agent.common.pid.PidUtils:75] - get pid by PidBySystemPropertyHandler success, pid is 8607 [13:57:12 [cpu-jstack-task-thread-1] INFO q.t.b.a.task.cpujstack.TaskRunner:101] - start cpu jstack task, pid 8607, timestamp 202102051357