Xylus
commented
7 years ago Hi @prachitripathi Is your hbase installed on your local machine?
I ask because it looks like you've ran quickstart's init-hbase, which will create the tables on the hbase running on localhost (installed by quickstart).
But from the collector log above, it looks like your collector is trying to connect to 01hw828620.india.tcs.com (which might not have all the necessary tables - namely TraceV2 and AgentStatV2).
Description
I am setting up pinpoint from scratch and following the steps at - https://github.com/naver/pinpoint/blob/master/quickstart/README.md including ensuring the versions. Would upgrade once I get a hang of the tool.
But a few challenges that I face are as follows:
Environment
Environment Details:
Additional Info
Full Stack Trace - SEVERE: Exception sending context initialized event to listener instance of class org.springframework.web.context.ContextLoaderListener org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'spanChunkHandler': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private com.navercorp.pinpoint.collector.dao.TraceDao com.navercorp.pinpoint.collector.handler.SpanChunkHandler.traceDao; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'hbaseTraceDaoFactory': FactoryBean threw exception on object creation; nested exception is com.navercorp.pinpoint.common.hbase.HbaseSystemException: Failed after attempts=36, exceptions: Wed Jan 25 16:23:37 IST 2017, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=76792: row 'TraceV2,,' on table 'hbase:meta' at region=hbas e:meta,,1.1588230740, hostname=01hw828620.india.tcs.com,52501,1485340786126, seq Num=0 ; nested exception is org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions: Wed Jan 25 16:23:37 IST 2017, null, java.net.SocketTimeoutException: callTimeout =60000, callDuration=76792: row 'TraceV2,,' on table 'hbase:meta' at region=hbas e:meta,,1.1588230740, hostname=01hw828620.india.tcs.com,52501,1485340786126, seq Num=0
ostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java :334) at org.springframework.beans.factory.support.AbstractAutowireCapableBean Factory.populateBean(AbstractAutowireCapableBeanFactory.java:1208) at org.springframework.beans.factory.support.AbstractAutowireCapableBean Factory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537) at org.springframework.beans.factory.support.AbstractAutowireCapableBean Factory.createBean(AbstractAutowireCapableBeanFactory.java:476) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getOb ject(AbstractBeanFactory.java:303) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistr y.getSingleton(DefaultSingletonBeanRegistry.java:230) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBe an(AbstractBeanFactory.java:299) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean (AbstractBeanFactory.java:194) at org.springframework.beans.factory.support.DefaultListableBeanFactory. preInstantiateSingletons(DefaultListableBeanFactory.java:755) at org.springframework.context.support.AbstractApplicationContext.finish BeanFactoryInitialization(AbstractApplicationContext.java:762) at org.springframework.context.support.AbstractApplicationContext.refres h(AbstractApplicationContext.java:480) at org.springframework.web.context.ContextLoader.configureAndRefreshWebA pplicationContext(ContextLoader.java:434) at org.springframework.web.context.ContextLoader.initWebApplicationConte xt(ContextLoader.java:306) at org.springframework.web.context.ContextLoaderListener.contextInitiali zed(ContextLoaderListener.java:106) at org.apache.catalina.core.StandardContext.listenerStart(StandardContex t.java:4939) at org.apache.catalina.core.StandardContext.startInternal(StandardContex t.java:5434) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase. java:1559) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase. java:1549) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor. java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor .java:603) at java.lang.Thread.run(Thread.java:722) Caused by: org.springframework.beans.factory.BeanCreationException: Could not autowire field: private com.navercorp.pinpoint.collector.dao.TraceDao com.navercor p.pinpoint.collector.handler.SpanChunkHandler.traceDao; nested exception is org. springframework.beans.factory.BeanCreationException: Error creating bean with na me 'hbaseTraceDaoFactory': FactoryBean threw exception on object creation; neste d exception is com.navercorp.pinpoint.common.hbase.HbaseSystemException: Failed after attempts=36, exceptions: Wed Jan 25 16:23:37 IST 2017, null, java.net.SocketTimeoutException: callTimeout =60000, callDuration=76792: row 'TraceV2,,' on table 'hbase:meta' at region=hbas e:meta,,1.1588230740, hostname=01hw828620.india.tcs.com,52501,1485340786126, seq Num=0 ; nested exception is org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions: Wed Jan 25 16:23:37 IST 2017, null, java.net.SocketTimeoutException: callTimeout =60000, callDuration=76792: row 'TraceV2,,' on table 'hbase:meta' at region=hbas e:meta,,1.1588230740, hostname=01hw828620.india.tcs.com,52501,1485340786126, seq Num=0
ostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.j ava:561) at org.springframework.beans.factory.annotation.InjectionMetadata.inject (InjectionMetadata.java:88) at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanP ostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java :331) ... 23 more Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'hbaseTraceDaoFactory': FactoryBean threw exception on object creation; nested exception is com.navercorp.pinpoint.common.hbase.HbaseSystemException: Failed after attempts=36, exceptions: Wed Jan 25 16:23:37 IST 2017, null, java.net.SocketTimeoutException: callTimeout =60000, callDuration=76792: row 'TraceV2,,' on table 'hbase:meta' at region=hbas e:meta,,1.1588230740, hostname=01hw828620.india.tcs.com,52501,1485340786126, seq Num=0 ; nested exception is org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions: Wed Jan 25 16:23:37 IST 2017, null, java.net.SocketTimeoutException: callTimeout =60000, callDuration=76792: row 'TraceV2,,' on table 'hbase:meta' at region=hbas e:meta,,1.1588230740, hostname=01hw828620.india.tcs.com,52501,1485340786126, seq Num=0
doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175) at org.springframework.beans.factory.support.FactoryBeanRegistrySupport. getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103) at org.springframework.beans.factory.support.AbstractBeanFactory.getObje ctForBeanInstance(AbstractBeanFactory.java:1525) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBe an(AbstractBeanFactory.java:251) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean (AbstractBeanFactory.java:194) at org.springframework.beans.factory.support.DefaultListableBeanFactory. findAutowireCandidates(DefaultListableBeanFactory.java:1120) at org.springframework.beans.factory.support.DefaultListableBeanFactory. doResolveDependency(DefaultListableBeanFactory.java:1044) at org.springframework.beans.factory.support.DefaultListableBeanFactory. resolveDependency(DefaultListableBeanFactory.java:942) at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanP ostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.j ava:533) ... 25 more Caused by: com.navercorp.pinpoint.common.hbase.HbaseSystemException: Failed afte r attempts=36, exceptions: Wed Jan 25 16:23:37 IST 2017, null, java.net.SocketTimeoutException: callTimeout =60000, callDuration=76792: row 'TraceV2,,' on table 'hbase:meta' at region=hbas e:meta,,1.1588230740, hostname=01hw828620.india.tcs.com,52501,1485340786126, seq Num=0 ; nested exception is org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions: Wed Jan 25 16:23:37 IST 2017, null, java.net.SocketTimeoutException: callTimeout =60000, callDuration=76792: row 'TraceV2,,' on table 'hbase:meta' at region=hbas e:meta,,1.1588230740, hostname=01hw828620.india.tcs.com,52501,1485340786126, seq Num=0
aseAdminTemplate.java:61) at com.navercorp.pinpoint.collector.dao.hbase.HbaseTraceDaoFactory.getOb ject(HbaseTraceDaoFactory.java:54) at com.navercorp.pinpoint.collector.dao.hbase.HbaseTraceDaoFactory.getOb ject(HbaseTraceDaoFactory.java:18) at org.springframework.beans.factory.support.FactoryBeanRegistrySupport. doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168) ... 33 more Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed afte r attempts=36, exceptions: Wed Jan 25 16:23:37 IST 2017, null, java.net.SocketTimeoutException: callTimeout =60000, callDuration=76792: row 'TraceV2,,' on table 'hbase:meta' at region=hbas e:meta,,1.1588230740, hostname=01hw828620.india.tcs.com,52501,1485340786126, seq Num=0
wEnrichedException(RpcRetryingCallerWithReadReplicas.java:276) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(Scann erCallableWithReplicas.java:210) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(Scann erCallableWithReplicas.java:60) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(R pcRetryingCaller.java:210) at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java: 327) at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanne r.java:302) at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConst ruction(ClientScanner.java:167) at org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.jav
a:162)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
at org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.
java:602)
at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccess
or.java:366)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java
:406)
at com.navercorp.pinpoint.common.hbase.HBaseAdminTemplate.tableExists(HB
aseAdminTemplate.java:59)
... 36 more
Caused by: java.net.SocketTimeoutException: callTimeout=60000, callDuration=7679
2: row 'TraceV2,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hos
tname=01hw828620.india.tcs.com,52501,1485340786126, seqNum=0
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcR
etryingCaller.java:169)
at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$Queuein
gFuture.run(ResultBoundedCompletionService.java:65)
... 3 more
Caused by: java.net.ConnectException: Connection refused: no further information
) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout .java:206) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:529) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:493) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupConnection( RpcClientImpl.java:416) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(R pcClientImpl.java:722) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(Rpc ClientImpl.java:906) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteReque st(RpcClientImpl.java:873) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:124 1) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(Abst ractRpcClient.java:227) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImple mentation.callBlockingMethod(AbstractRpcClient.java:336) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService $BlockingStub.scan(ClientProtos.java:34094) at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCal lable.java:400) at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.j ava:204) at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.j ava:65) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(R pcRetryingCaller.java:210) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRP C.call(ScannerCallableWithReplicas.java:364) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRP C.call(ScannerCallableWithReplicas.java:338) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcR etryingCaller.java:136) ... 4 more
Jan 25, 2017 4:23:37 PM org.apache.catalina.core.StandardContext startInternal SEVERE: Error listenerStart Jan 25, 2017 4:23:37 PM org.apache.catalina.core.StandardContext startInternal SEVERE: Context [] startup failed due to previous errors Jan 25, 2017 4:23:37 PM org.apache.catalina.core.ApplicationContext log INFO: Closing Spring root WebApplicationContext Jan 25, 2017 4:23:38 PM org.apache.catalina.loader.WebappClassLoader clearRefere ncesThreads SEVERE: The web application [] appears to have started a thread named [Thread-3] but has failed to stop it. This is very likely to create a memory leak. Jan 25, 2017 4:23:38 PM org.apache.coyote.AbstractProtocol start INFO: Starting ProtocolHandler ["http-bio-28082"]