[Bug] 使用SessionPool连接集群，insertAlignedTablets接口报错：获取session超时

zhaozhenzhou commented 1 year ago

Search before asking

[X] I searched in the issues and found nothing similar.

Version

0.13.3

Describe the bug and provide the minimal reproduce step

SessionPool连接三节点IoTDB集群：

` @Bean

public SessionPool getSessionPool() {

    SessionPool sessionPool = new SessionPool.Builder()

            .nodeUrls(clusterNodes)

            .user(account.getUsername())

            .password(account.getPassword())

            .fetchSize(fetchSize)

            .maxSize(maxSize)

            .build();

    return sessionPool;

}  `

fetchSize=10000 maxSize=8

通过设备遥测信息上报，触发时序数据写入，调用方法：insertAlignedTablets
当QPS大约40的时候，前几百条插入正常，然后报错： timeout to get a connection from null:-1

What did you expect to see?

success

What did you see instead?

org.apache.iotdb.rpc.IoTDBConnectionException: timeout to get a connection from null:-1 at org.apache.iotdb.session.pool.SessionPool.getSession(SessionPool.java:363) at org.apache.iotdb.session.pool.SessionPool.insertAlignedTablets(SessionPool.java:681) at org.apache.iotdb.session.pool.SessionPool.insertAlignedTablets(SessionPool.java:646) at com.sgcc.evs.qlhd.iotdb.utils.IoTDBUtils.insertTablets(IoTDBUtils.java:125) at com.sgcc.evs.qlhd.device.biz.impl.DeviceTsDataServiceImpl.saveBatteryProps(DeviceTsDataServiceImpl.java:229) at com.sgcc.evs.qlhd.device.biz.impl.DevicePropServiceImpl.saveProperties(DevicePropServiceImpl.java:80) at com.sgcc.evs.qlhd.device.biz.impl.DevicePropServiceImpl$$FastClassBySpringCGLIB$$a6197783.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) at com.sgcc.evs.qlhd.device.biz.impl.DevicePropServiceImpl$$EnhancerBySpringCGLIB$$cdd63b8d.saveProperties(<generated>) at com.sgcc.evs.qlhd.device.biz.impl.SwapDeviceServiceImpl.handleDeviceProps(SwapDeviceServiceImpl.java:232) at com.sgcc.evs.qlhd.device.web.controller.swap.SwapDeviceController.device(SwapDeviceController.java:104) at com.sgcc.evs.qlhd.device.web.controller.swap.SwapDeviceController$$FastClassBySpringCGLIB$$8cef8b42.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.framework.adapter.AfterReturningAdviceInterceptor.invoke(AfterReturningAdviceInterceptor.java:55) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88) at com.sgcc.evs.qlhd.common.config.WebLogAspect.doAround(WebLogAspect.java:68) at sun.reflect.GeneratedMethodAccessor140.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692) at com.sgcc.evs.qlhd.device.web.controller.swap.SwapDeviceController$$EnhancerBySpringCGLIB$$26318fb2.device(<generated>) at sun.reflect.GeneratedMethodAccessor145.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerM at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandler at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapt at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909) at javax.servlet.http.HttpServlet.service(HttpServlet.java:665) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:750) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at com.sgcc.evs.qlhd.trace.filter.HttpTraceFilter.doFilter(HttpTraceFilter.java:48) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:764) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1707) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)

Anything else?

No response

Are you willing to submit a PR?

[ ] I'm willing to submit a PR!

github-actions[bot] commented 1 year ago

Hi, this is your first issue in IoTDB project. Thanks for your report. Welcome to join the community!

zhaozhenzhou commented 1 year ago

原因已找到，是因为我设置的存储组在第三层，如：root.battery.WS00，当我使用insertTablet接口写入数据时，观察iotdb服务端日志一直打印：set storage group to [root.battery] ，猜测与自动创建存储组机制有关，由于路径上已有存储组，在root.battery上创建存储组是不成功的，从而影响了session的释放，导致后续的写入获取session失败，不清楚在1.0有没有修复

jixuan1989 commented 1 year ago

原因已找到，是因为我设置的存储组在第三层，如：root.battery.WS00，当我使用insertTablet接口写入数据时，观察iotdb服务端日志一直打印：set storage group to [root.battery] ，猜测与自动创建存储组机制有关，由于路径上已有存储组，在root.battery上创建存储组是不成功的，从而影响了session的释放，导致后续的写入获取session失败，不清楚在1.0有没有修复

Pay attention to this issue. @HTHou

HTHou commented 1 year ago

Cannot reproduce in v0.13 and v1.0...

apache / iotdb