search

sofastack / sofa-jraft

A production-grade java implementation of RAFT consensus algorithm.
https://www.sofastack.tech/projects/sofa-jraft/
Apache License 2.0
3.52k stars 1.12k forks source link

通过pd操作addReplica，发生了错误。提示新的节点Connection is null when do check!。同时操作addOrUpdateRegion发现没反应 #1077

Closed zxuanhong closed 4 months ago

zxuanhong commented 4 months ago

Your question

一、pd添加副本发生了错误

  1. 通过pd操作添加分区副本，但是发生了错误
  2. 调用代码如下 
    RemotePlacementDriverClient placementDriverClient = (RemotePlacementDriverClient) node.getRheaKVStore().getPlacementDriverClient();
placementDriverClient.addReplica(0, new Peer(-1, -1, new Endpoint("127.0.0.1", 8183)), true);
  3. 错误信息 
    
com.alipay.remoting.exception.RemotingException: Connection is null when do check!
at com.alipay.remoting.DefaultConnectionManager.check(DefaultConnectionManager.java:354) ~[bolt-1.6.7.jar:na]
at com.alipay.remoting.rpc.RpcClient.checkConnection(RpcClient.java:447) ~[bolt-1.6.7.jar:na]
at com.alipay.remoting.rpc.RpcClient.checkConnection(RpcClient.java:438) ~[bolt-1.6.7.jar:na]
at com.alipay.remoting.rpc.RpcClient.checkConnection(RpcClient.java:433) ~[bolt-1.6.7.jar:na]
at com.alipay.sofa.jraft.rpc.impl.BoltRpcClient.checkConnection(BoltRpcClient.java:73) ~[main/:na]
at com.alipay.sofa.jraft.rpc.impl.AbstractClientService.isConnected(AbstractClientService.java:81) ~[main/:na]
at com.alipay.sofa.jraft.rpc.impl.AbstractClientService.connect(AbstractClientService.java:142) ~[main/:na]
at com.alipay.sofa.jraft.core.CliServiceImpl.getLeader(CliServiceImpl.java:428) ~[main/:na]
at com.alipay.sofa.jraft.core.CliServiceImpl.checkLeaderAndConnect(CliServiceImpl.java:94) ~[main/:na]
at com.alipay.sofa.jraft.core.CliServiceImpl.addPeer(CliServiceImpl.java:114) ~[main/:na]
at com.alipay.sofa.jraft.rhea.client.pd.AbstractPlacementDriverClient.addReplica(AbstractPlacementDriverClient.java:209) ~[main/:na]
at com.alipay.sofa.boot.controller.TestApiController.selectNodeInfo(TestApiController.java:38) ~[main/:na]
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:580) ~[na:na]
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:261) ~[spring-web-6.1.3.jar:6.1.3]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:189) ~[spring-web-6.1.3.jar:6.1.3]
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118) ~[spring-webmvc-6.1.3.jar:6.1.3]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:917) ~[spring-webmvc-6.1.3.jar:6.1.3]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:829) ~[spring-webmvc-6.1.3.jar:6.1.3]
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-6.1.3.jar:6.1.3]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1089) ~[spring-webmvc-6.1.3.jar:6.1.3]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:979) ~[spring-webmvc-6.1.3.jar:6.1.3]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014) ~[spring-webmvc-6.1.3.jar:6.1.3]
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:903) ~[spring-webmvc-6.1.3.jar:6.1.3]
at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:527) ~[jakarta.servlet-api-6.0.0.jar:6.0.0]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885) ~[spring-webmvc-6.1.3.jar:6.1.3]
at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:614) ~[jakarta.servlet-api-6.0.0.jar:6.0.0]
at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-6.1.3.jar:6.1.3]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.3.jar:6.1.3]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:67) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-6.1.3.jar:6.1.3]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.3.jar:6.1.3]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:67) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-6.1.3.jar:6.1.3]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.3.jar:6.1.3]
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:67) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:117) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) ~[undertow-core-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) ~[undertow-core-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) ~[undertow-core-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) ~[undertow-core-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) ~[undertow-core-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) ~[undertow-core-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:276) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:132) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:256) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:101) ~[undertow-servlet-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:393) ~[undertow-core-2.3.10.Final.jar:2.3.10.Final]
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:859) ~[undertow-core-2.3.10.Final.jar:2.3.10.Final]
at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18) ~[jboss-threads-3.5.0.Final.jar:3.5.0.Final]
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513) ~[jboss-threads-3.5.0.Final.jar:3.5.0.Final]
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1512) ~[jboss-threads-3.5.0.Final.jar:3.5.0.Final]
at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282) ~[xnio-api-3.8.8.Final.jar:3.8.8.Final]
at java.base/java.lang.Thread.run(Thread.java:1583) ~[na:na]

2024-03-05 23:35:05 INFO 72124 --- [flow-demo] [rker-1-thread-3] com.alipay.sofa.common.log : Connection channel registered: UNKNOWN-ADDR 2024-03-05 23:35:05 INFO 72124 --- [flow-demo] [rker-1-thread-3] com.alipay.sofa.common.log : Try connect to 127.0.0.1:8181 2024-03-05 23:35:05 INFO 72124 --- [flow-demo] [rker-1-thread-3] com.alipay.sofa.common.log : Connection channel active: 127.0.0.1:8181 2024-03-05 23:35:05 INFO 72124 --- [flow-demo] [rker-1-thread-3] com.alipay.sofa.common.log : trigger user event, local[127.0.0.1:49981], remote[127.0.0.1:8181], event: CONNECT 2024-03-05 23:35:05 INFO 72124 --- [flow-demo] [ker-10-thread-6] com.alipay.sofa.common.log : trigger user event, local[127.0.0.1:8181], remote[127.0.0.1:49981], event: CONNECT 2024-03-05 23:35:05 INFO 72124 --- [flow-demo] [ker-10-thread-6] com.alipay.sofa.common.log : Connection channel registered: 127.0.0.1:49981 2024-03-05 23:35:05 INFO 72124 --- [flow-demo] [ker-10-thread-6] com.alipay.sofa.common.log : Connection channel active: 127.0.0.1:49981 2024-03-05 23:35:05 INFO 72124 --- [flow-demo] [ker-10-thread-6] c.a.s.j.r.u.c.NamedThreadFactory : Creates new Thread[#423,rheakv-cli-rpc-executor #0,5,main]. 2024-03-05 23:35:05 INFO 72124 --- [flow-demo] [ker-10-thread-6] c.a.s.j.r.u.c.NamedThreadFactory : Creates new Thread[#424,rheakv-cli-rpc-executor #1,5,main]. 2024-03-05 23:35:05 INFO 72124 --- [flow-demo] [rpc-executor #1] c.a.s.j.r.i.cli.BaseCliRequestProcessor : Receive AddPeerRequest to <pd_test-0/127.0.0.1:8181> from 127.0.0.1:49981, adding 127.0.0.1:8183 2024-03-05 23:35:05 INFO 72124 --- [flow-demo] [rpc-executor #1] com.alipay.sofa.jraft.core.NodeImpl : Adding learners: []. 2024-03-05 23:35:05 INFO 72124 --- [flow-demo] [rpc-executor #1] com.alipay.sofa.jraft.core.NodeImpl : Adding peers: [127.0.0.1:8183]. 2024-03-05 23:35:05 WARN 72124 --- [flow-demo] [rpc-executor #1] com.alipay.sofa.common.log : check failed. address: 127.0.0.1:8183, connection: null


4. 读取信息
<img width="736" alt="image" src="https://github.com/sofastack/sofa-jraft/assets/62740686/284f40e3-0e25-479f-bbf9-33dfb043d82f">

5. region数据目录在3节点也是没有创建起来

#### 二、通过pd获取RegionRouteTable然后addOrUpdateRegion一个分区，发现不生效。不知道需要怎么操作。还麻烦解惑下。

### Environment

- SOFAJRaft version: 1.3.14
- JVM version (e.g. `java -version`): 1.8
zxuanhong commented 4 months ago

因为重复关闭