sofastack / sofa-jraft

A production-grade java implementation of RAFT consensus algorithm.
https://www.sofastack.tech/projects/sofa-jraft/
Apache License 2.0
3.58k stars 1.15k forks source link

运行一段时间后集群不可用 #536

Closed bobbylv closed 3 years ago

bobbylv commented 3 years ago

Your question

集群运行一段时间后 出现状态错误,无法重新选举

日志

[demo--1/manager-master:18181-AppendEntriesThread0] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> is not in active state, currTerm=10. 2020-11-17T10:03:10.309+0800 MANAGER WARN 3065 - [Bolt-default-executor-4-thread-12] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> ignore InstallSnapshotRequest as it is not in active state STATE_ERROR. 2020-11-17T10:03:10.409+0800 MANAGER WARN 3065 - [demo--1/manager-master:18181-AppendEntriesThread0] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> is not in active state, currTerm=10. 2020-11-17T10:03:10.410+0800 MANAGER WARN 3065 - [Bolt-default-executor-4-thread-14] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> ignore InstallSnapshotRequest as it is not in active state STATE_ERROR. 2020-11-17T10:03:10.510+0800 MANAGER WARN 3065 - [demo--1/manager-master:18181-AppendEntriesThread0] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> is not in active state, currTerm=10. 2020-11-17T10:03:10.511+0800 MANAGER WARN 3065 - [Bolt-default-executor-4-thread-20] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> ignore InstallSnapshotRequest as it is not in active state STATE_ERROR. 2020-11-17T10:03:10.611+0800 MANAGER WARN 3065 - [demo--1/manager-master:18181-AppendEntriesThread0] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> is not in active state, currTerm=10. 2020-11-17T10:03:10.612+0800 MANAGER WARN 3065 - [Bolt-default-executor-4-thread-2] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> ignore InstallSnapshotRequest as it is not in active state STATE_ERROR. 2020-11-17T10:03:10.712+0800 MANAGER WARN 3065 - [demo--1/manager-master:18181-AppendEntriesThread0] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> is not in active state, currTerm=10. 2020-11-17T10:03:10.713+0800 MANAGER WARN 3065 - [Bolt-default-executor-4-thread-18] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> ignore InstallSnapshotRequest as it is not in active state STATE_ERROR. 2020-11-17T10:03:10.813+0800 MANAGER WARN 3065 - [demo--1/manager-master:18181-AppendEntriesThread0] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> is not in active state, currTerm=10. 2020-11-17T10:03:10.814+0800 MANAGER WARN 3065 - [Bolt-default-executor-4-thread-1] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> ignore InstallSnapshotRequest as it is not in active state STATE_ERROR. 2020-11-17T10:03:10.915+0800 MANAGER WARN 3065 - [demo--1/manager-master:18181-AppendEntriesThread0] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> is not in active state, currTerm=10. 2020-11-17T10:03:10.915+0800 MANAGER WARN 3065 - [Bolt-default-executor-4-thread-5] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> ignore InstallSnapshotRequest as it is not in active state STATE_ERROR. 2020-11-17T10:03:11.016+0800 MANAGER WARN 3065 - [demo--1/manager-master:18181-AppendEntriesThread0] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> is not in active state, currTerm=10. 2020-11-17T10:03:11.016+0800 MANAGER WARN 3065 - [Bolt-default-executor-4-thread-17] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> ignore InstallSnapshotRequest as it is not in active state STATE_ERROR. 2020-11-17T10:03:11.117+0800 MANAGER WARN 3065 - [demo--1/manager-master:18181-AppendEntriesThread0] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> is not in active state, currTerm=10. 2020-11-17T10:03:11.118+0800 MANAGER WARN 3065 - [Bolt-default-executor-4-thread-4] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> ignore InstallSnapshotRequest as it is not in active state STATE_ERROR. 2020-11-17T10:03:11.218+0800 MANAGER WARN 3065 - [demo--1/manager-master:18181-AppendEntriesThread0] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> is not in active state, currTerm=10. 2020-11-17T10:03:11.219+0800 MANAGER WARN 3065 - [Bolt-default-executor-4-thread-19] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> ignore InstallSnapshotRequest as it is not in active state STATE_ERROR. 2020-11-17T10:03:11.320+0800 MANAGER WARN 3065 - [demo--1/manager-master:18181-AppendEntriesThread0] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> is not in active state, currTerm=10. 2020-11-17T10:03:11.320+0800 MANAGER WARN 3065 - [Bolt-default-executor-4-thread-11] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> ignore InstallSnapshotRequest as it is not in active state STATE_ERROR. 2020-11-17T10:03:11.421+0800 MANAGER WARN 3065 - [demo--1/manager-master:18181-AppendEntriesThread0] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> is not in active state, currTerm=10. 2020-11-17T10:03:11.421+0800 MANAGER WARN 3065 - [Bolt-default-executor-4-thread-8] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> ignore InstallSnapshotRequest as it is not in active state STATE_ERROR. 2020-11-17T10:03:11.522+0800 MANAGER WARN 3065 - [demo--1/manager-master:18181-AppendEntriesThread0] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> is not in active state, currTerm=10. 2020-11-17T10:03:11.522+0800 MANAGER WARN 3065 - [Bolt-default-executor-4-thread-16] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> ignore InstallSnapshotRequest as it is not in active state STATE_ERROR. 2020-11-17T10:03:11.623+0800 MANAGER WARN 3065 - [demo--1/manager-master:18181-AppendEntriesThread0] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> is not in active state, currTerm=10. 2020-11-17T10:03:11.624+0800 MANAGER WARN 3065 - [Bolt-default-executor-4-thread-9] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> ignore InstallSnapshotRequest as it is not in active state STATE_ERROR. 2020-11-17T10:03:11.724+0800 MANAGER WARN 3065 - [demo--1/manager-master:18181-AppendEntriesThread0] com.alipay.sofa.jraft.core.NodeImpl : Node <demo--1/manager-master:18181> is not in active state, currTerm=10. 2020-11-17T10:03:11.725+0800 MANAGER WARN 3065 - [Bolt-default-executor-4-thread-12] com.alipay.sofa

配置

Describe the advice or solution you'd like

Environment

store存储节点的相关选项设置

storeEngineOptions: readIndexCoreThreads: 16 cliRpcCoreThreads: 16 kvRpcCoreThreads: 32 raftRpcCoreThreads: 32 useSharedRpcExecutor: true rocksDBOptions: dbPath: /rhea/rhea_db/

raft log存储目录

raftDataPath: /rhea/rhea_raft/ commonNodeOptions: cliRpcThreadPoolSize: 8 raftRpcThreadPoolSize: 12 serverAddress: ip: localhost

端口, 这个是必须配置的选项, 存储层提供rpc服务的监听端口

port: 18181

rpcOptions: callbackExecutorCorePoolSize: 16, callbackExecutorMaximumPoolSize: 32,

集群列表中所有节点的地址列表

onlyLeaderRead: true failoverRetries: 0 futureTimeoutMillis: 2000 useParallelKVExecutor: false initialServerList: localhost:18181,localhost:8182,localhost:8183

bobbylv commented 3 years ago

已解决

jemygraw commented 2 years ago

已解决

请问下怎么解决的,遇到类似问题。