v3.2.7 change_member增加副本无效

[realdalabengba]

v3.2.7 调用change_member接口增加副本后，接口显示调用成功。但新副本数据并没有加载，doc_num远远小于对应分片的doc_num，index_status为0。

[realdalabengba]

这应该是个bug，接口功能不正常

[kuailelijuan]

请使用github上最新的代码再试一下 change_member 接口。

[realdalabengba]

用了最新版后，文档逐渐趋于一致，但速度与通过raft逐条回放的速度并没有什么差别。3000多万数据，10个小时了还没恢复完成。下面是gamma log，可以看到10s插入10000条，1000条/s。

7084 INFO 2021-04-28 22:24:47,315 table.cc:54 Load doc_num [0] truncate to [0] 7085 INFO 2021-04-28 22:24:47,315 table.cc:82 Table load successed! doc num [0] 7086 INFO 2021-04-28 22:24:47,315 vector_manager.cc:706 vector [vector] load success! 7087 INFO 2021-04-28 22:24:47,315 gamma_engine.cc:1230 load engine success! max docid=0, load directory=, clean directorys(not done)=[] 7088 INFO 2021-04-28 22:24:47,315 async_flush.h:70 async flush executor is started! 7089 INFO 2021-04-28 22:24:47,641 table.cc:344 Add item _id [514019461459], num [0] 7090 INFO 2021-04-28 22:24:58,140 table.cc:344 Add item _id [514065729803], num [10000] 7091 INFO 2021-04-28 22:25:08,645 table.cc:344 Add item _id [12874589929], num [20000] 7092 INFO 2021-04-28 22:25:19,150 table.cc:344 Add item _id [12883886784], num [30000] 7093 INFO 2021-04-28 22:25:29,653 table.cc:344 Add item _id [12874586160], num [40000] 7094 INFO 2021-04-28 22:25:40,638 table.cc:344 Add item _id [514055953533], num [50000]

[realdalabengba]

目前集群两种部署方式： 1、master/router/ps； 2、router/ps/自建etcd。 第二种集群部署方式，change_member仍然无效， 第一种可以，但复制完无法启动，报错如下，重启后正常：

4791 2021-04-30 16:40:49,871 :0: DEBUG: raft[32] random election timeout randElectionTick=8, config.ElectionTick=5, randTick=3 4792 2021-04-30 16:40:49,871 :0: DEBUG: raft[32] random election timeout randElectionTick=8, config.ElectionTick=5, randTick=3 4793 2021-04-30 16:40:49,871 :0: DEBUG: raft[32] became candidate at term 8. 4794 2021-04-30 16:40:49,871 :0: DEBUG: raft[32] became candidate at term 8. 4795 2021-04-30 16:40:49,871 :0: DEBUG: raft[32] received vote from 8 at term 8. 4796 2021-04-30 16:40:49,871 :0: DEBUG: raft[32] received vote from 8 at term 8. 4797 2021-04-30 16:40:49,871 :0: DEBUG: [raft->campaign][32 logterm: 7, index: 5134910] sent vote request to 5 at term 8. raftFSM[0xc0003d2000] 4798 2021-04-30 16:40:49,871 :0: DEBUG: [raft->campaign][32 logterm: 7, index: 5134910] sent vote request to 5 at term 8. raftFSM[0xc0003d2000] 4799 2021-04-30 16:40:49,871 :0: DEBUG: [raft->campaign][32 logterm: 7, index: 5134910] sent vote request to 6 at term 8. raftFSM[0xc0003d2000] 4800 2021-04-30 16:40:49,871 :0: DEBUG: [raft->campaign][32 logterm: 7, index: 5134910] sent vote request to 6 at term 8. raftFSM[0xc0003d2000] 4801 2021-04-30 16:40:49,871 :0: DEBUG: [raft->campaign][32 logterm: 7, index: 5134910] sent vote request to 7 at term 8. raftFSM[0xc0003d2000] 4802 2021-04-30 16:40:49,871 :0: DEBUG: [raft->campaign][32 logterm: 7, index: 5134910] sent vote request to 7 at term 8. raftFSM[0xc0003d2000] 4803 2021-04-30 16:40:53,160 :0: DEBUG: [raft->Step][32 term: 8] received a [ReqMsgVote] message with higher term from [5 term: 9]. 4804 2021-04-30 16:40:53,160 :0: DEBUG: [raft->Step][32 term: 8] received a [ReqMsgVote] message with higher term from [5 term: 9]. 4805 2021-04-30 16:40:53,160 :0: DEBUG: raft[32] random election timeout randElectionTick=7, config.ElectionTick=5, randTick=2 4806 2021-04-30 16:40:53,160 :0: DEBUG: raft[32] random election timeout randElectionTick=7, config.ElectionTick=5, randTick=2 4807 2021-04-30 16:40:53,160 :0: DEBUG: [raft][32] became follower at term[9] leader[0]. 4808 2021-04-30 16:40:53,160 :0: DEBUG: [raft][32] became follower at term[9] leader[0]. 4809 2021-04-30 16:40:53,160 :0: DEBUG: raft[32] [logterm: 7, index: 5134910, vote: 0] voted for 5 [logterm: 7, index: 5134910] at term 9. 4810 2021-04-30 16:40:53,160 :0: DEBUG: raft[32] [logterm: 7, index: 5134910, vote: 0] voted for 5 [logterm: 7, index: 5134910] at term 9. 4811 fatal error: unexpected signal during runtime execution 4812 [signal SIGSEGV: segmentation violation code=0x1 addr=0x7f8840000000 pc=0x7f899d426b0c] 4813 4814 runtime stack: 4815 runtime.throw(0x165bffc, 0x2a) 4816 /tmp/github.com/vearch/vearch/build/third/go/src/runtime/panic.go:1116 +0x72 4817 runtime.sigpanic() 4818 /tmp/github.com/vearch/vearch/build/third/go/src/runtime/signal_unix.go:726 +0x4ac 4819 4820 goroutine 295 [syscall]: 4821 runtime.cgocall(0x12fe850, 0xc0085eb2b8, 0xc008232150) 4822 /tmp/github.com/vearch/vearch/build/third/go/src/runtime/cgocall.go:133 +0x5b fp=0xc0085eb288 sp=0xc0085eb250 pc=0x40607b 4823 github.com/vearch/vearch/ps/engine/gamma._Cfunc_Search(0x7f8964005760, 0xc0085f06f0, 0x910, 0xc008232150, 0xc007dae3f8, 0xc000000000) 4824 _cgo_gotypes.go:280 +0x4d fp=0xc0085eb2b8 sp=0xc0085eb288 pc=0x1101f2d 4825 github.com/vearch/vearch/ps/engine/gamma.Search.func1(0x7f8964005760, 0xc0085f06f0, 0x910, 0xc10, 0xc008232150, 0xc007dae3f8, 0x608bc299) 4826 /tmp/github.com/vearch/vearch/ps/engine/gamma/gamma.go:127 +0xba fp=0xc0085eb2f8 sp=0xc0085eb2b8 pc=0x1103b5a 4827 github.com/vearch/vearch/ps/engine/gamma.Search(0x7f8964005760, 0xc0085f06f0, 0x910, 0xc10, 0x0, 0x0, 0x0, 0x0) 4828 /tmp/github.com/vearch/vearch/ps/engine/gamma/gamma.go:127 +0xca fp=0xc0085eb368 sp=0xc0085eb2f8 pc=0x1102aea 4829 github.com/vearch/vearch/ps/engine/gammacb.(readerImpl).Search(0xc00041e058, 0x18110c0, 0xc0003de080, 0xc0085e0000, 0xc000c00300, 0x0, 0x0) 4830 /tmp/github.com/vearch/vearch/ps/engine/gammacb/reader.go:147 +0x1ad fp=0xc0085eb4f8 sp=0xc0085eb368 pc=0x129baed 4831 github.com/vearch/vearch/ps/storage/raftstore.(Store).Search(0xc0003e41e0, 0x18110c0, 0xc0003de080, 0xc0085e0000, 0xc000c00300, 0x52000000010ecb9c, 0x142fee0) 4832 /tmp/github.com/vearch/vearch/ps/storage/raftstore/store_read.go:82 +0x110 fp=0xc0085eb550 sp=0xc0085eb4f8 pc=0x12910d0 4833 github.com/vearch/vearch/ps.search(0x18110c0, 0xc0003de080, 0x182ed40, 0xc0003e41e0, 0xc0085e0000, 0xc000c00300) 4834 /tmp/github.com/vearch/vearch/ps/handler_document.go:265 +0x79 fp=0xc0085eb5c8 sp=0xc0085eb550 pc=0x12a50b9 4835 github.com/vearch/vearch/ps.(UnaryHandler).Execute(0xc00019a518, 0x18110c0, 0xc0003de080, 0xc008538900, 0xc0085389c0, 0x0, 0x0) 4836 /tmp/github.com/vearch/vearch/ps/handler_document.go:137 +0xb7c fp=0xc0085eb6d8 sp=0xc0085eb5c8 pc=0x12a3f1c 4837 github.com/vearch/vearch/util/server/rpc/handler.(Chain).Execute(0xc0080cc040, 0x18110c0, 0xc0003de080, 0xc008538900, 0xc0085389c0, 0x0, 0x0) 4838 /tmp/github.com/vearch/vearch/util/server/rpc/handler/chain.go:63 +0x11b fp=0xc0085eb768 sp=0xc0085eb6d8 pc=0x11d81bb 4839 runtime.call64(0xc008200180, 0xc00019a538, 0xc008584700, 0x2800000038)

[kuailelijuan]

您好，最新代码修复了一些问题，请您用master 最新的代码再进行测试。