The metaserver process consumes a lot of memory

[YunhuiChen]

Describe the bug (描述bug) The metaserver process consumes a lot of memory

To Reproduce (复现方法) [zone] zoneId:1, zoneName:zone1, poolId:1 serverList:{ 1 4 } zoneId:2, zoneName:zone2, poolId:1 serverList:{ 5 2 } zoneId:3, zoneName:zone3, poolId:1 serverList:{ 6 3 } [server] serverId:1, hostname:10.166.24.58_0_0, internalIp:10.166.24.58, internalPort:6710, externalIp:10.166.16.58, externalPort:7800, zoneId:1, poolId:1 metaserverList:{ 3 } serverId:2, hostname:10.166.24.59_1_0, internalIp:10.166.24.59, internalPort:6711, externalIp:10.166.16.59, externalPort:7801, zoneId:2, poolId:1 metaserverList:{ 5 } serverId:3, hostname:10.166.24.35_2_0, internalIp:10.166.24.35, internalPort:6712, externalIp:10.166.16.35, externalPort:7802, zoneId:3, poolId:1 metaserverList:{ 2 } serverId:4, hostname:10.166.24.58_3_0, internalIp:10.166.24.58, internalPort:6713, externalIp:10.166.16.58, externalPort:7803, zoneId:1, poolId:1 metaserverList:{ 4 } serverId:5, hostname:10.166.24.59_4_0, internalIp:10.166.24.59, internalPort:6714, externalIp:10.166.16.59, externalPort:7804, zoneId:2, poolId:1 metaserverList:{ 6 } serverId:6, hostname:10.166.24.35_5_0, internalIp:10.166.24.35, internalPort:6715, externalIp:10.166.16.35, externalPort:7805, zoneId:3, poolId:1 metaserverList:{ 1 }

1、test vdbench and mdtest 2、stop one metaserver metaserverId:3 3、

Expected behavior (期望行为)

Versions (各种版本) OS: Compiler: branch: commit id:

Additional context/screenshots (更多上下文/截图)

[Wine93]

Now the max_write_buffer_size_to_maintain is 192MB，the WriteBufferManager doesn't have the capability to trim the history, so the flushed memtable will not be deleted from memory, it leads the memory grows with copysets, to solve this problem, we can reducing the max_write_buffer_size_to_maintain to 32MB.

• jeprof

[Wine93]

For test, i deploy a cluster and write by mdtest, until now the cluster running abount 22 hours, and creates about 1.2E files, all metaserver's memory size is stable.

• configure

  • 2GB write buffer
  • 2GB cache

• mdtest (create 120,050,000 files)

• metaserver's memory

• cluster status